multiple definition of symbols" when linking executables on ARM32 and AArch64

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple definition of symbols" when linking executables on ARM32 and AArch64

Matthias Klose-6
In an archive test rebuild with binutils and GCC trunk, I see a lot of build
failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
"multiple definition of symbols" when linking executables, e.g.

https://launchpadlibrarian.net/459351186/buildlog_ubuntu-focal-armhf.tftp-hpa_5.2+20150808-1ubuntu4_BUILDING.txt.gz

arm-linux-gnueabihf-gcc  tftp.o main.o ../common/libcommon.a
/<<BUILDDIR>>/tftp-hpa-5.2+20150808/lib/libxtra.a  -o tftp
/usr/bin/ld: main.o:/<<BUILDDIR>>/tftp-hpa-5.2+20150808/tftp/main.c:98: multiple
definition of `toplevel';
tftp.o:/<<BUILDDIR>>/tftp-hpa-5.2+20150808/tftp/tftp.c:51: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:12: tftp] Error 1

https://launchpadlibrarian.net/459253363/buildlog_ubuntu-focal-arm64.bsd-mailx_8.1.2-0.20180807cvs-1_BUILDING.txt.gz

gcc -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -o mail version.o aux.o
cmd1.o cmd2.o cmd3.o cmdtab.o collect.o edit.o fio.o getname.o head.o v7.local.o
lex.o list.o main.o names.o popen.o quit.o send.o strings.o temp.o tty.o vars.o
-llockfile -lbsd
/usr/bin/ld: cmd1.o:./glob.h:77: multiple definition of `screenheight';
aux.o:./glob.h:77: first defined here
/usr/bin/ld: cmd1.o:./glob.h:66: multiple definition of `message';
aux.o:./glob.h:66: first defined here
/usr/bin/ld: cmd1.o:./glob.h:65: multiple definition of `dot';
aux.o:./glob.h:65: first defined here
/usr/bin/ld: cmd1.o:./glob.h:57: multiple definition of `myname';
aux.o:./glob.h:57: first defined here
/usr/bin/ld: cmd1.o:./glob.h:74: multiple definition of `altnames';
aux.o:./glob.h:74: first defined here

The compiler defaults to the usual hardening settings, PIE, stack-protector,
stack-collision, etc.  The failures are not seen on x86, powerpc64le, s390x.

Matthias
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Andrew Pinski-3
+GCC

On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>
> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
> "multiple definition of symbols" when linking executables, e.g.

THIS IS NOT A BINUTILS OR GCC BUG.
GCC changed the default to -fno-common.
It seems like for some reason, your non-aarch64/arm builds had changed
the default back to being with -fcommon turned on.
The error message is clear too that there packages out there that
depends on common symbols in C.

Thanks,
Andrew Pinski

>
> https://launchpadlibrarian.net/459351186/buildlog_ubuntu-focal-armhf.tftp-hpa_5.2+20150808-1ubuntu4_BUILDING.txt.gz
>
> arm-linux-gnueabihf-gcc  tftp.o main.o ../common/libcommon.a
> /<<BUILDDIR>>/tftp-hpa-5.2+20150808/lib/libxtra.a  -o tftp
> /usr/bin/ld: main.o:/<<BUILDDIR>>/tftp-hpa-5.2+20150808/tftp/main.c:98: multiple
> definition of `toplevel';
> tftp.o:/<<BUILDDIR>>/tftp-hpa-5.2+20150808/tftp/tftp.c:51: first defined here
> collect2: error: ld returned 1 exit status
> make[2]: *** [Makefile:12: tftp] Error 1
>
> https://launchpadlibrarian.net/459253363/buildlog_ubuntu-focal-arm64.bsd-mailx_8.1.2-0.20180807cvs-1_BUILDING.txt.gz
>
> gcc -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -o mail version.o aux.o
> cmd1.o cmd2.o cmd3.o cmdtab.o collect.o edit.o fio.o getname.o head.o v7.local.o
> lex.o list.o main.o names.o popen.o quit.o send.o strings.o temp.o tty.o vars.o
> -llockfile -lbsd
> /usr/bin/ld: cmd1.o:./glob.h:77: multiple definition of `screenheight';
> aux.o:./glob.h:77: first defined here
> /usr/bin/ld: cmd1.o:./glob.h:66: multiple definition of `message';
> aux.o:./glob.h:66: first defined here
> /usr/bin/ld: cmd1.o:./glob.h:65: multiple definition of `dot';
> aux.o:./glob.h:65: first defined here
> /usr/bin/ld: cmd1.o:./glob.h:57: multiple definition of `myname';
> aux.o:./glob.h:57: first defined here
> /usr/bin/ld: cmd1.o:./glob.h:74: multiple definition of `altnames';
> aux.o:./glob.h:74: first defined here
>
> The compiler defaults to the usual hardening settings, PIE, stack-protector,
> stack-collision, etc.  The failures are not seen on x86, powerpc64le, s390x.
>
> Matthias
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Szabolcs Nagy-2
In reply to this post by Matthias Klose-6
On 06/01/2020 09:51, Matthias Klose wrote:
> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
> "multiple definition of symbols" when linking executables, e.g.

gcc trunk made -fno-common the default which can cause
such failures i think (e.g. if tentative definitions
appear in header files it will create multiple definitions
everywhere where the header is included)

try if CC='gcc -fcommon' fixes the build.
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Matthias Klose-6
In reply to this post by Andrew Pinski-3
On 06.01.20 11:03, Andrew Pinski wrote:

> +GCC
>
> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>>
>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
>> "multiple definition of symbols" when linking executables, e.g.
>
> THIS IS NOT A BINUTILS OR GCC BUG.
> GCC changed the default to -fno-common.
> It seems like for some reason, your non-aarch64/arm builds had changed
> the default back to being with -fcommon turned on.

what would that be?  I'm not aware of any active change doing that.  Packages
build on x86, ppc64el and s390x at least.
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Wilco Dijkstra-2
In reply to this post by Matthias Klose-6
On 06.01.20 11:03, Andrew Pinski wrote:

> +GCC
>
> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>>
>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
>> "multiple definition of symbols" when linking executables, e.g.
>
> THIS IS NOT A BINUTILS OR GCC BUG.
> GCC changed the default to -fno-common.
> It seems like for some reason, your non-aarch64/arm builds had changed
> the default back to being with -fcommon turned on.

> what would that be?  I'm not aware of any active change doing that.  Packages
> build on x86, ppc64el and s390x at least.

Well if you want to build old archived code using latest GCC then you may need to
force -fcommon just like you need to add many warning disables. Maybe you were
using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.

Wilco
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Matthias Klose-6
On 06.01.20 13:30, Wilco Dijkstra wrote:

> On 06.01.20 11:03, Andrew Pinski wrote:
>> +GCC
>>
>> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>>>
>>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
>>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
>>> "multiple definition of symbols" when linking executables, e.g.
>>
>> THIS IS NOT A BINUTILS OR GCC BUG.
>> GCC changed the default to -fno-common.
>> It seems like for some reason, your non-aarch64/arm builds had changed
>> the default back to being with -fcommon turned on.
>
>> what would that be?  I'm not aware of any active change doing that.  Packages
>> build on x86, ppc64el and s390x at least.
>
> Well if you want to build old archived code using latest GCC then you may need to
> force -fcommon just like you need to add many warning disables. Maybe you were
> using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.

found out about why. Started the test rebuild with trunk 20191219, then gave
back all build failures yesterday with trunk 20200104.  And I saw most of the
armhf/arm64 ftbfs when I retriggered failing builds.  To get consistent results
I should finish that test rebuild with the -fno-common change reverted.

However, this is an undocumented change in the current NEWS, and seeing
literally hundreds of package failures, I doubt that's the right thing to do, at
least without any deprecation warning first.  Could that be handled, deprecating
in GCC 10 first, and the changing that for GCC 11?

Matthias
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Matthias Klose-6
On 06.01.20 14:29, Matthias Klose wrote:

> On 06.01.20 13:30, Wilco Dijkstra wrote:
>> On 06.01.20 11:03, Andrew Pinski wrote:
>>> +GCC
>>>
>>> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>>>>
>>>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
>>>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
>>>> "multiple definition of symbols" when linking executables, e.g.
>>>
>>> THIS IS NOT A BINUTILS OR GCC BUG.
>>> GCC changed the default to -fno-common.
>>> It seems like for some reason, your non-aarch64/arm builds had changed
>>> the default back to being with -fcommon turned on.
>>
>>> what would that be?  I'm not aware of any active change doing that.  Packages
>>> build on x86, ppc64el and s390x at least.
>>
>> Well if you want to build old archived code using latest GCC then you may need to
>> force -fcommon just like you need to add many warning disables. Maybe you were
>> using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.
>
> found out about why. Started the test rebuild with trunk 20191219, then gave
> back all build failures yesterday with trunk 20200104.

hmm, no. that change was made on November 20, not December 20 (r278509). So why
do I see these only on ARM32 and AArch64?

> And I saw most of the
> armhf/arm64 ftbfs when I retriggered failing builds.  To get consistent results
> I should finish that test rebuild with the -fno-common change reverted.
>
> However, this is an undocumented change in the current NEWS, and seeing
> literally hundreds of package failures, I doubt that's the right thing to do, at
> least without any deprecation warning first.  Could that be handled, deprecating
> in GCC 10 first, and the changing that for GCC 11?
>
> Matthias
>

Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Martin Liška
In reply to this post by Matthias Klose-6
On 1/6/20 2:29 PM, Matthias Klose wrote:

> On 06.01.20 13:30, Wilco Dijkstra wrote:
>> On 06.01.20 11:03, Andrew Pinski wrote:
>>> +GCC
>>>
>>> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
>>>>
>>>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
>>>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
>>>> "multiple definition of symbols" when linking executables, e.g.
>>>
>>> THIS IS NOT A BINUTILS OR GCC BUG.
>>> GCC changed the default to -fno-common.
>>> It seems like for some reason, your non-aarch64/arm builds had changed
>>> the default back to being with -fcommon turned on.
>>
>>> what would that be?  I'm not aware of any active change doing that.  Packages
>>> build on x86, ppc64el and s390x at least.
>>
>> Well if you want to build old archived code using latest GCC then you may need to
>> force -fcommon just like you need to add many warning disables. Maybe you were
>> using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.
>
> found out about why. Started the test rebuild with trunk 20191219, then gave
> back all build failures yesterday with trunk 20200104.  And I saw most of the
> armhf/arm64 ftbfs when I retriggered failing builds.  To get consistent results
> I should finish that test rebuild with the -fno-common change reverted.
>
> However, this is an undocumented change in the current NEWS, and seeing

Hello.

It's waiting for a review:
https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00311.html

> literally hundreds of package failures, I doubt that's the right thing to do, at
> least without any deprecation warning first.  Could that be handled, deprecating
> in GCC 10 first, and the changing that for GCC 11?

I can confirm that it will cause a significant fallout. We're expecting with
Jeff something like:
https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00333.html

Our strategy (as openSUSE) would be to use GCC default (-fno-common) and let
package maintainers to report issues and possible add -fcommon to $optflags
on package basis.

Martin

>
> Matthias
>

Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Andrew Pinski-3
In reply to this post by Matthias Klose-6
On Mon, Jan 6, 2020 at 5:29 AM Matthias Klose <[hidden email]> wrote:

>
> On 06.01.20 13:30, Wilco Dijkstra wrote:
> > On 06.01.20 11:03, Andrew Pinski wrote:
> >> +GCC
> >>
> >> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
> >>>
> >>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
> >>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
> >>> "multiple definition of symbols" when linking executables, e.g.
> >>
> >> THIS IS NOT A BINUTILS OR GCC BUG.
> >> GCC changed the default to -fno-common.
> >> It seems like for some reason, your non-aarch64/arm builds had changed
> >> the default back to being with -fcommon turned on.
> >
> >> what would that be?  I'm not aware of any active change doing that.  Packages
> >> build on x86, ppc64el and s390x at least.
> >
> > Well if you want to build old archived code using latest GCC then you may need to
> > force -fcommon just like you need to add many warning disables. Maybe you were
> > using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.
>
> found out about why. Started the test rebuild with trunk 20191219, then gave
> back all build failures yesterday with trunk 20200104.  And I saw most of the
> armhf/arm64 ftbfs when I retriggered failing builds.  To get consistent results
> I should finish that test rebuild with the -fno-common change reverted.
>
> However, this is an undocumented change in the current NEWS, and seeing
> literally hundreds of package failures, I doubt that's the right thing to do, at
> least without any deprecation warning first.  Could that be handled, deprecating
> in GCC 10 first, and the changing that for GCC 11?

It is hard to get a warning for things like this.
Also the documentation for -fcommon was changed and made clear for
8.1.0[1] that the default might be changing because it is not required
by ISO C.
"This is the behavior specified by -fcommon, and is the default for
GCC on most targets. On the other hand, this behavior is not required
by ISO C, ..."
-fno-common: "This inhibits the merging of tentative definitions by
the linker so you get a multiple-definition error if the same variable
is defined in more than one compilation unit. "
So I think it was "deprecated" in GCC for the last 2 major releases.
Though nobody reads the manual these days.

Thanks,
Andrew Pinski

[1] https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Code-Gen-Options.html#index-fno-common

>
> Matthias
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Wilco Dijkstra-2





Hi,

> However, this is an undocumented change in the current NEWS, and seeing
>> literally hundreds of package failures, I doubt that's the right thing to do, at
>> least without any deprecation warning first.  Could that be handled, deprecating
>> in GCC 10 first, and the changing that for GCC 11?

This change was first proposed for GCC8, and rejected because of failures in the
distros. Two years have passed, and there are still failures... Would this change if
we postpone it even longer? My feeling is that nobody is going to actively fix their
code if the default isn't changed first.

> It is hard to get a warning for things like this.

Could the linker warn whenever it merges common symbols or would that give
many false positives?

Wilco
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Andrew Pinski-3
On Mon, Jan 6, 2020 at 6:02 AM Wilco Dijkstra <[hidden email]> wrote:

>
>
> Hi,
>
> > However, this is an undocumented change in the current NEWS, and seeing
> >> literally hundreds of package failures, I doubt that's the right thing to do, at
> >> least without any deprecation warning first.  Could that be handled, deprecating
> >> in GCC 10 first, and the changing that for GCC 11?
>
> This change was first proposed for GCC8, and rejected because of failures in the
> distros. Two years have passed, and there are still failures... Would this change if
> we postpone it even longer? My feeling is that nobody is going to actively fix their
> code if the default isn't changed first.
>
> > It is hard to get a warning for things like this.
>
> Could the linker warn whenever it merges common symbols or would that give
> many false positives?

That might get some false positives for some Fortran code (where
common symbols are "common") (and maybe C++ code; C++ sometimes uses
common symbols but only if weak support does not exist).

Thanks,
Andrew Pinski


>
> Wilco
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Jonathan Wakely-4
In reply to this post by Martin Liška
On Mon, 6 Jan 2020 at 13:37, Martin Liška <[hidden email]> wrote:

>
> On 1/6/20 2:29 PM, Matthias Klose wrote:
> > On 06.01.20 13:30, Wilco Dijkstra wrote:
> >> On 06.01.20 11:03, Andrew Pinski wrote:
> >>> +GCC
> >>>
> >>> On Mon, Jan 6, 2020 at 1:52 AM Matthias Klose <[hidden email]> wrote:
> >>>>
> >>>> In an archive test rebuild with binutils and GCC trunk, I see a lot of build
> >>>> failures on both aarch64-linux-gnu and arm-linux-gnueabihf failing with
> >>>> "multiple definition of symbols" when linking executables, e.g.
> >>>
> >>> THIS IS NOT A BINUTILS OR GCC BUG.
> >>> GCC changed the default to -fno-common.
> >>> It seems like for some reason, your non-aarch64/arm builds had changed
> >>> the default back to being with -fcommon turned on.
> >>
> >>> what would that be?  I'm not aware of any active change doing that.  Packages
> >>> build on x86, ppc64el and s390x at least.
> >>
> >> Well if you want to build old archived code using latest GCC then you may need to
> >> force -fcommon just like you need to add many warning disables. Maybe you were
> >> using an older GCC for the other targets? As Andrew notes, this isn't Arm-specific.
> >
> > found out about why. Started the test rebuild with trunk 20191219, then gave
> > back all build failures yesterday with trunk 20200104.  And I saw most of the
> > armhf/arm64 ftbfs when I retriggered failing builds.  To get consistent results
> > I should finish that test rebuild with the -fno-common change reverted.
> >
> > However, this is an undocumented change in the current NEWS, and seeing
>
> Hello.
>
> It's waiting for a review:
> https://gcc.gnu.org/ml/gcc-patches/2019-12/msg00311.html

Does that need review?

The change is on trunk, documenting it in the release notes needs to be done.

But since it is waiting for review, I decided to reply with some comments :-)
Reply | Threaded
Open this post in threaded view
|

Re: multiple definition of symbols" when linking executables on ARM32 and AArch64

Matthias Klose-6
In reply to this post by Wilco Dijkstra-2
On 06.01.20 15:02, Wilco Dijkstra wrote:
>> However, this is an undocumented change in the current NEWS, and seeing
>>> literally hundreds of package failures, I doubt that's the right thing to do, at
>>> least without any deprecation warning first.  Could that be handled, deprecating
>>> in GCC 10 first, and the changing that for GCC 11?
>
> This change was first proposed for GCC8, and rejected because of failures in the
> distros. Two years have passed, and there are still failures... Would this change if
> we postpone it even longer? My feeling is that nobody is going to actively fix their
> code if the default isn't changed first.

I think we were missing a proper announcement for GCC 8, and maybe we were not
aware of the amount of code changes needed in third party packages.  So maybe we
should have had in GCC 8 a configure option to make that the default behavior,
announce it in the release notes, ask for testing using that option, and do the
defaults change in GCC 9.  We could do that for GCC 10/11, or just bite in the
apple and now fix all packages builds.

Matthias