binutils ld and new PT_GNU_PROPERTY segment

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
Hi,

binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
segment that overlaps with the PT_NOTE segment covering the
.note.gnu.property section data.

I cannot tell if this is an accident/experiment that happened to end up
in a release or if it is proposed as an official new segment type.

As far as I can tell binutils ld is the only linker which adds this
extra segment and there is no runtime loader which uses it. It doesn't
provide any new information that cannot be found in the existing
PT_NOTE segment.

On 64 bit architectures it simply covers the extra existing PT_NOTE
with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On
32bit architectures it covers a sub-range of the existing PT_NOTE
segment.

It isn't clear to me how other tools should handle this. It seems to
prevent normal merging of note sections. Since some notes are probably
special if they need to be covered by this new segment type. And it
isn't clear how the linker knows which of the SHT_NOTE sections is what
needs to be covered by the new segment type. Or is the idea that this
will eventually come with a new section type too and GNU properties
will no longer use NOTE sections?

Thanks,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Tue, Feb 18, 2020 at 4:38 AM Mark Wielaard <[hidden email]> wrote:
>
> Hi,
>
> binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
> segment that overlaps with the PT_NOTE segment covering the
> .note.gnu.property section data.
>
> I cannot tell if this is an accident/experiment that happened to end up
> in a release or if it is proposed as an official new segment type.

https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html
https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI

> As far as I can tell binutils ld is the only linker which adds this

Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not,
it is an lld bug.

> extra segment and there is no runtime loader which uses it. It doesn't
> provide any new information that cannot be found in the existing
> PT_NOTE segment.

Kernel loader uses it for both ARM and x86.

> On 64 bit architectures it simply covers the extra existing PT_NOTE
> with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On
> 32bit architectures it covers a sub-range of the existing PT_NOTE
> segment.
>
> It isn't clear to me how other tools should handle this. It seems to
> prevent normal merging of note sections. Since some notes are probably
> special if they need to be covered by this new segment type. And it
> isn't clear how the linker knows which of the SHT_NOTE sections is what
> needs to be covered by the new segment type. Or is the idea that this
> will eventually come with a new section type too and GNU properties
> will no longer use NOTE sections?
>

PT_GNU_PROPERTY covers .note.gnu.property section.


--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Sourceware - binutils list mailing list
On 2020-02-18, H.J. Lu wrote:

>On Tue, Feb 18, 2020@4:38 AM Mark Wielaard <[hidden email]> wrote:
>>
>> Hi,
>>
>> binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
>> segment that overlaps with the PT_NOTE segment covering the
>> .note.gnu.property section data.
>>
>> I cannot tell if this is an accident/experiment that happened to end up
>> in a release or if it is proposed as an official new segment type.
>
>https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html
>https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
>
>> As far as I can tell binutils ld is the only linker which adds this
>
>Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not,
>it is an lld bug.
>
>> extra segment and there is no runtime loader which uses it. It doesn't
>> provide any new information that cannot be found in the existing
>> PT_NOTE segment.
>
>Kernel loader uses it for both ARM and x86.
>
>> On 64 bit architectures it simply covers the extra existing PT_NOTE
>> with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On
>> 32bit architectures it covers a sub-range of the existing PT_NOTE
>> segment.
>>
>> It isn't clear to me how other tools should handle this. It seems to
>> prevent normal merging of note sections. Since some notes are probably
>> special if they need to be covered by this new segment type. And it
>> isn't clear how the linker knows which of the SHT_NOTE sections is what
>> needs to be covered by the new segment type. Or is the idea that this
>> will eventually come with a new section type too and GNU properties
>> will no longer use NOTE sections?
>>
>
>PT_GNU_PROPERTY covers .note.gnu.property section.

https://reviews.llvm.org/D70961 added PT_GNU_PROPERTY support to lld.
The change will be included by lld 10.0.0 (currently at rc2).

https://reviews.llvm.org/D70959 added the dump support to llvm-readelf -l and
llvm-objdump -p.
The change will be included by LLVM 10.0.0 (currently at rc2).

 From what I can see, neither the Linux kernel nor glibc uses PT_GNU_PROPERTY.
glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.

I tend to agree with Cary
(https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that
.note.gnu.property should have been designed as a different section type
because its combining semantics are different from other notes
(we could apply "Rules for Linking Unrecognized Sections" to all
SHT_NOTE sections) but it is too late to change the section type.

A separate segment type (PT_GNU_PROPERTY) looks fine to me.
glibc should probably be updated to parse PT_GNU_PROPERTY instead.

(Recently I read some ABI decisions and I noticed that I frequently see
the term "it is too late". As a contributor of both lld and LLVM binary
utilities (and the implementer of a bunch of GNU_PROPERTY changes), I
hope that the LLVM community can be informed of such changes earlier.  A
lot of people are not subscribed to any of the mailing lists (recently I
visit the archives from time to time).

Looks like Annita's job? :) )
Reply | Threaded
Open this post in threaded view
|

RE: binutils ld and new PT_GNU_PROPERTY segment

Zhang, Annita
Regarding x86-64 ABI, we have internal communication if there's some update. And we will raise it up in LLVM community if necessary.

Thanks,
Annita

-----Original Message-----
From: Fangrui Song <[hidden email]>
Sent: Wednesday, February 19, 2020 10:31 AM
To: H.J. Lu <[hidden email]>; Zhang, Annita <[hidden email]>
Cc: Mark Wielaard <[hidden email]>; gnu-gabi <[hidden email]>; Binutils <[hidden email]>
Subject: Re: binutils ld and new PT_GNU_PROPERTY segment

On 2020-02-18, H.J. Lu wrote:

>On Tue, Feb 18, 2020@4:38 AM Mark Wielaard <[hidden email]> wrote:
>>
>> Hi,
>>
>> binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
>> segment that overlaps with the PT_NOTE segment covering the
>> .note.gnu.property section data.
>>
>> I cannot tell if this is an accident/experiment that happened to end
>> up in a release or if it is proposed as an official new segment type.
>
>https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html
>https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
>
>> As far as I can tell binutils ld is the only linker which adds this
>
>Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not, it
>is an lld bug.
>
>> extra segment and there is no runtime loader which uses it. It
>> doesn't provide any new information that cannot be found in the
>> existing PT_NOTE segment.
>
>Kernel loader uses it for both ARM and x86.
>
>> On 64 bit architectures it simply covers the extra existing PT_NOTE
>> with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned).
>> On 32bit architectures it covers a sub-range of the existing PT_NOTE
>> segment.
>>
>> It isn't clear to me how other tools should handle this. It seems to
>> prevent normal merging of note sections. Since some notes are
>> probably special if they need to be covered by this new segment type.
>> And it isn't clear how the linker knows which of the SHT_NOTE
>> sections is what needs to be covered by the new segment type. Or is
>> the idea that this will eventually come with a new section type too
>> and GNU properties will no longer use NOTE sections?
>>
>
>PT_GNU_PROPERTY covers .note.gnu.property section.

https://reviews.llvm.org/D70961 added PT_GNU_PROPERTY support to lld.
The change will be included by lld 10.0.0 (currently at rc2).

https://reviews.llvm.org/D70959 added the dump support to llvm-readelf -l and llvm-objdump -p.
The change will be included by LLVM 10.0.0 (currently at rc2).

 From what I can see, neither the Linux kernel nor glibc uses PT_GNU_PROPERTY.
glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.

I tend to agree with Cary
(https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that .note.gnu.property should have been designed as a different section type because its combining semantics are different from other notes (we could apply "Rules for Linking Unrecognized Sections" to all SHT_NOTE sections) but it is too late to change the section type.

A separate segment type (PT_GNU_PROPERTY) looks fine to me.
glibc should probably be updated to parse PT_GNU_PROPERTY instead.

(Recently I read some ABI decisions and I noticed that I frequently see the term "it is too late". As a contributor of both lld and LLVM binary utilities (and the implementer of a bunch of GNU_PROPERTY changes), I hope that the LLVM community can be informed of such changes earlier.  A lot of people are not subscribed to any of the mailing lists (recently I visit the archives from time to time).

Looks like Annita's job? :) )
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
In reply to this post by H.J. Lu-30
Hi,

On Tue, 2020-02-18 at 04:48 -0800, H.J. Lu wrote:

> On Tue, Feb 18, 2020 at 4:38 AM Mark Wielaard <[hidden email]> wrote:
> > binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553)
> > segment that overlaps with the PT_NOTE segment covering the
> > .note.gnu.property section data.
> >
> > I cannot tell if this is an accident/experiment that happened to
> > end up
> > in a release or if it is proposed as an official new segment type.
>
> https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html
> https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI

I see you added it, but I don't see consensus for it. It also doesn't
really make sense if it still uses the same section type and has to
rely on a magic section name.

> > As far as I can tell binutils ld is the only linker which adds this
>
> Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not,
> it is an lld bug.

I think it is the other way around, binutils ld generates it, but
nobody uses it, it is redundant and the interaction with other tools
dealing with SHT_NOTE sections isn't really clear.

> > extra segment and there is no runtime loader which uses it. It
> > doesn't
> > provide any new information that cannot be found in the existing
> > PT_NOTE segment.
>
> Kernel loader uses it for both ARM and x86.

I cannot find it. Could you point to where it uses it?
Why doesn't it simply use the PT_NOTE segment?

> > On 64 bit architectures it simply covers the extra existing PT_NOTE
> > with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On
> > 32bit architectures it covers a sub-range of the existing PT_NOTE
> > segment.
> >
> > It isn't clear to me how other tools should handle this. It seems to
> > prevent normal merging of note sections. Since some notes are probably
> > special if they need to be covered by this new segment type. And it
> > isn't clear how the linker knows which of the SHT_NOTE sections is what
> > needs to be covered by the new segment type. Or is the idea that this
> > will eventually come with a new section type too and GNU properties
> > will no longer use NOTE sections?
> >
>
> PT_GNU_PROPERTY covers .note.gnu.property section.

Using magic section names is a problem. It means other tools can no
longer rely on the section type.

Can we please simply remove this because it isn't used, doesn't provide
any new information, depends on magic section names and makes it harder
for other tools to deal with generic SHT_NOTE sections.

Thanks,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
In reply to this post by Sourceware - binutils list mailing list
Hi,

On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote:
>  From what I can see, neither the Linux kernel nor glibc uses
> PT_GNU_PROPERTY.
> glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.

That is my impression too.

> I tend to agree with Cary
> (https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that
> .note.gnu.property should have been designed as a different section
> type
> because its combining semantics are different from other notes
> (we could apply "Rules for Linking Unrecognized Sections" to all
> SHT_NOTE sections) but it is too late to change the section type.

Agreed.

> A separate segment type (PT_GNU_PROPERTY) looks fine to me.
> glibc should probably be updated to parse PT_GNU_PROPERTY instead.

I think it is confusing to now introduce a new segment type which
basically provides the same information as the PT_NOTE segement. It is
ill defined because it needs to be matched to a magic section name.
Which makes things harder for tools dealing with generic SHT_NOTE
sections (they would have to preserve the magic section name, might not
be able to merge notes, etc.)

> (Recently I read some ABI decisions and I noticed that I frequently see
> the term "it is too late". As a contributor of both lld and LLVM binary
> utilities (and the implementer of a bunch of GNU_PROPERTY changes), I
> hope that the LLVM community can be informed of such changes earlier.  A
> lot of people are not subscribed to any of the mailing lists (recently I
> visit the archives from time to time).

Yeah, we really need to do a better job coordinating and communicating.
Even though I am trying to keep up with these lists I am often
surprised by changes like this which seem to have just been added to
binutils without trying to get consensus with other GNU/ELF tool
implementations about the precise semantics or whether a feature is
simply redundant with existing practice.

Cheers,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <[hidden email]> wrote:
>
> Hi,
>
> On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote:
> >  From what I can see, neither the Linux kernel nor glibc uses
> > PT_GNU_PROPERTY.
> > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.
>
> That is my impression too.

See:

https://patchwork.kernel.org/patch/11285409/

It is for both x86 and arm64.

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
Hi,

On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote:

> On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <[hidden email]> wrote:
> > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote:
> > >  From what I can see, neither the Linux kernel nor glibc uses
> > > PT_GNU_PROPERTY.
> > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.
> >
> > That is my impression too.
>
> See:
>
> https://patchwork.kernel.org/patch/11285409/
>
> It is for both x86 and arm64.

So that is not upstream in the mainline kernel? Why can't that patch
use the existing PT_NOTE segment? That would make it compatible with
existing binaries that don't have this PT_GNU_PROPERTY program header.

Thanks,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:

>
> Hi,
>
> On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote:
> > On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <[hidden email]> wrote:
> > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote:
> > > >  From what I can see, neither the Linux kernel nor glibc uses
> > > > PT_GNU_PROPERTY.
> > > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.
> > >
> > > That is my impression too.
> >
> > See:
> >
> > https://patchwork.kernel.org/patch/11285409/
> >
> > It is for both x86 and arm64.
>
> So that is not upstream in the mainline kernel? Why can't that patch
> use the existing PT_NOTE segment? That would make it compatible with
> existing binaries that don't have this PT_GNU_PROPERTY program header.

Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
only wants to check PT_XXX.

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 4:28 AM H.J. Lu <[hidden email]> wrote:

>
> On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
> >
> > Hi,
> >
> > On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote:
> > > On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <[hidden email]> wrote:
> > > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote:
> > > > >  From what I can see, neither the Linux kernel nor glibc uses
> > > > > PT_GNU_PROPERTY.
> > > > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE.
> > > >
> > > > That is my impression too.
> > >
> > > See:
> > >
> > > https://patchwork.kernel.org/patch/11285409/
> > >
> > > It is for both x86 and arm64.
> >
> > So that is not upstream in the mainline kernel? Why can't that patch
> > use the existing PT_NOTE segment? That would make it compatible with
> > existing binaries that don't have this PT_GNU_PROPERTY program header.
>
> Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
> only wants to check PT_XXX.
>

This is one thread on kernel mailing list:

https://lore.kernel.org/lkml/87imt4jwpt.fsf@.../

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
In reply to this post by H.J. Lu-30
On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote:

> On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
> > > https://patchwork.kernel.org/patch/11285409/
> > >
> > > It is for both x86 and arm64.
> >
> > So that is not upstream in the mainline kernel? Why can't that patch
> > use the existing PT_NOTE segment? That would make it compatible with
> > existing binaries that don't have this PT_GNU_PROPERTY program header.
>
> Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
> only wants to check PT_XXX.

So they can check PT_NOTE because it provides the same information and
is already available in existing binaries.

Cheers,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <[hidden email]> wrote:

>
> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote:
> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
> > > > https://patchwork.kernel.org/patch/11285409/
> > > >
> > > > It is for both x86 and arm64.
> > >
> > > So that is not upstream in the mainline kernel? Why can't that patch
> > > use the existing PT_NOTE segment? That would make it compatible with
> > > existing binaries that don't have this PT_GNU_PROPERTY program header.
> >
> > Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
> > only wants to check PT_XXX.
>
> So they can check PT_NOTE because it provides the same information and
> is already available in existing binaries.
>

Please take a look at glibc note.gnu.property parser.  It is very complicated to
check for invalid .note.gnu.property sections generated by the older
linkers with
the new object.  Kernel loader doesn't want to do it.

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Fangrui Song-2
On 2020-02-19, H.J. Lu wrote:

>On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <[hidden email]> wrote:
>>
>> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote:
>> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
>> > > > https://patchwork.kernel.org/patch/11285409/
>> > > >
>> > > > It is for both x86 and arm64.
>> > >
>> > > So that is not upstream in the mainline kernel? Why can't that patch
>> > > use the existing PT_NOTE segment? That would make it compatible with
>> > > existing binaries that don't have this PT_GNU_PROPERTY program header.
>> >
>> > Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
>> > only wants to check PT_XXX.
>>
>> So they can check PT_NOTE because it provides the same information and
>> is already available in existing binaries.
>>
>
>Please take a look at glibc note.gnu.property parser.  It is very complicated to
>check for invalid .note.gnu.property sections generated by the older
>linkers with
>the new object.  Kernel loader doesn't want to do it.

One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html

* Define SHT_GNU_PROPERTY
* Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY
* Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment

The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc).
New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE.
   ( https://patchwork.kernel.org/patch/11285409/ needs no change)

This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections.

I will be happy to make changes to lld and LLVM binary utilities if this
scheme reaches consensus.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <[hidden email]> wrote:

>
> On 2020-02-19, H.J. Lu wrote:
> >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <[hidden email]> wrote:
> >>
> >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote:
> >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
> >> > > > https://patchwork.kernel.org/patch/11285409/
> >> > > >
> >> > > > It is for both x86 and arm64.
> >> > >
> >> > > So that is not upstream in the mainline kernel? Why can't that patch
> >> > > use the existing PT_NOTE segment? That would make it compatible with
> >> > > existing binaries that don't have this PT_GNU_PROPERTY program header.
> >> >
> >> > Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
> >> > only wants to check PT_XXX.
> >>
> >> So they can check PT_NOTE because it provides the same information and
> >> is already available in existing binaries.
> >>
> >
> >Please take a look at glibc note.gnu.property parser.  It is very complicated to
> >check for invalid .note.gnu.property sections generated by the older
> >linkers with
> >the new object.  Kernel loader doesn't want to do it.
>
> One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html
>
> * Define SHT_GNU_PROPERTY
> * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY
> * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment
>
> The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc).
> New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE.
>    ( https://patchwork.kernel.org/patch/11285409/ needs no change)
>
> This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections.
>
> I will be happy to make changes to lld and LLVM binary utilities if this
> scheme reaches consensus.

It is kind of too late now.

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
Hi,

On Wed, 2020-02-19 at 11:29 -0800, H.J. Lu wrote:

> On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <[hidden email]> wrote:
> > One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html
> >
> > * Define SHT_GNU_PROPERTY
> > * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY
> > * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment
> >
> > The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc).
> > New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE.
> >    ( https://patchwork.kernel.org/patch/11285409/ needs no change)
> >
> > This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections.
> >
> > I will be happy to make changes to lld and LLVM binary utilities if this
> > scheme reaches consensus.
>
> It is kind of too late now.

This code isn't in the kernel yet. So either it gets changed to use the
existing scheme with gnu property notes found through PT_NOTE to work
with existing binaries. Then there is no need for PT_GNU_PROPERTY
headers.

Or some future kernel will start using PT_GNU_PROPERTY headers to find
the gnu property notes. But that means it won't work with existing
binaries that do not have that header. So there is no backwards
compatibility anyway and we can define SHT_GNU_PROPERTY like above.

So this actually seems the perfect time to make this decision.

Cheers,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <[hidden email]> wrote:

>
> Hi,
>
> On Wed, 2020-02-19 at 11:29 -0800, H.J. Lu wrote:
> > On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <[hidden email]> wrote:
> > > One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html
> > >
> > > * Define SHT_GNU_PROPERTY
> > > * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY
> > > * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment
> > >
> > > The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc).
> > > New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE.
> > >    ( https://patchwork.kernel.org/patch/11285409/ needs no change)
> > >
> > > This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections.
> > >
> > > I will be happy to make changes to lld and LLVM binary utilities if this
> > > scheme reaches consensus.
> >
> > It is kind of too late now.
>
> This code isn't in the kernel yet. So either it gets changed to use the
> existing scheme with gnu property notes found through PT_NOTE to work
> with existing binaries. Then there is no need for PT_GNU_PROPERTY
> headers.
>
> Or some future kernel will start using PT_GNU_PROPERTY headers to find
> the gnu property notes. But that means it won't work with existing
> binaries that do not have that header. So there is no backwards
> compatibility anyway and we can define SHT_GNU_PROPERTY like above.
>
> So this actually seems the perfect time to make this decision.
>

Binaries with .note.gnu.property section have been put into many
OS releases.  We must support them.


--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
Hi,

On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote:

> On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <[hidden email]> wrote:
> > This code isn't in the kernel yet. So either it gets changed to use the
> > existing scheme with gnu property notes found through PT_NOTE to work
> > with existing binaries. Then there is no need for PT_GNU_PROPERTY
> > headers.
> >
> > Or some future kernel will start using PT_GNU_PROPERTY headers to find
> > the gnu property notes. But that means it won't work with existing
> > binaries that do not have that header. So there is no backwards
> > compatibility anyway and we can define SHT_GNU_PROPERTY like above.
> >
> > So this actually seems the perfect time to make this decision.
>
> Binaries with .note.gnu.property section have been put into many
> OS releases.  We must support them.

OK. Then it is option 1. The kernel will need to support PT_NOTE for
parsing the properties, since such older binaries won't have a
PT_GNU_PROPERTY program header. Then we can simply get rid of
PT_GNU_PROPERTY since nobody uses it and all information is already
available through the PT_NOTE segment.

Cheers,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

H.J. Lu-30
On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <[hidden email]> wrote:

>
> Hi,
>
> On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote:
> > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <[hidden email]> wrote:
> > > This code isn't in the kernel yet. So either it gets changed to use the
> > > existing scheme with gnu property notes found through PT_NOTE to work
> > > with existing binaries. Then there is no need for PT_GNU_PROPERTY
> > > headers.
> > >
> > > Or some future kernel will start using PT_GNU_PROPERTY headers to find
> > > the gnu property notes. But that means it won't work with existing
> > > binaries that do not have that header. So there is no backwards
> > > compatibility anyway and we can define SHT_GNU_PROPERTY like above.
> > >
> > > So this actually seems the perfect time to make this decision.
> >
> > Binaries with .note.gnu.property section have been put into many
> > OS releases.  We must support them.
>
> OK. Then it is option 1. The kernel will need to support PT_NOTE for
> parsing the properties, since such older binaries won't have a
> PT_GNU_PROPERTY program header. Then we can simply get rid of
> PT_GNU_PROPERTY since nobody uses it and all information is already
> available through the PT_NOTE segment.
>

Kernel loader only checks ld.so and static executable.  Re-link them with
newer linker will get PT_GNU_PROPERTY.  But ld.so needs to check
PT_NOTE for older binaries.

--
H.J.
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Fangrui Song-2
On 2020-02-19, H.J. Lu wrote:

>On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <[hidden email]> wrote:
>>
>> On 2020-02-19, H.J. Lu wrote:
>> >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <[hidden email]> wrote:
>> >>
>> >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote:
>> >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <[hidden email]> wrote:
>> >> > > > https://patchwork.kernel.org/patch/11285409/
>> >> > > >
>> >> > > > It is for both x86 and arm64.
>> >> > >
>> >> > > So that is not upstream in the mainline kernel? Why can't that patch
>> >> > > use the existing PT_NOTE segment? That would make it compatible with
>> >> > > existing binaries that don't have this PT_GNU_PROPERTY program header.
>> >> >
>> >> > Kernel loader is one of motivations of PT_GNU_PROPERTY.  Kernel loader
>> >> > only wants to check PT_XXX.
>> >>
>> >> So they can check PT_NOTE because it provides the same information and
>> >> is already available in existing binaries.
>> >>
>> >
>> >Please take a look at glibc note.gnu.property parser.  It is very complicated to
>> >check for invalid .note.gnu.property sections generated by the older
>> >linkers with
>> >the new object.  Kernel loader doesn't want to do it.
>>
>> One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html
>>
>> * Define SHT_GNU_PROPERTY
>> * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY
>> * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment
>>
>> The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc).
>> New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE.
>>    ( https://patchwork.kernel.org/patch/11285409/ needs no change)
>>
>> This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections.
>>
>> I will be happy to make changes to lld and LLVM binary utilities if this
>> scheme reaches consensus.
>
>It is kind of too late now.

Better late than never. It is never late to fix the section type if we do intend to fix it.

Loaders don't read sections => the section type change is backward compatible.

On 2020-02-20, H.J. Lu wrote:

>On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <[hidden email]> wrote:
>>
>> Hi,
>>
>> On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote:
>> > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <[hidden email]> wrote:
>> > > This code isn't in the kernel yet. So either it gets changed to use the
>> > > existing scheme with gnu property notes found through PT_NOTE to work
>> > > with existing binaries. Then there is no need for PT_GNU_PROPERTY
>> > > headers.
>> > >
>> > > Or some future kernel will start using PT_GNU_PROPERTY headers to find
>> > > the gnu property notes. But that means it won't work with existing
>> > > binaries that do not have that header. So there is no backwards
>> > > compatibility anyway and we can define SHT_GNU_PROPERTY like above.
>> > >
>> > > So this actually seems the perfect time to make this decision.
>> >
>> > Binaries with .note.gnu.property section have been put into many
>> > OS releases.  We must support them.

We can teach newer assemblers to emit SHT_GNU_PROPERTY.
Newer linkers can support both SHT_GNU_PROPERTY/SHT_NOTE .note.gnu.property

At some point in the future, linkers can drop support for SHT_NOTE .note.gnu.property
Then it will become a graceful degradation: the old SHT_NOTE object files will not be
different from older object files without .note.gnu.property

>> OK. Then it is option 1. The kernel will need to support PT_NOTE for
>> parsing the properties, since such older binaries won't have a
>> PT_GNU_PROPERTY program header. Then we can simply get rid of
>> PT_GNU_PROPERTY since nobody uses it and all information is already
>> available through the PT_NOTE segment.
>>
>
>Kernel loader only checks ld.so and static executable.  Re-link them with
>newer linker will get PT_GNU_PROPERTY.  But ld.so needs to check
>PT_NOTE for older binaries.

The current PT_GNU_PROPERTY usage is all about hints. They are "nice to have" but not
"necessary to have". I don't see any problem teaching newer loaders to forget
PT_NOTE, if we do think PT_GNU_PROPERTY is the way forward.

The currently mixed status is annoying:

glibc: PT_NOTE
Proposed Linux kernel patch: PT_GNU_PROPERTY
Reply | Threaded
Open this post in threaded view
|

Re: binutils ld and new PT_GNU_PROPERTY segment

Mark Wielaard
In reply to this post by H.J. Lu-30
Hi,

On Thu, 2020-02-20 at 03:51 -0800, H.J. Lu wrote:

> On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <[hidden email]> wrote:
> > On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote:
> > >
> > > Binaries with .note.gnu.property section have been put into many
> > > OS releases.  We must support them.
> >
> > OK. Then it is option 1. The kernel will need to support PT_NOTE for
> > parsing the properties, since such older binaries won't have a
> > PT_GNU_PROPERTY program header. Then we can simply get rid of
> > PT_GNU_PROPERTY since nobody uses it and all information is already
> > available through the PT_NOTE segment.
> >
>
> Kernel loader only checks ld.so and static executable.  Re-link them with
> newer linker will get PT_GNU_PROPERTY.  But ld.so needs to check
> PT_NOTE for older binaries.

Having different loaders check different program headers seems very
confusing. And it doesn't really seem to provide backwards
compatibility since depending on how the code was linked previously you
might still require a rebuild.

Having a mix of PT_NOTE/PT_GNU_PROPERTY segments both based on
SHT_NOTEs, but one of them based on magic section names, makes things
really complicated for other tools.

Please lets either really keep things as they are or redesign things
based on PT_GNU_PROPERTY and a new section type.

Thanks,

Mark
12