glibc at the Toolchains microconference at LPC 2019

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

glibc at the Toolchains microconference at LPC 2019

Florian Weimer-5
glibc system call wrappers are on the agenda:

<https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>

Will anyone from the glibc community attend and can set the right
expectations?

I won't be there because I'm traveling to Cauldron instead, which is
immediately after LPC, but in a different corner of the planet.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Dmitry V. Levin
On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
> glibc system call wrappers are on the agenda:
>
> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>
> Will anyone from the glibc community attend and can set the right
> expectations?

What are the right expectations?


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Zack Weinberg-2
On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]> wrote:
> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
> > glibc system call wrappers are on the agenda:
> >
> > <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
> >
> > Will anyone from the glibc community attend and can set the right
> > expectations?
>
> What are the right expectations?

Well, _I_ think glibc should provide wrappers for all Linux system
calls, except those that cannot be used without stomping on internal
glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
those that have been completely superseded by newer syscalls.  Other
people have disagreed with me pretty strenuously, but they haven't
done any of the work required to make forward progress on their more
conservative policies.  I am tempted to post a patch early in the 2.31
cycle that adds wrappers for everything, and then threaten to apply it
unilaterally unless I hear concrete objections within a week or so.

zw
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Florian Weimer-5
* Zack Weinberg:

> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]> wrote:
>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
>> > glibc system call wrappers are on the agenda:
>> >
>> > <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>> >
>> > Will anyone from the glibc community attend and can set the right
>> > expectations?
>>
>> What are the right expectations?
>
> Well, _I_ think glibc should provide wrappers for all Linux system
> calls, except those that cannot be used without stomping on internal
> glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
> those that have been completely superseded by newer syscalls.  Other
> people have disagreed with me pretty strenuously, but they haven't
> done any of the work required to make forward progress on their more
> conservative policies.  I am tempted to post a patch early in the 2.31
> cycle that adds wrappers for everything, and then threaten to apply it
> unilaterally unless I hear concrete objections within a week or so.

In my experience, it's been difficult to get reviewers.  So what the
project says it wants and what the project actually makes happen is
rather different.

There is currently a requirement that every wrapper needs a manual entry
(and, presumably, a test case, although I have not tested the waters on
that).  membarrier is not included only because we could not agree on
the manual text.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Yann Droneaud
In reply to this post by Zack Weinberg-2
Hi,

Le mercredi 26 juin 2019 à 12:50 -0400, Zack Weinberg a écrit :

> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]> wrote:
> > On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
> > > glibc system call wrappers are on the agenda:
> > >
> > > <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
> > >
> > > Will anyone from the glibc community attend and can set the right
> > > expectations?
> >
> > What are the right expectations?
>
> Well, _I_ think glibc should provide wrappers for all Linux system
> calls, except those that cannot be used without stomping on internal
> glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
> those that have been completely superseded by newer syscalls.  

I believe many more syscalls must not be exposed by glibc by being too
specific, architecture dedicated syscalls in particular.

I strongly think that syscall wrappers that cannot be implemented with
a single line of code should be avoided, as it's a sign a dedicated
library would probably be better.

Bigger wrappers should only be allowed when the provided syscall is
intended to emulate some standard and/or cross-platform API.

So, yes, you could probably say I'm on "conservative" side :)

Regards.

--
Yann Droneaud
OPTEYA


Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Zack Weinberg-2
In reply to this post by Florian Weimer-5
On Wed, Jun 26, 2019 at 12:56 PM Florian Weimer <[hidden email]> wrote:

> * Zack Weinberg:
> > Well, _I_ think glibc should provide wrappers for all Linux system
> > calls, except those that cannot be used without stomping on internal
> > glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
> > those that have been completely superseded by newer syscalls.  Other
> > people have disagreed with me pretty strenuously, but they haven't
> > done any of the work required to make forward progress on their more
> > conservative policies.  I am tempted to post a patch early in the 2.31
> > cycle that adds wrappers for everything, and then threaten to apply it
> > unilaterally unless I hear concrete objections within a week or so.
>
> In my experience, it's been difficult to get reviewers.  So what the
> project says it wants and what the project actually makes happen is
> rather different.

For the record, the shade in my message was not aimed at you.  I think
you're one of the only people who's managed to make forward progress
on adding syscall wrappers in the past several years.

zw
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Christian Brauner-5
In reply to this post by Zack Weinberg-2
On June 26, 2019 6:50:23 PM GMT+02:00, Zack Weinberg <[hidden email]> wrote:

>On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]>
>wrote:
>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
>> > glibc system call wrappers are on the agenda:
>> >
>> >
><https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>> >
>> > Will anyone from the glibc community attend and can set the right
>> > expectations?
>>
>> What are the right expectations?
>
>Well, _I_ think glibc should provide wrappers for all Linux system
>calls, except those that cannot be used without stomping on internal
>glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
>those that have been completely superseded by newer syscalls.  Other
>people have disagreed with me pretty strenuously, but they haven't
>done any of the work required to make forward progress on their more
>conservative policies.  I am tempted to post a patch early in the 2.31
>cycle that adds wrappers for everything, and then threaten to apply it
>unilaterally unless I hear concrete objections within a week or so.
>
>zw

Strongly agree.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Carlos O'Donell-6
In reply to this post by Florian Weimer-5
On 6/26/19 12:56 PM, Florian Weimer wrote:

> * Zack Weinberg:
>
>> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]> wrote:
>>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
>>>> glibc system call wrappers are on the agenda:
>>>>
>>>> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>>>>
>>>> Will anyone from the glibc community attend and can set the right
>>>> expectations?
>>>
>>> What are the right expectations?
>>
>> Well, _I_ think glibc should provide wrappers for all Linux system
>> calls, except those that cannot be used without stomping on internal
>> glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
>> those that have been completely superseded by newer syscalls.  Other
>> people have disagreed with me pretty strenuously, but they haven't
>> done any of the work required to make forward progress on their more
>> conservative policies.  I am tempted to post a patch early in the 2.31
>> cycle that adds wrappers for everything, and then threaten to apply it
>> unilaterally unless I hear concrete objections within a week or so.
>
> In my experience, it's been difficult to get reviewers.  So what the
> project says it wants and what the project actually makes happen is
> rather different.

It is difficult to get reviewers for *all* patches.

Therefore I don't think this is particular to syscall wrappers.

I've tried hard to review many of your syscall wrappers and make good
on the promise we gave to the kernel community that we would do so.

Lastly, if you do reviews please provide your "Reviewed-by" markers
since it will let me run metrics on how many people we have reviewing
and who they are, and use that information to for a long-term strategy
for getting more reviewers.

> There is currently a requirement that every wrapper needs a manual entry
> (and, presumably, a test case, although I have not tested the waters on
> that).  membarrier is not included only because we could not agree on
> the manual text.

And rightly so. I would hope that we all agree that we need documentation
and testing of interfaces in order to provide our users with the information
they need to use these interfaces.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Christian Brauner-5
On June 26, 2019 10:33:37 PM GMT+02:00, Carlos O'Donell <[hidden email]> wrote:

>On 6/26/19 12:56 PM, Florian Weimer wrote:
>> * Zack Weinberg:
>>
>>> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]>
>wrote:
>>>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
>>>>> glibc system call wrappers are on the agenda:
>>>>>
>>>>>
><https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>>>>>
>>>>> Will anyone from the glibc community attend and can set the right
>>>>> expectations?
>>>>
>>>> What are the right expectations?
>>>
>>> Well, _I_ think glibc should provide wrappers for all Linux system
>>> calls, except those that cannot be used without stomping on internal
>>> glibc data structures (e.g. set_tid_address, set_robust_list, brk)
>and
>>> those that have been completely superseded by newer syscalls.  Other
>>> people have disagreed with me pretty strenuously, but they haven't
>>> done any of the work required to make forward progress on their more
>>> conservative policies.  I am tempted to post a patch early in the
>2.31
>>> cycle that adds wrappers for everything, and then threaten to apply
>it
>>> unilaterally unless I hear concrete objections within a week or so.
>>
>> In my experience, it's been difficult to get reviewers.  So what the
>> project says it wants and what the project actually makes happen is
>> rather different.
>
>It is difficult to get reviewers for *all* patches.
>
>Therefore I don't think this is particular to syscall wrappers.
>
>I've tried hard to review many of your syscall wrappers and make good
>on the promise we gave to the kernel community that we would do so.
>
>Lastly, if you do reviews please provide your "Reviewed-by" markers
>since it will let me run metrics on how many people we have reviewing
>and who they are, and use that information to for a long-term strategy
>for getting more reviewers.
>
>> There is currently a requirement that every wrapper needs a manual
>entry
>> (and, presumably, a test case, although I have not tested the waters
>on
>> that).  membarrier is not included only because we could not agree on
>> the manual text.
>
>And rightly so. I would hope that we all agree that we need
>documentation
>and testing of interfaces in order to provide our users with the
>information
>they need to use these interfaces.

Difficult to get reviewers in the sense of kernel people who wrote the syscalls?
I'm trying as hard as I can to bridge the libc/kernel barrier
by always cc'ing e.g. Florian or Dmitry
and ask for input on what you need.
I'm happy to work as closely as I can.

Christian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Carlos O'Donell-6
On 6/26/19 4:39 PM, Christian Brauner wrote:
> Difficult to get reviewers in the sense of kernel people who wrote
> the syscalls? I'm trying as hard as I can to bridge the libc/kernel
> barrier by always cc'ing e.g. Florian or Dmitry and ask for input on
> what you need. I'm happy to work as closely as I can.

Anyone can review patches on libc-alpha, and provide a "Reviewed-by"
sign-off. With enough of these the patch gains consensus that the right
people have agreed to it. This make subsequent review even easier if
I know the original author of the syscall signs off on the glibc
implementation as meeting their expectations.

For syscall patches we generally want a peer review from another glibc
developer who knows the interfaces and is looking for any mistakes,
or missing information e.g. versions, internal macro usage, docs,
test case usage etc.

So in summary:
- A nod from the original syscall author that their expectations are
  translated into userspace.
- A nod from another glibc developer as a "belt-and-suspenders" that
  the patch doens't have any problems.

That's enough for a commit to master in my book.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Carlos O'Donell-6
In reply to this post by Christian Brauner-5
On 6/26/19 1:41 PM, Christian Brauner wrote:

> On June 26, 2019 6:50:23 PM GMT+02:00, Zack Weinberg <[hidden email]> wrote:
>> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]>
>> wrote:
>>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
>>>> glibc system call wrappers are on the agenda:
>>>>
>>>>
>> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>>>>
>>>> Will anyone from the glibc community attend and can set the right
>>>> expectations?
>>>
>>> What are the right expectations?
>>
>> Well, _I_ think glibc should provide wrappers for all Linux system
>> calls, except those that cannot be used without stomping on internal
>> glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
>> those that have been completely superseded by newer syscalls.  Other
>> people have disagreed with me pretty strenuously, but they haven't
>> done any of the work required to make forward progress on their more
>> conservative policies.  I am tempted to post a patch early in the 2.31
>> cycle that adds wrappers for everything, and then threaten to apply it
>> unilaterally unless I hear concrete objections within a week or so.
>>
>> zw
>
> Strongly agree.
>

Zach,

Could you please review the language here:
https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers

I drafted it in 2014-10-23 based on comments from Joseph Myers and the
community (almost 5 years ago!)

If you agree to that langauge I'll propose this again as being accepted
consensus.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Dmitry V. Levin
On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
[...]
> Could you please review the language here:
> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers

I suggest adding that there is no need to add wrappers for those syscalls
that already have dedicated libraries.

For example, such multiplexers as bpf(2) and keyctl(2) already have
dedicated libraries (libbpf and libkeyutils, respectively) that provide
APIs on top of these raw syscalls.

keyctl(2) manual page explicitly says that "rather than using this system
call directly, you probably want to use the various library functions
mentioned in the descriptions of individual operations".


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Szabolcs Nagy-2
On 27/06/2019 10:39, Dmitry V. Levin wrote:

> On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
> [...]
>> Could you please review the language here:
>> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
>
> I suggest adding that there is no need to add wrappers for those syscalls
> that already have dedicated libraries.
>
> For example, such multiplexers as bpf(2) and keyctl(2) already have
> dedicated libraries (libbpf and libkeyutils, respectively) that provide
> APIs on top of these raw syscalls.

there are many issues doing raw syscalls e.g.
the x32 type mess or cancellation support.

external library projects can have different level
of quality, supported abis, header conformance,
security process etc. and they almost always mix
libc and linux uapi headers and types.

so i'm against relying on external libraries
doing raw syscalls (they may provide additional
functionality but the syscall itself should
be in libc)

>
> keyctl(2) manual page explicitly says that "rather than using this system
> call directly, you probably want to use the various library functions
> mentioned in the descriptions of individual operations".
>
>

Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Christian Brauner-5
On Thu, Jun 27, 2019 at 10:05:24AM +0000, Szabolcs Nagy wrote:

> On 27/06/2019 10:39, Dmitry V. Levin wrote:
> > On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
> > [...]
> >> Could you please review the language here:
> >> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
> >
> > I suggest adding that there is no need to add wrappers for those syscalls
> > that already have dedicated libraries.
> >
> > For example, such multiplexers as bpf(2) and keyctl(2) already have
> > dedicated libraries (libbpf and libkeyutils, respectively) that provide
> > APIs on top of these raw syscalls.
>
> there are many issues doing raw syscalls e.g.
> the x32 type mess or cancellation support.
>
> external library projects can have different level
> of quality, supported abis, header conformance,
> security process etc. and they almost always mix
> libc and linux uapi headers and types.
>
> so i'm against relying on external libraries
> doing raw syscalls (they may provide additional
> functionality but the syscall itself should
> be in libc)

I agree.
Not just does the quality of those libraries often vary wildly.
Sometimes there are competing libraries and it is unclear which one is
properly maintained and which one is not. (A good example are the
capability libraries libcap2 and libcap-ng (or whatever it's called).)

I think that (g)libc should not make a judgement on whether or not a
syscall should be exposed apart from the caveats Carlos and Zack already
pointed out. It's a massive paintpoint for userspace if the only options
are: link against a (possibly unmaintained) library or do the raw
syscall yourself.

Christian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Dmitry V. Levin
In reply to this post by Szabolcs Nagy-2
On Thu, Jun 27, 2019 at 10:05:24AM +0000, Szabolcs Nagy wrote:

> On 27/06/2019 10:39, Dmitry V. Levin wrote:
> > On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
> > [...]
> >> Could you please review the language here:
> >> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
> >
> > I suggest adding that there is no need to add wrappers for those syscalls
> > that already have dedicated libraries.
> >
> > For example, such multiplexers as bpf(2) and keyctl(2) already have
> > dedicated libraries (libbpf and libkeyutils, respectively) that provide
> > APIs on top of these raw syscalls.
>
> there are many issues doing raw syscalls e.g.
> the x32 type mess or cancellation support.
Yet raw syscalls have always been in use and this is not going to change.
If there are issues, we should consider providing appropriate interfaces
for invoking raw syscalls without these issues.

> external library projects can have different level
> of quality, supported abis, header conformance,
> security process etc. and they almost always mix
> libc and linux uapi headers and types.

This is all true but ...

> so i'm against relying on external libraries
> doing raw syscalls (they may provide additional
> functionality but the syscall itself should
> be in libc)

... some syscalls seem to have interfaces explicitly designed to scare
regular users off and encourage them to use library functions instead.
bpf(2) and keyctl(2) are examples of such interfaces, providing glibc
wrappers for them would mislead people into invoking these system calls
directly.


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Zack Weinberg-2
In reply to this post by Carlos O'Donell-6
On Wed, Jun 26, 2019 at 5:04 PM Carlos O'Donell <[hidden email]> wrote:

>
> On 6/26/19 1:41 PM, Christian Brauner wrote:
> > On June 26, 2019 6:50:23 PM GMT+02:00, Zack Weinberg <[hidden email]> wrote:
> >> On Wed, Jun 26, 2019 at 12:39 PM Dmitry V. Levin <[hidden email]>
> >> wrote:
> >>> On Wed, Jun 26, 2019 at 06:01:28PM +0200, Florian Weimer wrote:
> >>>> glibc system call wrappers are on the agenda:
> >>>>
> >>>>
> >> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
> >>>>
> >>>> Will anyone from the glibc community attend and can set the right
> >>>> expectations?
> >>>
> >>> What are the right expectations?
> >>
> >> Well, _I_ think glibc should provide wrappers for all Linux system
> >> calls, except those that cannot be used without stomping on internal
> >> glibc data structures (e.g. set_tid_address, set_robust_list, brk) and
> >> those that have been completely superseded by newer syscalls.  Other
> >> people have disagreed with me pretty strenuously, but they haven't
> >> done any of the work required to make forward progress on their more
> >> conservative policies.  I am tempted to post a patch early in the 2.31
> >> cycle that adds wrappers for everything, and then threaten to apply it
> >> unilaterally unless I hear concrete objections within a week or so.
> >>
> >> zw
> >
> > Strongly agree.
> >
>
> Zach,
>
> Could you please review the language here:
> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
>
> I drafted it in 2014-10-23 based on comments from Joseph Myers and the
> community (almost 5 years ago!)
>
> If you agree to that langauge I'll propose this again as being accepted
> consensus.
>
> --
> Cheers,
> Carlos.
Reply | Threaded
Open this post in threaded view
|

syscall wrappers policy (was re: glibc at the Toolchains microconference)

Zack Weinberg-2
In reply to this post by Carlos O'Donell-6
[sorry about the blank reply earlier, I pushed the wrong button]

On Wed, Jun 26, 2019 at 5:04 PM Carlos O'Donell <[hidden email]> wrote:
>
> Could you please review the language here:
> https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
>
> I drafted it in 2014-10-23 based on comments from Joseph Myers and the
> community (almost 5 years ago!)

This seems mostly right to me, but I have some concerns.

First, I think we need a definition of “syscall wrapper.”  Proposed: A
syscall wrapper is a function that, on a system with a traditional
Unix-style kernel (i.e. we don’t look at Hurd when assessing this)
does substantially all of its work by calling into the kernel.  It may
shuffle its arguments around, it may translate between the glibc ABI
and some lower-level ABI, and it may not actually perform a
system-call trap (e.g. `clock_gettime`), but it couldn’t do what it
does without invoking kernel code, and it doesn’t do any nontrivial
work itself.

The “nontrivial work itself” part is meant to draw a line with `open`
and `signal` on the “is a syscall wrapper” side (even though, on
modern systems, the “true” system calls they use are named `openat`
and `rt_sigaction` respectively), and `pthread_mutex_lock` on the
“isn’t a syscall wrapper” side (even though it does make system calls
under some circumstances, it does a lot of work itself).  I would like
to tighten this up but I’m not having any luck thinking of a better way
to phrase it.

Second, I think we need to talk a bit about the rationale for the
policy.  Something about how it has traditionally been the C library’s
responsibility to provide minimally mediated access to all of the
functionality of the kernel, and how in the past GNU libc has
hesitated to add Linux-specific syscall wrappers on portability and/or
backward compatibility grounds, but we now think that was a mistake.

> * If a syscall is obsoleted by another syscall (or otherwise
>   considered obsolete), there is no need to add a wrapper to glibc.

In the user-facing documentation (not necessarily the consensus rules)
it might be worth adding some kind of reassurance that we’re not going
to get rid of `open` even though it _is_ traditionally a “syscall
wrapper” and `openat` supersedes it.  “Existing functions that
traditionally were syscall wrappers will be preserved, even if the
syscall that does exactly what they do is obsoleted by a newer one
with broader capabilities, if they are specified by ISO C and/or
POSIX, or if they are widely used” or something like that.

But, at the same time, we _do_ need to say that syscall wrappers that
are OS-specific are subject to weaker compatibility guarantees.  We
have, after all, been removing things like nfsservctl and ustat.

> * If a syscall cannot meaningfully be used behind glibc's back, or
>   is not useful in the glibc context except for in the ways in which
>   it is used by glibc

I would prefer to narrow this to something like what I said in my
previous message: “glibc will not provide wrappers for syscalls that
are _impossible_ to use from a program linked against glibc without
corrupting glibc’s internal data structures.  For instance,
`set_thread_area`, `set_tid_address`, `set_robust_list`.”
“Meaningfully used” and “not useful in the glibc context” are too
fuzzy and I’m worried we will be setting ourselves up for arguments.

> * If there's a glibc function that's not quite a direct wrapper of
>   the syscall but provides all the functionality of it that can
>   usefully be used in a program using glibc, there is no need

> * Wrappers should be added … unless there is a clear reason not to

I do not understand the rationale for these exceptions.  Did you have
specific cases in mind when you wrote these?

(I’m particularly concerned that the “not quite a direct wrapper” rule
would be used to argue against exposing a variant of `clone` that
returns twice like `fork` does, which is a thing I think we should
have.  You probably _can_ do any fork-with-options operation with the
`clone` wrapper we have, but having to separate the child-side code to
its own function and allocate stack space for it can be a serious obstacle.)

zw
Reply | Threaded
Open this post in threaded view
|

Re: syscall wrappers policy (was re: glibc at the Toolchains microconference)

DJ Delorie-2
Zack Weinberg <[hidden email]> writes:
> First, I think we need a definition of “syscall wrapper.”  Proposed:

I offer alternate wording, not because I think mine is better, but to
stir the pot a bit and get people thinking :-)

"A syscall wrapper is a function whose primary (preferably sole) purpose
is to provide a minimal C-callable interface to a kernel function."

I think that explains why glibc has them (C-callable), limits its scope
(primary/sole purpose) and defines its purpose (kernel function).

How glibc implements them, and how much it needs to do to retain
compatiblity with other C APIs and standards, can be documented
separately.  Or we could add ", consistent with other glibc functions"
to the end of the above.

> Second, I think we need to talk a bit about the rationale for the
> policy.

I think definining it as a "C API" makes it clear that the C library is
the right place for C functions.  That also keeps us from trying to be
the ONLY kernel API library, when other languages need non-C wrappers.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Zack Weinberg-2
In reply to this post by Dmitry V. Levin
On Thu, Jun 27, 2019 at 5:39 AM Dmitry V. Levin <[hidden email]> wrote:

>
> On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
> [...]
> > Could you please review the language here:
> > https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
>
> I suggest adding that there is no need to add wrappers for those syscalls
> that already have dedicated libraries.
>
> For example, such multiplexers as bpf(2) and keyctl(2) already have
> dedicated libraries (libbpf and libkeyutils, respectively) that provide
> APIs on top of these raw syscalls.

I specifically disagree with this.  The existence of these dedicated
libraries does not mean that there is no need for a minimal wrapper in
the C library.  In fact, providing a minimal wrapper in the C library
would make the implementation of dedicated libraries easier, since
they can concentrate on designing their higher-level API rather than
wasting engineering effort on system call wrappers.  glibc has already
done all of the low-level work necessary.

I am a little disappointed to see that Linux is still inventing new
multiplexed system calls, though.  I thought that was demonstrated to
be a bad idea back in the days of __NR_ipc.

zw
Reply | Threaded
Open this post in threaded view
|

Re: syscall wrappers policy (was re: glibc at the Toolchains microconference)

Zack Weinberg-2
In reply to this post by DJ Delorie-2
On Thu, Jun 27, 2019 at 11:49 AM DJ Delorie <[hidden email]> wrote:

>
> Zack Weinberg <[hidden email]> writes:
> > First, I think we need a definition of “syscall wrapper.”  Proposed:
>
> I offer alternate wording, not because I think mine is better, but to
> stir the pot a bit and get people thinking :-)
>
> "A syscall wrapper is a function whose primary (preferably sole) purpose
> is to provide a minimal C-callable interface to a kernel function."
>
> I think that explains why glibc has them (C-callable), limits its scope
> (primary/sole purpose) and defines its purpose (kernel function).

I like this.  "Minimal interface" is better than "doesn't do any
nontrivial work itself."

> > Second, I think we need to talk a bit about the rationale for the
> > policy.
>
> I think definining it as a "C API" makes it clear that the C library is
> the right place for C functions.  That also keeps us from trying to be
> the ONLY kernel API library, when other languages need non-C wrappers.

Yeah.  I feel like there's a need, nowadays, for a language-agnostic
kernel API library and a dynamic loader completely decoupled from the
C library, but that's a big can of worms and we shouldn't let it get
in the way of anything else.

zw
12