glibc at the Toolchains microconference at LPC 2019

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: syscall wrappers policy (was re: glibc at the Toolchains microconference)

Carlos O'Donell-6
On 6/27/19 10:10 AM, Zack Weinberg wrote:
>> * Wrappers should be added … unless there is a clear reason not to
>
> I do not understand the rationale for these exceptions.  Did you have
> specific cases in mind when you wrote these?

I think it was just a general escape hatch which is not needed.

If we find a really bad example then we'll have a public discussion about
it and can use consensus to change the rules.

Therefore the rules as written can be as strict as we agree to.

> (I’m particularly concerned that the “not quite a direct wrapper” rule
> would be used to argue against exposing a variant of `clone` that
> returns twice like `fork` does, which is a thing I think we should
> have.  You probably _can_ do any fork-with-options operation with the
> `clone` wrapper we have, but having to separate the child-side code to
> its own function and allocate stack space for it can be a serious obstacle.)

Updated. Please review.

https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Florian Weimer-5
In reply to this post by Zack Weinberg-2
* Zack Weinberg:

> I specifically disagree with this.  The existence of these dedicated
> libraries does not mean that there is no need for a minimal wrapper in
> the C library.  In fact, providing a minimal wrapper in the C library
> would make the implementation of dedicated libraries easier, since
> they can concentrate on designing their higher-level API rather than
> wasting engineering effort on system call wrappers.  glibc has already
> done all of the low-level work necessary.

We would have to begin backporting syscall wrappers, though.  Otherwise
these libraries are blocked until a glibc upgrade, which may not happen
any time soon.

Maybe we can move well-established libraries into glibc eventually, but
that can have unpredictable results if those libraries did not use
symbol versioning from the start (so that their implementation
interposes a newer glibc implementation for the entire process).

But I don't know if this (no syscall wrappers except in glibc) works as
a default policy.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Maciej W. Rozycki-5
In reply to this post by Florian Weimer-5
Hi Florian,

> glibc system call wrappers are on the agenda:
>
> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>
> Will anyone from the glibc community attend and can set the right
> expectations?

 I plan to be there and I can carry the message.

> I won't be there because I'm traveling to Cauldron instead, which is
> immediately after LPC, but in a different corner of the planet.

 I actually plan to be there at both events, so I can bring back to the
Cauldron whatever we'll have come up with at LPC too.

  Maciej
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Carlos O'Donell-6
In reply to this post by Florian Weimer-5
On 6/27/19 1:21 PM, Florian Weimer wrote:

> * Zack Weinberg:
>
>> I specifically disagree with this.  The existence of these dedicated
>> libraries does not mean that there is no need for a minimal wrapper in
>> the C library.  In fact, providing a minimal wrapper in the C library
>> would make the implementation of dedicated libraries easier, since
>> they can concentrate on designing their higher-level API rather than
>> wasting engineering effort on system call wrappers.  glibc has already
>> done all of the low-level work necessary.
>
> We would have to begin backporting syscall wrappers, though.  Otherwise
> these libraries are blocked until a glibc upgrade, which may not happen
> any time soon.

Correct.

> Maybe we can move well-established libraries into glibc eventually, but
> that can have unpredictable results if those libraries did not use
> symbol versioning from the start (so that their implementation
> interposes a newer glibc implementation for the entire process).

I don't think anyone is asking for this, but it will be a question that
comes up when established practice is migrated to glibc.

How did libstdc++'s adoption of boost APIs handle this? I think they
just renamed things?

> But I don't know if this (no syscall wrappers except in glibc) works as
> a default policy.

I didn't read Zack's comment as meaning "no syscall wrappers *except* in glibc,"
but "users can trust glibc to *always* provide syscall wrappers" in that we
as a community should not shirk our duty to provide syscall wrappers because
project X, Y, or Z says they will do it for us or are already doing it as
part of some userspace interface. There will always be users that want that
minimal C-callable interface.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Dmitry V. Levin
In reply to this post by Zack Weinberg-2
On Thu, Jun 27, 2019 at 11:57:46AM -0400, Zack Weinberg wrote:

> On Thu, Jun 27, 2019 at 5:39 AM Dmitry V. Levin wrote:
> >
> > On Wed, Jun 26, 2019 at 05:04:52PM -0400, Carlos O'Donell wrote:
> > [...]
> > > Could you please review the language here:
> > > https://sourceware.org/glibc/wiki/Consensus#WIP:_Kernel_syscalls_wrappers
> >
> > I suggest adding that there is no need to add wrappers for those syscalls
> > that already have dedicated libraries.
> >
> > For example, such multiplexers as bpf(2) and keyctl(2) already have
> > dedicated libraries (libbpf and libkeyutils, respectively) that provide
> > APIs on top of these raw syscalls.
>
> I specifically disagree with this.  The existence of these dedicated
> libraries does not mean that there is no need for a minimal wrapper in
> the C library.  In fact, providing a minimal wrapper in the C library
> would make the implementation of dedicated libraries easier, since
> they can concentrate on designing their higher-level API rather than
> wasting engineering effort on system call wrappers.  glibc has already
> done all of the low-level work necessary.
>
> I am a little disappointed to see that Linux is still inventing new
> multiplexed system calls, though.  I thought that was demonstrated to
> be a bad idea back in the days of __NR_ipc.
Speaking of multiplexed syscalls, if we don't event contemplate an idea
of adding a glibc wrapper for __NR_ipc, why would we want to add
a wrapper for __NR_bpf or __NR_keyctl?

Since these syscalls have interfaces explicitly designed to scare regular
users off and encourage them to use library functions instead, providing
glibc wrappers for them would mislead people into invoking these system
calls directly.  These wrappers would lack type checking available in
higher-level library functions, causing unnecessary runtime errors that
are easily avoidable at compile time when higher-level library functions
are used.

Can you demonstrate any potential usefulness of glibc wrappers for
__NR_bpf and __NR_keyctl?


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Florian Weimer
* Dmitry V. Levin:

> Speaking of multiplexed syscalls, if we don't event contemplate an idea
> of adding a glibc wrapper for __NR_ipc,

I think someonme did argue for socketcall when the topic came up a
while back.

> why would we want to add a wrapper for __NR_bpf or __NR_keyctl?

We have wrappers for ioctl, fcntl, ptrace, prctl.  I suppose they
could serve as models.

We'll see what the future will bring on the Linux.  I expect that
system calls are now easier to wire up consistently across
architectures, so perhaps we will see fewer multiplexer system calls
in the future.  I suspect that, for architecture-independent features,
these multiplexers had the important property that a kernel
contributor wouldn't need to touch arch-specific code to add further
sub-calls, which made them very attractive if you want to roll out new
features relatively quickly.

Regarding fcntl, I did look into providing a type-safe version of it
using _Generic/__builtin_types_compatible_p/__builtin_choose_expr and
so on.  But then I realized that instead, we probably should have
separate fcntl_* functions for each sub-call, similar to what we did
for socketcall.  gnulib could easily provide high-quality
implementations for libcs which do not pick up on this idea.

So yes, I tend to agree that exposing wrappers is problematic.

> Since these syscalls have interfaces explicitly designed to scare regular
> users off and encourage them to use library functions instead, providing
> glibc wrappers for them would mislead people into invoking these system
> calls directly.  These wrappers would lack type checking available in
> higher-level library functions, causing unnecessary runtime errors that
> are easily avoidable at compile time when higher-level library functions
> are used.

Right.  And fcntl isn't so different, actually.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Dmitry V. Levin
On Thu, Jun 27, 2019 at 11:22:45PM +0200, Florian Weimer wrote:

> * Dmitry V. Levin:
> > why would we want to add a wrapper for __NR_bpf or __NR_keyctl?
>
> We have wrappers for ioctl, fcntl, ptrace, prctl.  I suppose they
> could serve as models.
>
> We'll see what the future will bring on the Linux.  I expect that
> system calls are now easier to wire up consistently across
> architectures, so perhaps we will see fewer multiplexer system calls
> in the future.  I suspect that, for architecture-independent features,
> these multiplexers had the important property that a kernel
> contributor wouldn't need to touch arch-specific code to add further
> sub-calls, which made them very attractive if you want to roll out new
> features relatively quickly.
>
> Regarding fcntl, I did look into providing a type-safe version of it
> using _Generic/__builtin_types_compatible_p/__builtin_choose_expr and
> so on.  But then I realized that instead, we probably should have
> separate fcntl_* functions for each sub-call, similar to what we did
> for socketcall.  gnulib could easily provide high-quality
> implementations for libcs which do not pick up on this idea.
>
> So yes, I tend to agree that exposing wrappers is problematic.
>
> > Since these syscalls have interfaces explicitly designed to scare regular
> > users off and encourage them to use library functions instead, providing
> > glibc wrappers for them would mislead people into invoking these system
> > calls directly.  These wrappers would lack type checking available in
> > higher-level library functions, causing unnecessary runtime errors that
> > are easily avoidable at compile time when higher-level library functions
> > are used.
>
> Right.  And fcntl isn't so different, actually.
What if we start adding separate functions for new interfaces of already
existing multiplexed system call wrappers?

For example, ptrace is going to gain a new command (PTRACE_GET_SYSCALL_INFO)
in Linux 5.3.  My initial plan was just to update sys/ptrace.h with a new
constant and bits/ptrace-shared.h with a new structure, but I could add
a new function as well:

extern int ptrace_get_syscall_info (__pid_t __pid,
                                    struct __ptrace_syscall_info *__infop)
        __THROW;


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Florian Weimer-5
* Dmitry V. Levin:

> What if we start adding separate functions for new interfaces of already
> existing multiplexed system call wrappers?
>
> For example, ptrace is going to gain a new command (PTRACE_GET_SYSCALL_INFO)
> in Linux 5.3.  My initial plan was just to update sys/ptrace.h with a new
> constant and bits/ptrace-shared.h with a new structure, but I could add
> a new function as well:
>
> extern int ptrace_get_syscall_info (__pid_t __pid,
>    struct __ptrace_syscall_info *__infop)
> __THROW;

I think this makes a lot of sense.  We should also do this for existing
constants (and for fcntl, too).  Perhaps after we have added more of the
wrappers that are *completely* missing, though.

For futex, it will also help with the time_t transition.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Zack Weinberg-2
On Fri, Jun 28, 2019 at 5:35 PM Florian Weimer <[hidden email]> wrote:

>
> * Dmitry V. Levin:
>
> > What if we start adding separate functions for new interfaces of already
> > existing multiplexed system call wrappers?
> >
> > For example, ptrace is going to gain a new command (PTRACE_GET_SYSCALL_INFO)
> > in Linux 5.3.  My initial plan was just to update sys/ptrace.h with a new
> > constant and bits/ptrace-shared.h with a new structure, but I could add
> > a new function as well:
> >
> > extern int ptrace_get_syscall_info (__pid_t __pid,
> >                                   struct __ptrace_syscall_info *__infop)
> >       __THROW;
>
> I think this makes a lot of sense.  We should also do this for existing
> constants (and for fcntl, too).  Perhaps after we have added more of the
> wrappers that are *completely* missing, though.

Yeah, I like this idea as well.  I think we should keep exposing the
multiplexed interfaces, though, just because sometimes they do add new
opcodes (e.g. the OFD locks).

zw
Reply | Threaded
Open this post in threaded view
|

Re: wrappers for multiplexed syscalls (was Re: glibc at the Toolchains microconference at LPC 2019)

Florian Weimer-5
* Zack Weinberg:

> On Fri, Jun 28, 2019 at 5:35 PM Florian Weimer <[hidden email]> wrote:
>>
>> * Dmitry V. Levin:
>>
>> > What if we start adding separate functions for new interfaces of already
>> > existing multiplexed system call wrappers?
>> >
>> > For example, ptrace is going to gain a new command (PTRACE_GET_SYSCALL_INFO)
>> > in Linux 5.3.  My initial plan was just to update sys/ptrace.h with a new
>> > constant and bits/ptrace-shared.h with a new structure, but I could add
>> > a new function as well:
>> >
>> > extern int ptrace_get_syscall_info (__pid_t __pid,
>> >                                   struct __ptrace_syscall_info *__infop)
>> >       __THROW;
>>
>> I think this makes a lot of sense.  We should also do this for existing
>> constants (and for fcntl, too).  Perhaps after we have added more of the
>> wrappers that are *completely* missing, though.
>
> Yeah, I like this idea as well.  I think we should keep exposing the
> multiplexed interfaces, though, just because sometimes they do add new
> opcodes (e.g. the OFD locks).

Maybe.  But not as varargs functions, so that there's no ambiguity how
many arguments need to be extracted and passed to the system call (which
caused interoperability problems with open/openat).

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Florian Weimer-5
In reply to this post by Maciej W. Rozycki-5
* Maciej Rozycki:

>> glibc system call wrappers are on the agenda:
>>
>> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
>>
>> Will anyone from the glibc community attend and can set the right
>> expectations?
>
>  I plan to be there and I can carry the message.

Great, then we need to decide what it would be.

Maybe the following is a reasonable starting point?

The glibc project decided a while back that it wants to add wrappers for
system calls which are useful for general application usage.  However,
that doesn't mean that all those missing system calls are added
immediately.

System call wrappers still need documentation in the manual, which is
can be difficult in areas where there is no consensus how to describe
the desired semantics (e.g., in the area of concurrency).  (Copyright
assignment to the FSF is needed for both the code and the manual update,
but can usually performed electronically these days, and is reasonably
straightforward.)  On top of that, the glibc project is seriously
constrained by available reviewer bandwidth.

Some more specific notes:

Emulation of the system call is not required.  It has been historically
very problematic.  The only thing that has not come back to bite us is
checking if a new flag argument is zero and call the old, equivalent
system call instead in this case.

Wrapper names should be architecture-independent if at all possible.
Sharing system call names as much as possible between architectures in
the UAPI headers helps with that.

Mutiplexing system calls are difficult to wrap, particularly if the
types and number of arguments vary.  Previous attempts to use varargs
for this have lead to bugs.  For example, open/openat would not pass
down the mode flag for O_TMPFILE initially, or cannot be called with a
non-variadic prototype/function pointer on some architectures.  We
wouldn't want to wrap socketcall (even if it had not been superseded),
and may wrap futex as separate functions.

We strongly prefer if a system call that is not inherently
architecture-specific (e.g., some new VFS functionality) is enabled for
all architectures in the same kernel release.

When it comes to exposes the system call, we prefer to use size_t for
buffer sizes (even if the kernel uses int or unsigned int), purely for
documentation purposes.  Flag arguments should not be long int because
it is unclear whether in the future, more than 32 flags will be added on
64-bit architectures.  Except for pthread_* functions, error reporting
is based on errno and special return values.

Passing file offsets through off64_t * arguments is fine with us.
Otherwise, off64_t parameter passing tends to vary too much.

If constants and types related to a particular system call are defined
in a separate header which does not contain much else, we can incldue
that from the glibc headers if available.  As a result, new kernel flags
will become available to application developers immediately once they
install newer kernel headers.  (This may not work for multiplexing
system calls, of course, even if we wrap the multiplexer.)

Any other comments?

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Paul Eggert
Florian Weimer wrote:
> When it comes to exposes the system call, we prefer to use size_t for
> buffer sizes (even if the kernel uses int or unsigned int), purely for
> documentation purposes.

I suggest using "ssize_t or size_t" to keep the text parallel with "int or
unsigned int". We shouldn't insist on size_t for buffer sizes when the natural
interpretation of the value is signed (as in the return value for 'read' etc.)
or is too large for malloc etc.
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Dmitry V. Levin
In reply to this post by Florian Weimer-5
On Mon, Jul 08, 2019 at 02:57:32PM +0200, Florian Weimer wrote:

> * Maciej Rozycki:
>
> >> glibc system call wrappers are on the agenda:
> >>
> >> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>
> >> Will anyone from the glibc community attend and can set the right
> >> expectations?
> >
> >  I plan to be there and I can carry the message.
>
> Great, then we need to decide what it would be.
>
> Maybe the following is a reasonable starting point?
Looks good to me (modulo typos).
Apparently, I'm going to be at both events, too.


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Florian Weimer-5
In reply to this post by Paul Eggert
* Paul Eggert:

> Florian Weimer wrote:
>> When it comes to exposes the system call, we prefer to use size_t for
>> buffer sizes (even if the kernel uses int or unsigned int), purely for
>> documentation purposes.
>
> I suggest using "ssize_t or size_t" to keep the text parallel with
> "int or unsigned int". We shouldn't insist on size_t for buffer sizes
> when the natural interpretation of the value is signed (as in the
> return value for 'read' etc.) or is too large for malloc etc.

Right.  Incoming buffer sizes should be size_t (no in-band signaling).
Return values tend to have in-band signaling for errors and thus need to
be ssize_t.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Maciej W. Rozycki-5
On Fri, 19 Jul 2019, Florian Weimer wrote:

> > I suggest using "ssize_t or size_t" to keep the text parallel with
> > "int or unsigned int". We shouldn't insist on size_t for buffer sizes
> > when the natural interpretation of the value is signed (as in the
> > return value for 'read' etc.) or is too large for malloc etc.
>
> Right.  Incoming buffer sizes should be size_t (no in-band signaling).
> Return values tend to have in-band signaling for errors and thus need to
> be ssize_t.

 FYI, the response was positive and a rough transcript of the session is
available from <https://etherpad.net/p/LPC2019_TC>.  I'll be there at the
glibc BoF available to comment.

  Maciej
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Dmitry V. Levin
On Sat, Sep 14, 2019 at 05:33:03AM +0100, Maciej W. Rozycki wrote:
[...]
>  FYI, the response was positive and a rough transcript of the session is
> available from <https://etherpad.net/p/LPC2019_TC>.  I'll be there at the
> glibc BoF available to comment.

I had a short conversation with H. Peter Anvin (Cc'ed) who missed the talk.
Apparently, he wants to proceed with his plan to add a fallback library
with syscall wrappers into the kernel tree.


--
ldv

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Maciej W. Rozycki-5
On Sat, 14 Sep 2019, Dmitry V. Levin wrote:

> I had a short conversation with H. Peter Anvin (Cc'ed) who missed the talk.
> Apparently, he wants to proceed with his plan to add a fallback library
> with syscall wrappers into the kernel tree.

 Oh, great (on both accounts).

  Maciej
Reply | Threaded
Open this post in threaded view
|

Re: glibc at the Toolchains microconference at LPC 2019

Christian Brauner-2
In reply to this post by Florian Weimer-5
On Mon, Jul 08, 2019 at 02:57:32PM +0200, Florian Weimer wrote:
> * Maciej Rozycki:
>
> >> glibc system call wrappers are on the agenda:
> >>
> >> <https://www.linuxplumbersconf.org/blog/2019/toolchains-microconference-accepted-into-2019-linux-plumbers-conference/>

This was an excellent session and I'm quite pleased with the
outcome/agreement.
While we're at it: kcmp() has no glibc syscall wrapper. Would be great
if there would be one. :)

Thanks!
Christian
12