[Bug libc/214] sbrk() doesn't detect brk() failures. Malloc doesn't handle sbrk() failures

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[Bug libc/214] sbrk() doesn't detect brk() failures. Malloc doesn't handle sbrk() failures

tromey at sourceware dot org

------- Additional Comments From rsa at us dot ibm dot com  2006-03-16 00:29 -------
Created an attachment (id=921)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=921&action=view)
demonstration of malloc() with-regard-to RLIMIT_AS

The attached source file demonstrates three behaviors invoked with the

1) gcc -o v6 v6glibc214.c
2) gcc -o v6 v6glibc214.c -DEXCEED_BRK
3) gcc -o v6 v6glibc214.c -DSET_RLIMIT_AS

1) RLIMIT_DATA rlim_cur soft limit of 2 pages is set.  ld maps this program on
a page boundary and gives us one page (out of the two) to work with so we brk
at the start_brk + 1 page.  We are allowed to read/write memory up to the brk
set by brk() without mallocing.  Subsequent mallocs grab memory further up the
address space, but not directly following curbrk.

2) We attempt to read/write to start_brk + 1 page + 4 bytes which exceeds cur
brk and creates a segmentation fault.

3) Setting RLIMIT_AS rlim_cur soft limit to RLIMIT_DATA rlim_cur soft limit
limits the address space memory that can be consumed by the program.
Effectively this stops the mallocs from succeeding after we brk up to the
RLIMIT_DATA soft limit.  Malloc returns NULL.


1.) mmap isn't governed by RLIMIT_DATA and malloc may resort to mmap when
sbrk() fails.  Setting the RLIMIT_AS soft-limit to RLIMIT_DATA soft-limit will
prevent malloc from exceeding the RLIMIT_DATA soft-limit.  I haven't tried
setting RLIMIT_AS soft-limit below RLIMIT_DATA soft-limit but I

1a.) Setting RLIMIT_AS below RLIMIT_DATA doesn't seem to prevent malloc from
using all the memory in the data segment up the RLIMIT_DATA because malloc
probably uses sbrk() until it returns 0.

2.) The kernel allocates memory on page boundaries so all soft-limits are only
enforced at the upward page boundary.

3.) I never experienced a seg fault with malloc() when mallocing data beyond
RLIMIT_DATA rlim_cur soft-limit.

After further analysis I still think this is working as designed.



------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.