Problem in pthread_create (allocate_stack inline code) on MIPS.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem in pthread_create (allocate_stack inline code) on MIPS.

Kaz Kylheku-3
Hey everyone,

We've discovered a problem of a large application (many processes, with
lots of threads) occupying an unusually large amount of virtual memory
on mips/n32 but not on the i686 version of the same software and OS.

The problem is that setting the pthread stack size attribute on MIPS
doesn't seem to be doing what it's supposed to, and so the threads end
up with some incorrect stack size.

Note, by the way, that I am still on glibc 2.5 here!

If we look at allocate_stack in nptl/allocatestack.c, we have this nasty
GCC extension:

  size = attr->stacksize ?: __default_stacksize;

This is apparently being mistranslated by gcc 4.1.1. Look at this:

  # v1 := attr->stacksize
    5410:       8e630014        lw      v1,20(s3)

  # if (v1 == 0) goto 5778
    5414:       106000d8        beqz    v1,5778
<pthread_create@@GLIBC_2.2+0x3d8>

  # s1 := v1
    5418:       0060882d        move    s1,v1

  # v0 := attr->flags
    541c:       8e620008        lw      v0,8(s3)

  # if ((v0 & ATTR_FLAG_STACKADDR) != 0) goto 5b30
    5420:       30420008        andi    v0,v0,0x8
    5424:       144001c2        bnez    v0,5b30
<pthread_create@@GLIBC_2.2+0x790

  # ... etc

So the logic is: the stack size is loaded into v1. If it's zero, then we
branch to some fixup code located elsewhere which implements the
alternative clause. Either way, the size is then copied into s1. Here is
the alternate code at 5778:

  # v0 := __default_stacksize     (presumably! Haven't verified this
global offset)
    5778:       8f82804c        lw      v0,-32692(gp)
 
  # goto 541c
    577c:       1000ff27        b       541c
<pthread_create@@GLIBC_2.2+0x7c>
 
This is inconsistent register use! The v0 register is wrong, because it
gets clobbered right away by the instruction at the branch target 541c.
How can the default stack size ever be used, in the common case?

This should be loading v1, and jumping back to 5418, so v1 is copied
into s1.

I have no idea what the stack calculation will actually end up doing; I
stopped looking when I found this.

Our application does use a nonzero value for the stack size, and so it
shouldn't be taking the branch to 5778, yet it's not behaving as it
should.
Reply | Threaded
Open this post in threaded view
|

RE: Problem in pthread_create (allocate_stack inline code) on MIPS.

Kaz Kylheku-3
Minutes ago, I wrote:
> The problem is that setting the pthread stack size attribute on MIPS
> doesn't seem to be doing what it's supposed to, and so the threads end
> up with some incorrect stack size.

More info:

Strangely, the application does respond to changes in the stack size via
"ulimit -s". The stack size specified in the pthread_attr_t is ignored,
but somehow it's still using the default stack size in spite of the
strange machine code.
Reply | Threaded
Open this post in threaded view
|

Re: Problem in pthread_create (allocate_stack inline code) on MIPS.

Atsushi Nemoto
In reply to this post by Kaz Kylheku-3
On Thu, 29 Nov 2007 16:02:04 -0800, "Kaz Kylheku" <[hidden email]> wrote:

>   # v0 := __default_stacksize     (presumably! Haven't verified this
> global offset)
>     5778:       8f82804c        lw      v0,-32692(gp)
>  
>   # goto 541c
>     577c:       1000ff27        b       541c
> <pthread_create@@GLIBC_2.2+0x7c>
>  
> This is inconsistent register use! The v0 register is wrong, because it
> gets clobbered right away by the instruction at the branch target 541c.
> How can the default stack size ever be used, in the common case?
>
> This should be loading v1, and jumping back to 5418, so v1 is copied
> into s1.

What is a instruction at 5780?  Is it a "lw s1, nn(v0)" or something,
isn't it?

I suppose LW at 5778 loads an address of __default_stacksize variable
and there should be an another load instruction to load its value.

---
Atsushi Nemoto