mips RDHWR instruction in glibc

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

mips RDHWR instruction in glibc

Atsushi Nemoto
I got many "Reserved Instruction" exceptions with gcc 4.1 + glibc 2.4
userland.  They were due to RDHWR instruction to support TLS.

If a system call returned an error, glibc must save the result to
errno, which is thread-local, so RDHWR used.  I can understand this
scenario.  But it seems the RDHWR is often called on non-error cases.

For example, in the code below, RDHWR is placed _before_ checking the
error.  I suppose these instructions were reordered by gcc's
optimization, but the optimization would have large negative effect in
this case.

00566fc4 <_IO_file_read>:
  566fc4: 3c1c0016 lui gp,0x16
  566fc8: 279c87ac addiu gp,gp,-30804
  566fcc: 0399e021 addu gp,gp,t9
  566fd0: 8c82003c lw v0,60(a0)
  566fd4: 30420002 andi v0,v0,0x2
  566fd8: 14400003 bnez v0,566fe8 <_IO_file_read+0x24>
  566fdc: 8f999e9c lw t9,-24932(gp)
  566fe0: 03200008 jr t9
  566fe4: 8c840038 lw a0,56(a0)
  566fe8: 8c840038 lw a0,56(a0)
  566fec: 24020fa3 li v0,4003
  566ff0: 0000000c syscall
  566ff4: 8f84a528 lw a0,-23256(gp)
  566ff8: 7c03e83b rdhwr v1,$29
  566ffc: 00832021 addu a0,a0,v1
  567000: 14e00003 bnez a3,567010 <_IO_file_read+0x4c>
  567004: 00401821 move v1,v0
  567008: 03e00008 jr ra
  56700c: 00601021 move v0,v1
  567010: 2403ffff li v1,-1
  567014: 1000fffc b 567008 <_IO_file_read+0x44>
  567018: ac820000 sw v0,0(a0)

I'm not sure where to fix, but I doubt some inline asm code in glibc
lack "volatile" keyword.

Does anyone have a clue on this?
---
Atsushi Nemoto
Reply | Threaded
Open this post in threaded view
|

Re: mips RDHWR instruction in glibc

Daniel Jacobowitz
On Thu, Jun 15, 2006 at 12:12:38AM +0900, Atsushi Nemoto wrote:
> If a system call returned an error, glibc must save the result to
> errno, which is thread-local, so RDHWR used.  I can understand this
> scenario.  But it seems the RDHWR is often called on non-error cases.

Libc uses TLS for many things other than just errno.  The GCC port
knows how to generate the agreed-upon rdhwr instruction directly.

> For example, in the code below, RDHWR is placed _before_ checking the
> error.  I suppose these instructions were reordered by gcc's
> optimization, but the optimization would have large negative effect in
> this case.

You'd have to figure out how to get GCC not to eagerly schedule the
rdhwr.  This might be quite hard.  I don't know much about this part of
the scheduler.

--
Daniel Jacobowitz
CodeSourcery
Reply | Threaded
Open this post in threaded view
|

Re: mips RDHWR instruction in glibc

Atsushi Nemoto
On Wed, 14 Jun 2006 12:50:40 -0400, Daniel Jacobowitz <[hidden email]> wrote:
> > For example, in the code below, RDHWR is placed _before_ checking the
> > error.  I suppose these instructions were reordered by gcc's
> > optimization, but the optimization would have large negative effect in
> > this case.
>
> You'd have to figure out how to get GCC not to eagerly schedule the
> rdhwr.  This might be quite hard.  I don't know much about this part of
> the scheduler.

I really did not understand yet how errno is bound TLS.  I found some
"rdhwr" in glibc-ports source code (tls-macros.h, nptl/tls.h).  The
RDHWR instruction in the example code comes from one of them, no?

I also found a "rdhwr" in gcc's mips.md file ("tls_get_tp_<mode>").
Is this the origin?  MD is a very foreign language for me...

---
Atsushi Nemoto
Reply | Threaded
Open this post in threaded view
|

Re: mips RDHWR instruction in glibc

Daniel Jacobowitz
On Fri, Jun 16, 2006 at 12:28:37AM +0900, Atsushi Nemoto wrote:

> On Wed, 14 Jun 2006 12:50:40 -0400, Daniel Jacobowitz <[hidden email]> wrote:
> > > For example, in the code below, RDHWR is placed _before_ checking the
> > > error.  I suppose these instructions were reordered by gcc's
> > > optimization, but the optimization would have large negative effect in
> > > this case.
> >
> > You'd have to figure out how to get GCC not to eagerly schedule the
> > rdhwr.  This might be quite hard.  I don't know much about this part of
> > the scheduler.
>
> I really did not understand yet how errno is bound TLS.  I found some
> "rdhwr" in glibc-ports source code (tls-macros.h, nptl/tls.h).  The
> RDHWR instruction in the example code comes from one of them, no?

No.

> I also found a "rdhwr" in gcc's mips.md file ("tls_get_tp_<mode>").
> Is this the origin?  MD is a very foreign language for me...

Yes.  Compile something like this with -O2 but without -fpic:

__thread int x;
int foo() { return x; }

It should use the IE model, which will generate a rdhwr.

--
Daniel Jacobowitz
CodeSourcery
Reply | Threaded
Open this post in threaded view
|

Re: mips RDHWR instruction in glibc

Atsushi Nemoto
On Thu, 15 Jun 2006 11:32:52 -0400, Daniel Jacobowitz <[hidden email]> wrote:
> > I also found a "rdhwr" in gcc's mips.md file ("tls_get_tp_<mode>").
> > Is this the origin?  MD is a very foreign language for me...
>
> Yes.  Compile something like this with -O2 but without -fpic:
>
> __thread int x;
> int foo() { return x; }
>
> It should use the IE model, which will generate a rdhwr.

Thanks.  So this must be a gcc issue, not glibc issue.

extern __thread int x;
int foo(int arg)
{
        if (arg)
                return x;
        return 0;
}

If I compiled this program with -O2 I got:

foo:
        .frame $sp,0,$31 # vars= 0, regs= 0/0, args= 0, gp= 0
        .mask 0x00000000,0
        .fmask 0x00000000,0
        .set noreorder
        .cpload $25
        .set nomacro
       
        lw $2,%gottprel(x)($28)
        .set push
        .set mips32r2
        rdhwr $3,$29
        .set pop
        addu $2,$2,$3
        beq $4,$0,$L4
        move $3,$0

        lw $3,0($2)
$L4:
        j $31
        move $2,$3

It looks too bad for arg == 0 case.  I should ask on gcc list.

---
Atsushi Nemoto
Reply | Threaded
Open this post in threaded view
|

Re: mips RDHWR instruction in glibc

Atsushi Nemoto
On Sat, 17 Jun 2006 00:58:45 +0900 (JST), Atsushi Nemoto <[hidden email]> wrote:
> It looks too bad for arg == 0 case.  I should ask on gcc list.

Filed to gcc bugzilla.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28126

---
Atsushi Nemoto