Unwinding CFI gcc practice of assumed `same value' regs

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Unwinding CFI gcc practice of assumed `same value' regs

Jan Kratochvil-2
Hello,

currently (on x86_64) the gdb backtrace does not properly stop at the outermost
frame:

#3  0x00000036ddb0610a in start_thread () from /lib64/tls/libpthread.so.0
#4  0x00000036dd0c68c3 in clone () from /lib64/tls/libc.so.6
#5  0x0000000000000000 in ?? ()

Currently it relies only on clearing %rbp (0x0000000000000000 above is
unrelated to it, it got read from uninitialized memory).

http://sourceware.org/ml/gdb/2004-08/msg00060.html suggests frame pointer 0x0
should be enough for a debugger not finding CFI to stop unwinding, still it is
a heuristic.  In the -fno-frame-pointer compiled code there is no indication
the frame pointer register became a regular one and 0x0 is its valid value.

The right unwinding stop indication should be CFI-undefined PC:
        http://dwarf.freestandards.org/Dwarf3.pdf - page 118 (130/267)
        - If a Return Address register is defined in the virtual unwind table,
          and its rule is undefined (for example, by DW_CFA_undefined), then
          there is no return address and no call address, and the virtual
          unwind of stack activations is complete.

It has been recently patched for glibc clone() (as the outermost frame)
        http://sourceware.org/ml/libc-alpha/2006-11/msg00082.html
but it had to be reverted as the current libgcc_s unwinder is incompatible:
        http://sourceware.org/ml/libc-alpha/2006-12/msg00078.html

Current libgcc_s unwinder does not differentiate between `DW_CFA_undefined' and
`DW_CFA_same_value' as both set a register state to its `REG_UNSAVED'.

Updating its behavior unfortunately relies on another current ABI incoherency
as DWARF specifies all the registers should be `undefined' by default
        http://dwarf.freestandards.org/Dwarf3.pdf - page 111 (123/267)
        - The default rule for all columns before interpretation of the initial
          instructions is the undefined rule.  However, an ABI authoring body
          or a compilation system authoring body may specify an alternate
          default value for any or all columns.

but current gcc produces CFI code assuming the default state of all the
registers is `same value'.

The strictly DWARF compliant gcc behavior would be to initialize all the callee
saved registers to `DW_CFA_same_value' in all the CIE initial instructions
blocks.  This is currently not done and it would introduce a large increase of
`.eh_frame' size, despite the merging of common CIEs.


Therefore (Jakub is) proposing for gcc `compilation system authoring' to define
the default callee saved registers as `same value' and callee unsaved registers
as `undefined'.  It would make the current gcc CFI practice stanards compliant
and the unwinding stop CFI would became clear and -fomit-frame-pointer reliable.


GCC: `unwind-dw2' needs update to differentiate `undefined' and `same value'.
     AFAIK its code (CFI) generator may remain the same.

GLIBC: Reintroduce the patch above utilizing `.cfi_undefined rip'.

GDB: The currently disabled (default `set complaints 0') warnings made obsolete:
      During symbol reading, incomplete CFI data; unspecified registers (e.g., eax) at 0xfc1400.

Other unwinders: May need fix to support `DW_CFA_undefined', done for libunwind.


Most of the problem analysis by the courtesy of Jakub Jelinek.

Not providing any patches so far until the final solution decision is made.


Regards,
Jan
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Roland McGrath
GCC's unwinder doesn't distinguish undefined from same_value, because it
doesn't matter for EH unwinding purposes.  Both mean "nothing to be done
for this register".  The distinction only matters to informative unwinding
purposes like debugging.  I'm not sure why libgcc's unwinder really ought
to care.  It's not that I'm against it knowing the difference; that
certainly seems a cleaner way for it to be internally.  But as to the idea
that it needs to distinguish them for correctness, and thus other things
need to rely on having a libgcc_s version that does, and so forth, I don't
see the motivation.  

In the ideal world, things would use cfi_undefined on the pc regno to
indicate the base frame, as the dwarf3 spec says to.  I certainly think it
would be cleanest for everything to do that.  But again, in practice on
i386 and x86_64, I'm not sure I see the need.  Correct unwind info should
always restore the caller's bp register value.  When that unwinds to the
outermost frame, that will be a zero value as the runtime code of base
frames sets it.  

My reading is that the "ABI authoring body" for GNU systems or the
"compilation system authoring body" for GNU compilers already specifies
that the default rule is same_value for callee-saves registers (as chosen
by each particular ABI), even if this has not been formally documented
anywhere heretofore.  (This is how I've written ABI support in another
unwinder implementation I've worked on.)  As you've said, this is the only
reading by which current CFI is correct and complete for getting the values
of callee-saves registers.  I presume that GCC's omission of rules for
those registers is in fact simply because EH unwinding doesn't care and
people on the generation side just didn't think about it beyond that.
Regardless of the true reasons for the history, the description above
applies to the manifest practice that constitutes what we want the formal
specification to mean.


Thanks,
Roland
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Andrew Haley
In reply to this post by Jan Kratochvil-2
Jan Kratochvil writes:

 > currently (on x86_64) the gdb backtrace does not properly stop at
 > the outermost frame:
 >
 > #3  0x00000036ddb0610a in start_thread () from /lib64/tls/libpthread.so.0
 > #4  0x00000036dd0c68c3 in clone () from /lib64/tls/libc.so.6
 > #5  0x0000000000000000 in ?? ()
 >
 > Currently it relies only on clearing %rbp (0x0000000000000000 above is
 > unrelated to it, it got read from uninitialized memory).

That's how it's defined to work: %rbp is zero.  

 > http://sourceware.org/ml/gdb/2004-08/msg00060.html suggests frame
 > pointer 0x0 should be enough for a debugger not finding CFI to stop
 > unwinding, still it is a heuristic.

Not by my understanding it isn't.  It's set up by the runtime system,
and 0 (i.e. NULL on x86-64) marks the end of the stack.  Officially.

See page 28, AMD64 ABI Draft 0.98 \u2013 September 27, 2006 -- 9:24.

Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Mark Kettenis
>  Jan Kratochvil writes:
>
>   > currently (on x86_64) the gdb backtrace does not properly stop at
>   > the outermost frame:
>   >
>   > #3  0x00000036ddb0610a in start_thread () from
>  /lib64/tls/libpthread.so.0
>   > #4  0x00000036dd0c68c3 in clone () from /lib64/tls/libc.so.6
>   > #5  0x0000000000000000 in ?? ()
>   >
>   > Currently it relies only on clearing %rbp (0x0000000000000000 above is
>   > unrelated to it, it got read from uninitialized memory).
>
>  That's how it's defined to work: %rbp is zero.
>
>   > http://sourceware.org/ml/gdb/2004-08/msg00060.html suggests frame
>   > pointer 0x0 should be enough for a debugger not finding CFI to stop
>   > unwinding, still it is a heuristic.
>
>  Not by my understanding it isn't.  It's set up by the runtime system,
>  and 0 (i.e. NULL on x86-64) marks the end of the stack.  Officially.
>
>  See page 28, AMD64 ABI Draft 0.98 \u2013 September 27, 2006 -- 9:24.

Unfortunately whoever wrote that down didn't think it through.  In
Figure 3.4 on page 20, %rbp is listed as "callee-saved register;
optionally used as frame pointer".  So %rbp can be used for anything, as
long as you save its contents and restore it before you return.  Since it
may be used for anything, it may contain 0 at any point in the middle of
the call stack.  So it is unusable as a stack trace termination condition.
The only viable option is explicitly marking it as such in the CFI.

Initializing %rbp to 0 in the outermost frame is sort of pointless on amd64.

Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Andrew Haley
Mark Kettenis writes:
 > >  Jan Kratochvil writes:
 > >
 > >   > currently (on x86_64) the gdb backtrace does not properly stop at
 > >   > the outermost frame:
 > >   >
 > >   > #3  0x00000036ddb0610a in start_thread () from
 > >  /lib64/tls/libpthread.so.0
 > >   > #4  0x00000036dd0c68c3 in clone () from /lib64/tls/libc.so.6
 > >   > #5  0x0000000000000000 in ?? ()
 > >   >
 > >   > Currently it relies only on clearing %rbp (0x0000000000000000 above is
 > >   > unrelated to it, it got read from uninitialized memory).
 > >
 > >  That's how it's defined to work: %rbp is zero.
 > >
 > >   > http://sourceware.org/ml/gdb/2004-08/msg00060.html suggests frame
 > >   > pointer 0x0 should be enough for a debugger not finding CFI to stop
 > >   > unwinding, still it is a heuristic.
 > >
 > >  Not by my understanding it isn't.  It's set up by the runtime system,
 > >  and 0 (i.e. NULL on x86-64) marks the end of the stack.  Officially.
 > >
 > >  See page 28, AMD64 ABI Draft 0.98 \u2013 September 27, 2006 -- 9:24.
 >
 > Unfortunately whoever wrote that down didn't think it through.  In
 > Figure 3.4 on page 20, %rbp is listed as "callee-saved register;
 > optionally used as frame pointer".  So %rbp can be used for anything, as
 > long as you save its contents and restore it before you return.

Null-terminating the call stack is too well-established practice to be
changed now.

In practice, %ebp either points to a call frame -- not necessarily the
most recent one -- or is null.  I don't think that having an optional
frame pointer mees you can use %ebp for anything random at all, but we
need to make a clarification request of the ABI.

 > Since it may be used for anything, it may contain 0 at any point in
 > the middle of the call stack.

 > So it is unusable as a stack trace termination condition.  The only
 > viable option is explicitly marking it as such in the CFI.
 >
 > Initializing %rbp to 0 in the outermost frame is sort of pointless
 > on amd64.

The right way to fix the ABI is to specify that %ebp mustn't be
[mis]used in this way, not to add a bunch more unwinder data.

Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Ulrich Drepper
Andrew Haley wrote:
> Null-terminating the call stack is too well-established practice to be
> changed now.

Which does not mean that the mistake should hold people back.  This is
just one of the mistakes in the x86-64 ABI.  It was copied from x86 and
it was wrong there already.


> In practice, %ebp either points to a call frame -- not necessarily the
> most recent one -- or is null.  I don't think that having an optional
> frame pointer mees you can use %ebp for anything random at all,

Of course it means that.


> The right way to fix the ABI is to specify that %ebp mustn't be
> [mis]used in this way, not to add a bunch more unwinder data.

Nope.  The right way is to specify things like backtraces with the
adequate mechanism.  I fully support adding the Dwarf3 unwinder
requirements.

--
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Andrew Haley
Ulrich Drepper writes:
 > Andrew Haley wrote:
 > > Null-terminating the call stack is too well-established practice to be
 > > changed now.
 >
 > Which does not mean that the mistake should hold people back.

Sure it does.  Not breaking things is an excellent reason, probably
one of the the best reasons you can have.

 > This is just one of the mistakes in the x86-64 ABI.  It was copied
 > from x86 and it was wrong there already.
 >
 > > In practice, %ebp either points to a call frame -- not necessarily the
 > > most recent one -- or is null.  I don't think that having an optional
 > > frame pointer mees you can use %ebp for anything random at all,
 >
 > Of course it means that.

Really?  Well, that's one interpretation.  I don't believe that,
though.  It's certainly an inconsistency in the specification, which
says that null-termination is supported, and this implies that you
can't put a zero in there.

 > > The right way to fix the ABI is to specify that %ebp mustn't be
 > > [mis]used in this way, not to add a bunch more unwinder data.
 >
 > Nope.  The right way is to specify things like backtraces with the
 > adequate mechanism.  I fully support adding the Dwarf3 unwinder
 > requirements.

"All of these" might be the right way to go.  That is, keep
null-terminating the stack, strengthen the rules about what you might
do with %ebp, and extend debuginfo.

Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Ulrich Drepper
Andrew Haley wrote:
> Sure it does.  Not breaking things is an excellent reason, probably
> one of the the best reasons you can have.

Nothing breaks if the responsible tools are updated in unison.


> Really?  Well, that's one interpretation.  I don't believe that,
> though.  It's certainly an inconsistency in the specification, which
> says that null-termination is supported, and this implies that you
> can't put a zero in there.

Again, this is just because the "authors" of the ABI didn't think.  x86
has the same problem.  ebp is freely used and not just for non-NULL
values.  Register's a scarce and I doubt  you'll find any support
introducing a register class which says that the register can only hold
non-zero value.


> "All of these" might be the right way to go.  That is, keep
> null-terminating the stack, strengthen the rules about what you might
> do with %ebp, and extend debuginfo.

The thread setup and the startup code certainly does initialize the
register with zero.  But this means nothing, the register can have zero
values in all kinds of other places.

--
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Jan Kratochvil-2
In reply to this post by Andrew Haley
On Tue, 12 Dec 2006 16:26:34 +0100, Andrew Haley wrote:
...
> It's certainly an inconsistency in the specification, which says that
> null-termination is supported, and this implies that you can't put a zero in
> there.

I tested now that you can put the zero there for both the libgcc unwinder and
for gdb(1) [tested only Fedora Core gdb-6.5-13.fc6],
attached, on x86_64 compile [gcc-4.1.1-30] and run by:
        gcc -o fp0-x86_64 -O9 -fomit-frame-pointer -Wall fp0-x86_64.c -ggdb3; ./fp0-x86_64

The output provided at the end of this mail.

Therefore I believe this sentence is wrong and it should be removed:
        http://www.x86-64.org/documentation/abi.pdf (draft 0.98) Page 28 (29/124)
        %rbp ... but the user code should mark the deepest stack frame by
             %setting the frame pointer to zero.

On the other hand the right stack terminator for libgcc unwinder is `PC == 0':
unwind-dw2.c:uw_frame_state_for ():
        if (context->ra == 0)
                return _URC_END_OF_STACK;

And gdb should just get updated to behave the same way.
libunwind already assumed end of stack on `PC == 0' before and I do not expect
any platform would consider `PC == 0' as a valid _return_address_ (which is
usually several bytes after any starting function address due to the call
instruction).

...
> "All of these" might be the right way to go.  That is, keep
> null-terminating the stack, strengthen the rules about what you might
> do with %ebp, and extend debuginfo.

For best compatibility null terminate the stack but by CFI and its indicated
return address. Do not use %rbp (frame pointer register) in any way (regarding
the stack termination condition).
Believe only CFI-specified CFA address, unrelated to %rbp content.


Regards,
Jan

------------------------------------------------------------------------------

GNU gdb Red Hat Linux (6.5-13.fc6rh)
This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/libthread_db.so.1".

(gdb) b 12
Breakpoint 1 at 0x40060b: file fp0-x86_64.c, line 12.
(gdb) r
Starting program: /root/jkratoch/redhat/unwind/fp0-x86_64

Breakpoint 1, backtracer () at fp0-x86_64.c:12
12 int i = backtrace (buf, 512);
(gdb) bt
#0  backtracer () at fp0-x86_64.c:12
#1  0x0000000000400739 in badone () at fp0-x86_64.c:31
#2  0x000000000040074e in main () at fp0-x86_64.c:47
(gdb) p/x $rbp
$1 = 0x7fff6ed61ca0
(gdb) up
#1  0x0000000000400739 in badone () at fp0-x86_64.c:31
31 backtracer ();
(gdb) p/x $rbp
$2 = 0x0
(gdb) c
Continuing.
dummy
5
/root/jkratoch/redhat/unwind/fp0-x86_64[0x40062c]
/root/jkratoch/redhat/unwind/fp0-x86_64[0x400739]
/root/jkratoch/redhat/unwind/fp0-x86_64[0x40074e]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x301e81da44]
/root/jkratoch/redhat/unwind/fp0-x86_64[0x400559]

Program received signal SIGSEGV, Segmentation fault.
0x0000000000400684 in backtracer () at fp0-x86_64.c:21
21 RAFA(1);
(gdb) q

fp0-x86_64.c (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Ian Lance Taylor-3
In reply to this post by Andrew Haley
Andrew Haley <[hidden email]> writes:

> In practice, %ebp either points to a call frame -- not necessarily the
> most recent one -- or is null.  I don't think that having an optional
> frame pointer mees you can use %ebp for anything random at all, but we
> need to make a clarification request of the ABI.

I don't see that as feasible.  If %ebp/%rbp may be used as a general
callee-saved register, then it can hold any value.  And permitting
%ebp/%rbp to hold any value is a very useful optimization in a
function which does not require a frame pointer, since it gives the
compiler an extra register to use.

If you want to require %ebp/%rbp to hold a non-zero value, then you
are effectively saying that this optimization is forbidden.  There is
no meaningful way to tell gcc "this is a general register, but you may
not store zero in it."  It would be a poor tradeoff to forbid that
optimization in order to provide better support for exception
handling: exception handling is supposed to be unusual.

Ian
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Andrew Haley
Ian Lance Taylor writes:
 > Andrew Haley <[hidden email]> writes:
 >
 > > In practice, %ebp either points to a call frame -- not necessarily the
 > > most recent one -- or is null.  I don't think that having an optional
 > > frame pointer mees you can use %ebp for anything random at all, but we
 > > need to make a clarification request of the ABI.
 >
 > I don't see that as feasible.  If %ebp/%rbp may be used as a general
 > callee-saved register, then it can hold any value.

Sure, we already know that, as has been clear.  The question is *if*
%rbp may be used as a general callee-saved register that can hold any
value.

 > And permitting %ebp/%rbp to hold any value is a very useful
 > optimization in a function which does not require a frame pointer,
 > since it gives the compiler an extra register to use.
 >
 > If you want to require %ebp/%rbp to hold a non-zero value, then you
 > are effectively saying that this optimization is forbidden.  There is
 > no meaningful way to tell gcc "this is a general register, but you may
 > not store zero in it."  It would be a poor tradeoff to forbid that
 > optimization in order to provide better support for exception
 > handling: exception handling is supposed to be unusual.

Sure, that's reasonable: it's a good reason to suggest that the ABI
spec (still in DRAFT state, I note!) might be changed.

Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Mark Kettenis
>  Ian Lance Taylor writes:
>   > Andrew Haley <[hidden email]> writes:
>   >
>   > > In practice, %ebp either points to a call frame -- not necessarily
>  the
>   > > most recent one -- or is null.  I don't think that having an optional
>   > > frame pointer mees you can use %ebp for anything random at all, but
>  we
>   > > need to make a clarification request of the ABI.
>   >
>   > I don't see that as feasible.  If %ebp/%rbp may be used as a general
>   > callee-saved register, then it can hold any value.
>
>  Sure, we already know that, as has been clear.  The question is *if*
>  %rbp may be used as a general callee-saved register that can hold any
>  value.

The amd64 ABI is specifically *designed* to allow this.

Mark

Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Michael Matz
In reply to this post by Andrew Haley
Hi,

On Tue, 12 Dec 2006, Andrew Haley wrote:

>  > > In practice, %ebp either points to a call frame -- not necessarily
>  > > the most recent one -- or is null.  I don't think that having an
>  > > optional frame pointer mees you can use %ebp for anything random at
>  > > all, but we need to make a clarification request of the ABI.
>  >
>  > I don't see that as feasible.  If %ebp/%rbp may be used as a general
>  > callee-saved register, then it can hold any value.
>
> Sure, we already know that, as has been clear.  The question is *if*
> %rbp may be used as a general callee-saved register that can hold any
> value.

Yes of course it was meant to be used such.  The ABI actually only gives a
recommendation that %rbp should be zero in the outermost frame, it's not a
must.  The ABI _requires_ proper .eh_frame descriptors when unwinding is
desired; so it's useless (and wrong) for any unwinder to look at %rbp and
determine if it should stop.

Alternatively (though not sanctioned by the ABI) all functions through
which unwinding is desired but for which no unwind info is created _have_
to use %rbp as frame pointer and not as general register.  In that case
the zeroing of %rbp would be a usable stop condition for functions without
unwind info.  But that's already outside the ABI.


Ciao,
Michael.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Michael Matz
In reply to this post by Jan Kratochvil-2
Hi,

On Mon, 11 Dec 2006, Jan Kratochvil wrote:

> currently (on x86_64) the gdb backtrace does not properly stop at the outermost
> frame:
>
> #3  0x00000036ddb0610a in start_thread () from /lib64/tls/libpthread.so.0
> #4  0x00000036dd0c68c3 in clone () from /lib64/tls/libc.so.6
> #5  0x0000000000000000 in ?? ()
>
> Currently it relies only on clearing %rbp (0x0000000000000000 above is
> unrelated to it, it got read from uninitialized memory).
>
> http://sourceware.org/ml/gdb/2004-08/msg00060.html suggests frame
> pointer 0x0 should be enough for a debugger not finding CFI to stop
> unwinding, still it is a heuristic.  In the -fno-frame-pointer compiled
> code there is no indication the frame pointer register became a regular
> one and 0x0 is its valid value.

Right.  Unwinding through functions (without frame pointer) requires CFI.  
If there is CFI for a function the unwinder must not look at %rbp for stop
condition.  If there's no CFI for a function it can't be unwound (strictly
per ABI).  If one relaxes that and wants to unwind through CFI-less
functions it has to have a frame pointer.  In that case zero in that frame
pointer could indicate the outermost frame (_if_ the suggestion in the ABI
is adhered to, which noone is required to).


Ciao,
Michael.
Reply | Threaded
Open this post in threaded view
|

Re: Unwinding CFI gcc practice of assumed `same value' regs

Michael Matz
In reply to this post by Ulrich Drepper
Hi,

On Tue, 12 Dec 2006, Ulrich Drepper wrote:

> > Really?  Well, that's one interpretation.  I don't believe that,
> > though.  It's certainly an inconsistency in the specification, which
> > says that null-termination is supported, and this implies that you
> > can't put a zero in there.
>
> Again, this is just because the "authors" of the ABI didn't think.

[Blaeh, Ulrich talk] No, I think it's because the "readers" of the ABI
can't read.


Ciao,
Michael.
Reply | Threaded
Open this post in threaded view
|

[PATCH] Re: Unwinding CFI gcc practice of assumed `same value' regs

Jan Kratochvil-2
In reply to this post by Roland McGrath
Hi,

On Tue, 12 Dec 2006 16:52:33 +0100, Jakub Jelinek wrote:
...
> Here is something that would handle by default same_value retaddr_column:
[ http://sources.redhat.com/ml/gdb/2006-12/msg00100.html ]

Thanks for this backward compatible glibc unwinder patch.  I wish to have it
accepted as a step in preparing the environment to use `.cfi_undefined PC'
sometimes in the future.

Attaching patch for current glibc CVS which removes the `.cfi_undefined PC'
unwinder handling requirement but which provides explicit return address 0 from
the `__clone' function.  Currently the 0 is already present there but it is
uninitialized value out of some TLS or 'struct pthread' area (did not check).

Attaching patch for current gdb CVS to properly terminate on return address 0.
The check was already present there but it got applied one backward step later.

I hope these three patches are be 100% reliable and also backward compatible.


Regards,
Jan

glibc-unwind-compat2.patch (2K) Download Attachment
gdb-bt-clone-stop.patch (4K) Download Attachment