how GDB use ptrace to return from a function

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

how GDB use ptrace to return from a function

Yubin Ruan
Hi GDB developer ;-)

I am writing a toy debugger and currently looking into the GDB source
because I want to know: after setting the tracee's registers and
trying to let it execute a function with ptrace(PTRACE_CONT, ...), how
can the tracee return to the tracer?

Currently I manipulate the tracee's stack and place a NULL return
address there (I am on X86), so that after ptrace(PTRACE_CONT, ...),
the tracee will execute a function and return, at which point a SIGSEV
is generated (because the return address is NULL), so tracee will be
caught by the tracer again.

I don't know whether GDB is using this kind of technique. If anyone
know that, can you enlighten me, and probably point me to the source?

Yubin
Reply | Threaded
Open this post in threaded view
|

Re: how GDB use ptrace to return from a function

Joel Brobecker
> I don't know whether GDB is using this kind of technique. If anyone
> know that, can you enlighten me, and probably point me to the source?

GDB sets the call up so that the return address is at specific
location (usually the program's entry point, but that's arch-
dependent), and then places a breakpoint at that address. It
then knows, when receiving the corresponding breakpoint event,
that a breakpoint at that address corresponds to the end of
the function that we called.

--
Joel
Reply | Threaded
Open this post in threaded view
|

Re: how GDB use ptrace to return from a function

Yubin Ruan
Thanks Joel,

2017-11-18 0:10 GMT+08:00 Joel Brobecker <[hidden email]>:
>> I don't know whether GDB is using this kind of technique. If anyone
>> know that, can you enlighten me, and probably point me to the source?
>
> GDB sets the call up so that the return address is at specific
> location (usually the program's entry point, but that's arch-
> dependent), and then places a breakpoint at that address. It
> then knows, when receiving the corresponding breakpoint event,
> that a breakpoint at that address corresponds to the end of
> the function that we called.

what break point events are common for X86?

Yubin
Reply | Threaded
Open this post in threaded view
|

Re: how GDB use ptrace to return from a function

Joel Brobecker
> what break point events are common for X86?

IIRC, the breakpoint instruction on x86 is the int3 instruction.
It should generate a SIGTRAP upon execution, just like any user-
inserted breakpoints.

--
Joel
Reply | Threaded
Open this post in threaded view
|

Re: how GDB use ptrace to return from a function

Yubin Ruan
2017-11-18 12:03 GMT+08:00 Joel Brobecker <[hidden email]>:
>> what break point events are common for X86?
>
> IIRC, the breakpoint instruction on x86 is the int3 instruction.
> It should generate a SIGTRAP upon execution, just like any user-
> inserted breakpoints.

Thanks Joel ;-)

Yubin
Reply | Threaded
Open this post in threaded view
|

Re: how GDB use ptrace to return from a function

Jan Kratochvil-2
In reply to this post by Joel Brobecker
On Sat, 18 Nov 2017 05:03:11 +0100, Joel Brobecker wrote:
> > what break point events are common for X86?
>
> IIRC, the breakpoint instruction on x86 is the int3 instruction.
> It should generate a SIGTRAP upon execution, just like any user-
> inserted breakpoints.

A toy debugger should have available also a hardware breakpoint (hbreak) using
the hardware registers commonly used only for hardware watchpoints.

Then the return address can be arbitrary as the debugger does not have to
write anything into that address.


Jan