Crash when re-initializing as static library

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Crash when re-initializing as static library

Klaus Fischer
Dear pthreads-win32 developers,

I have experienced a crash when building pthreads-win32 as static
library and re-initializing it using the following sequence:

- pthread_win32_process_attach_np()
- pthread_win32_process_detach_np()
- pthread_win32_process_attach_np()

The global variable ptw32_threadReuseTop still points to memory used
between the first attach/detach run, but this memory was already freed
in function ptw32_processTerminate(), which was called during detaching.

When using e.g. pthread_self() afterwards, the global
ptw32_threadReuseTop now points to invalid memory, causing an access
violation writing to that memory location.

A simple code change fixed that problem by assigning the default value
PTW32_THREAD_REUSE_EMPTY to that global variable at the end of function
ptw32_processTerminate(), after the while loop freeing all still
allocated thread handles.

A better way to fix that would probably be to initialize all the library
globals of global.c during the attaching stage. In a static library,
those globals are only initialized once when the process starts, but
_should_ be re-initialized on every attach.

I know this is not a concern when using this library as dynamic library,
but since there is an option to use it as static library and other
people also use it that way according to the mailing list, it would be
great if it could survive multiple re-initializations.

Thanks in advance,
Klaus

--
_________________________________________________________

Dipl. Inf. Klaus FISCHER  -  Software Engineer

     mailto: [hidden email]
_________________________________________________________
Reply | Threaded
Open this post in threaded view
|

Re: Crash when re-initializing as static library

Ross Johnson-2
Hi,

Applications statically linked with current versions of the library no
longer need to call those routines explicitly. From README.NONPORTABLE:

     These functions contain the code normally run via DllMain
     when the library is used as a dll. As of version 2.9.0 of the
     library, static builds using either MSC or GCC will call
     pthread_win32_process_* automatically at application startup and
     exit respectively.

But you are also detaching and reattaching within the same process,
which is unintended use. Is this how you expect to use the library or
are you just analysing?

On 7/11/2013 4:53 AM, Klaus Fischer wrote:

> Dear pthreads-win32 developers,
>
> I have experienced a crash when building pthreads-win32 as static
> library and re-initializing it using the following sequence:
>
> - pthread_win32_process_attach_np()
> - pthread_win32_process_detach_np()
> - pthread_win32_process_attach_np()
>
> The global variable ptw32_threadReuseTop still points to memory used
> between the first attach/detach run, but this memory was already freed
> in function ptw32_processTerminate(), which was called during detaching.
>
> When using e.g. pthread_self() afterwards, the global
> ptw32_threadReuseTop now points to invalid memory, causing an access
> violation writing to that memory location.
>
> A simple code change fixed that problem by assigning the default value
> PTW32_THREAD_REUSE_EMPTY to that global variable at the end of
> function ptw32_processTerminate(), after the while loop freeing all
> still allocated thread handles.
>
> A better way to fix that would probably be to initialize all the
> library globals of global.c during the attaching stage. In a static
> library, those globals are only initialized once when the process
> starts, but _should_ be re-initialized on every attach.
>
> I know this is not a concern when using this library as dynamic
> library, but since there is an option to use it as static library and
> other people also use it that way according to the mailing list, it
> would be great if it could survive multiple re-initializations.
>
> Thanks in advance,
> Klaus
>

Reply | Threaded
Open this post in threaded view
|

Re: Crash when re-initializing as static library

Klaus Fischer
In reply to this post by Klaus Fischer
Let me give you a little background in order to understand my situation:

I'm actually using the Portable SDK for UPnP Devices (libupnp), which in
turn relies on pthreads. I'm developing cross-plattform (Windows/Linux),
so I need Pthread-Win32 (henceforth called PTW32) to enable libupnp to
run under Windows.

libupnp in turn initializes and deinitializes PTW32 every time libupnp
itself gets initialized/deinitialized. De-/initialization of PTW32 is
explicitly done via pthread_win32_process_* when libupnp is compiled for
WIN32 and PTW32 is used as a static library. This is still done that way
in the latest libupnp release (1.6.18).

One of my first tests was to repeatedly call UpnpInit()/UpnpFinish() to
check for general stability, which is also always a good test for
software cleanliness according to my experience; unfortunately, this
immediately resulted in the access violation in PTW32.

It looks like libupnp still has to adapt to the changed calling behavior
of PTW32, although they had plenty of time for it (the latest libupnp
release is from January 2013, while the latest PTW32 is from May 2012).
They probably overlooked the change in calling pthread_win32_process_*;
I will send them a note for it.

Nevertheless, I will continue to use my locally patched version of PTW32
until either libupnp adapted to the changed PTW32 behavior or PTW32 is
available with initialization of globals during the attaching phase.
I've ran into the same problem in some of my libraries too, and decided
to give my globals a proper initialization in an Init()-function to
avoid such issues during repeated de-/initialization. It would be nice
if you could consider doing this in PTW32 too, just for the sake of
backwards-compatibility (if this actually worked in pre-2.9.0 versions).

Best regards,
Klaus


 > Hi,
 >
 > Applications statically linked with current versions of the library
no longer need to call those routines explicitly. From README.NONPORTABLE:
 >
 >     These functions contain the code normally run via DllMain
 >     when the library is used as a dll. As of version 2.9.0 of the
 >     library, static builds using either MSC or GCC will call
 >     pthread_win32_process_* automatically at application startup and
 >     exit respectively.
 >
 > But you are also detaching and reattaching within the same process,
which is unintended use. Is this how you expect to use the library or
are you just analysing?
 >
 >> On 7/11/2013 4:53 AM, Klaus Fischer wrote:
 >> Dear pthreads-win32 developers,
 >>
 >> I have experienced a crash when building pthreads-win32 as static
library and re-initializing it using the following sequence:
 >>
 >> - pthread_win32_process_attach_np()
 >> - pthread_win32_process_detach_np()
 >> - pthread_win32_process_attach_np()
 >>
 >> The global variable ptw32_threadReuseTop still points to memory used
between the first attach/detach run, but this memory was already freed
in function ptw32_processTerminate(), which was called during detaching.
 >>
 >> When using e.g. pthread_self() afterwards, the global
ptw32_threadReuseTop now points to invalid memory, causing an access
violation writing to that memory location.
 >>
 >> A simple code change fixed that problem by assigning the default
value PTW32_THREAD_REUSE_EMPTY to that global variable at the end of
function ptw32_processTerminate(), after the while loop freeing all
still allocated thread handles.
 >>
 >> A better way to fix that would probably be to initialize all the
library globals of global.c during the attaching stage. In a static
library, those globals are only initialized once when the process
starts, but _should_ be re-initialized on every attach.
 >>
 >> I know this is not a concern when using this library as dynamic
library, but since there is an option to use it as static library and
other people also use it that way according to the mailing list, it
would be great if it could survive multiple re-initializations.
 >>
 >> Thanks in advance,
 >> Klaus
Reply | Threaded
Open this post in threaded view
|

Re: Crash when re-initializing as static library

Ross Johnson-2
Hi Klaus,

Thanks for providing the extra context. I'll apply your suggested
change/s, probably early next week.

Regards.
Ross

On 8/11/2013 1:19 AM, Klaus Fischer wrote:

> Let me give you a little background in order to understand my situation:
>
> I'm actually using the Portable SDK for UPnP Devices (libupnp), which
> in turn relies on pthreads. I'm developing cross-plattform
> (Windows/Linux), so I need Pthread-Win32 (henceforth called PTW32) to
> enable libupnp to run under Windows.
>
> libupnp in turn initializes and deinitializes PTW32 every time libupnp
> itself gets initialized/deinitialized. De-/initialization of PTW32 is
> explicitly done via pthread_win32_process_* when libupnp is compiled
> for WIN32 and PTW32 is used as a static library. This is still done
> that way in the latest libupnp release (1.6.18).
>
> One of my first tests was to repeatedly call UpnpInit()/UpnpFinish()
> to check for general stability, which is also always a good test for
> software cleanliness according to my experience; unfortunately, this
> immediately resulted in the access violation in PTW32.
>
> It looks like libupnp still has to adapt to the changed calling
> behavior of PTW32, although they had plenty of time for it (the latest
> libupnp release is from January 2013, while the latest PTW32 is from
> May 2012). They probably overlooked the change in calling
> pthread_win32_process_*; I will send them a note for it.
>
> Nevertheless, I will continue to use my locally patched version of
> PTW32 until either libupnp adapted to the changed PTW32 behavior or
> PTW32 is available with initialization of globals during the attaching
> phase. I've ran into the same problem in some of my libraries too, and
> decided to give my globals a proper initialization in an
> Init()-function to avoid such issues during repeated
> de-/initialization. It would be nice if you could consider doing this
> in PTW32 too, just for the sake of backwards-compatibility (if this
> actually worked in pre-2.9.0 versions).
>
> Best regards,
> Klaus
>
>
> > Hi,
> >
> > Applications statically linked with current versions of the library
> no longer need to call those routines explicitly. From
> README.NONPORTABLE:
> >
> >     These functions contain the code normally run via DllMain
> >     when the library is used as a dll. As of version 2.9.0 of the
> >     library, static builds using either MSC or GCC will call
> >     pthread_win32_process_* automatically at application startup and
> >     exit respectively.
> >
> > But you are also detaching and reattaching within the same process,
> which is unintended use. Is this how you expect to use the library or
> are you just analysing?
> >
> >> On 7/11/2013 4:53 AM, Klaus Fischer wrote:
> >> Dear pthreads-win32 developers,
> >>
> >> I have experienced a crash when building pthreads-win32 as static
> library and re-initializing it using the following sequence:
> >>
> >> - pthread_win32_process_attach_np()
> >> - pthread_win32_process_detach_np()
> >> - pthread_win32_process_attach_np()
> >>
> >> The global variable ptw32_threadReuseTop still points to memory
> used between the first attach/detach run, but this memory was already
> freed in function ptw32_processTerminate(), which was called during
> detaching.
> >>
> >> When using e.g. pthread_self() afterwards, the global
> ptw32_threadReuseTop now points to invalid memory, causing an access
> violation writing to that memory location.
> >>
> >> A simple code change fixed that problem by assigning the default
> value PTW32_THREAD_REUSE_EMPTY to that global variable at the end of
> function ptw32_processTerminate(), after the while loop freeing all
> still allocated thread handles.
> >>
> >> A better way to fix that would probably be to initialize all the
> library globals of global.c during the attaching stage. In a static
> library, those globals are only initialized once when the process
> starts, but _should_ be re-initialized on every attach.
> >>
> >> I know this is not a concern when using this library as dynamic
> library, but since there is an option to use it as static library and
> other people also use it that way according to the mailing list, it
> would be great if it could survive multiple re-initializations.
> >>
> >> Thanks in advance,
> >> Klaus

Reply | Threaded
Open this post in threaded view
|

Re: Crash when re-initializing as static library

Klaus Fischer
Hi Ross,

great, it's nice to hear that! :)

I am currently also in touch with Marcelo Roberto Jimenez from libupnp
over at Sourceforge.net regarding this issue:

https://sourceforge.net/p/pupnp/bugs/119/

He sent me a patch for the latest libupnp that eliminated the calls to
pthread_win32_process_*() inside libupnp when using PTW32 as static
library as you suggested, as long as I had a correct understanding of
your quoted passage from README.NONPORTABLE. I have applied that patch,
but now I am facing the problem that libupnp hangs indefinitely in a
call to pthread_cond_wait().

It looks to me that pthread_win32_process_attach_np() was not called
automatically at application start-up. When my application hits the
breakpoint right before the call to the blocking pthread_cond_wait(),
the global variable "ptw32_processInitialized" is still 0, although it
should have been set to 1 inside pthread_win32_process_attach_np().

I actually don't understand how that function can be called
automatically at application start-up when building PTW32 as a static
library. How is this achieved? AFAIK there is no DllMain-equivalent for
static libraries. Am I wrong here? Or is it maybe that I'm using Visual
Studio 2005 and it doesn't support that? Are there any options to
activate in the project solution for doing so? I should mention that I'm
not using the Visual Studio 6.0 project files delivered with PTW32,
since Visual Studio 2005 tells me it "Cannot load the project due to a
corrupt project file.".

Thanks for any help here, and also for your quick and kind response!

Best regards,
Klaus


On 08.11.2013 00:47, Ross Johnson wrote:

> Hi Klaus,
>
> Thanks for providing the extra context. I'll apply your suggested
> change/s, probably early next week.
>
> Regards.
> Ross
>
> On 8/11/2013 1:19 AM, Klaus Fischer wrote:
>> Let me give you a little background in order to understand my situation:
>>
>> I'm actually using the Portable SDK for UPnP Devices (libupnp), which
>> in turn relies on pthreads. I'm developing cross-plattform
>> (Windows/Linux), so I need Pthread-Win32 (henceforth called PTW32) to
>> enable libupnp to run under Windows.
>>
>> libupnp in turn initializes and deinitializes PTW32 every time libupnp
>> itself gets initialized/deinitialized. De-/initialization of PTW32 is
>> explicitly done via pthread_win32_process_* when libupnp is compiled
>> for WIN32 and PTW32 is used as a static library. This is still done
>> that way in the latest libupnp release (1.6.18).
>>
>> One of my first tests was to repeatedly call UpnpInit()/UpnpFinish()
>> to check for general stability, which is also always a good test for
>> software cleanliness according to my experience; unfortunately, this
>> immediately resulted in the access violation in PTW32.
>>
>> It looks like libupnp still has to adapt to the changed calling
>> behavior of PTW32, although they had plenty of time for it (the latest
>> libupnp release is from January 2013, while the latest PTW32 is from
>> May 2012). They probably overlooked the change in calling
>> pthread_win32_process_*; I will send them a note for it.
>>
>> Nevertheless, I will continue to use my locally patched version of
>> PTW32 until either libupnp adapted to the changed PTW32 behavior or
>> PTW32 is available with initialization of globals during the attaching
>> phase. I've ran into the same problem in some of my libraries too, and
>> decided to give my globals a proper initialization in an
>> Init()-function to avoid such issues during repeated
>> de-/initialization. It would be nice if you could consider doing this
>> in PTW32 too, just for the sake of backwards-compatibility (if this
>> actually worked in pre-2.9.0 versions).
>>
>> Best regards,
>> Klaus
>>
>>
>> > Hi,
>> >
>> > Applications statically linked with current versions of the library
>> no longer need to call those routines explicitly. From
>> README.NONPORTABLE:
>> >
>> >     These functions contain the code normally run via DllMain
>> >     when the library is used as a dll. As of version 2.9.0 of the
>> >     library, static builds using either MSC or GCC will call
>> >     pthread_win32_process_* automatically at application startup and
>> >     exit respectively.
>> >
>> > But you are also detaching and reattaching within the same process,
>> which is unintended use. Is this how you expect to use the library or
>> are you just analysing?
>> >
>> >> On 7/11/2013 4:53 AM, Klaus Fischer wrote:
>> >> Dear pthreads-win32 developers,
>> >>
>> >> I have experienced a crash when building pthreads-win32 as static
>> library and re-initializing it using the following sequence:
>> >>
>> >> - pthread_win32_process_attach_np()
>> >> - pthread_win32_process_detach_np()
>> >> - pthread_win32_process_attach_np()
>> >>
>> >> The global variable ptw32_threadReuseTop still points to memory
>> used between the first attach/detach run, but this memory was already
>> freed in function ptw32_processTerminate(), which was called during
>> detaching.
>> >>
>> >> When using e.g. pthread_self() afterwards, the global
>> ptw32_threadReuseTop now points to invalid memory, causing an access
>> violation writing to that memory location.
>> >>
>> >> A simple code change fixed that problem by assigning the default
>> value PTW32_THREAD_REUSE_EMPTY to that global variable at the end of
>> function ptw32_processTerminate(), after the while loop freeing all
>> still allocated thread handles.
>> >>
>> >> A better way to fix that would probably be to initialize all the
>> library globals of global.c during the attaching stage. In a static
>> library, those globals are only initialized once when the process
>> starts, but _should_ be re-initialized on every attach.
>> >>
>> >> I know this is not a concern when using this library as dynamic
>> library, but since there is an option to use it as static library and
>> other people also use it that way according to the mailing list, it
>> would be great if it could survive multiple re-initializations.
>> >>
>> >> Thanks in advance,
>> >> Klaus
>

--
_________________________________________________________

Dipl. Inf. Klaus FISCHER  -  Software Engineer

     mailto: [hidden email]
TARA Systems GmbH       |  Tel:      +49 89 747121 37
Gmunder Str. 53         |  Fax:      +49 89 747121 20
81379 Munich - Germany  |  http://www.tara-systems.de
_________________________________________________________

Registered as HRB 103149 at Munich Local Court
Managing Director: Dipl.Ing. A. Wass
Reply | Threaded
Open this post in threaded view
|

Re: Crash when re-initializing as static library

Klaus Fischer
Hi Ross,

I took the liberty of including your last mail response that somehow
didn't make it to the mailing list, so the context of my answer becomes
more clear.

Sorry for the delay in response; my focus shifted to other things, and
it also took me a bit to figure out how that autostatic.c file works
that provides automatic function calls for static libraries.

For unknown reasons I couldn't get it to work in my Visual Studio 2005
project though, so I'm actually using the Nmakefile delivered with the
sources now, which I should've done in the first place. Automatic
initialization works now as expected when compiling PTW32 as static
library. I consider it a bit of a hack though, since it's heavily
compiler dependent; I don't know what Intel's icc or clang would make of it.

Anyway, thanks for your help!

Best regards,
Klaus


On 08.11.2013 14:26, Ross Johnson wrote:

 > See the autostatic.c source file. The techniques there have been
 > borrowed from elsewhere so we did not invent it.
 >
 > We also have makefile targets that run the tests with static linking
 > and we have not seen problems.


On 08.11.2013 11:39, Klaus Fischer wrote:

> Hi Ross,
>
> great, it's nice to hear that! :)
>
> I am currently also in touch with Marcelo Roberto Jimenez from libupnp
> over at Sourceforge.net regarding this issue:
>
> https://sourceforge.net/p/pupnp/bugs/119/
>
> He sent me a patch for the latest libupnp that eliminated the calls to
> pthread_win32_process_*() inside libupnp when using PTW32 as static
> library as you suggested, as long as I had a correct understanding of
> your quoted passage from README.NONPORTABLE. I have applied that patch,
> but now I am facing the problem that libupnp hangs indefinitely in a
> call to pthread_cond_wait().
>
> It looks to me that pthread_win32_process_attach_np() was not called
> automatically at application start-up. When my application hits the
> breakpoint right before the call to the blocking pthread_cond_wait(),
> the global variable "ptw32_processInitialized" is still 0, although it
> should have been set to 1 inside pthread_win32_process_attach_np().
>
> I actually don't understand how that function can be called
> automatically at application start-up when building PTW32 as a static
> library. How is this achieved? AFAIK there is no DllMain-equivalent for
> static libraries. Am I wrong here? Or is it maybe that I'm using Visual
> Studio 2005 and it doesn't support that? Are there any options to
> activate in the project solution for doing so? I should mention that I'm
> not using the Visual Studio 6.0 project files delivered with PTW32,
> since Visual Studio 2005 tells me it "Cannot load the project due to a
> corrupt project file.".
>
> Thanks for any help here, and also for your quick and kind response!
>
> Best regards,
> Klaus
>
>
> On 08.11.2013 00:47, Ross Johnson wrote:
>> Hi Klaus,
>>
>> Thanks for providing the extra context. I'll apply your suggested
>> change/s, probably early next week.
>>
>> Regards.
>> Ross
>>
>> On 8/11/2013 1:19 AM, Klaus Fischer wrote:
>>> Let me give you a little background in order to understand my situation:
>>>
>>> I'm actually using the Portable SDK for UPnP Devices (libupnp), which
>>> in turn relies on pthreads. I'm developing cross-plattform
>>> (Windows/Linux), so I need Pthread-Win32 (henceforth called PTW32) to
>>> enable libupnp to run under Windows.
>>>
>>> libupnp in turn initializes and deinitializes PTW32 every time libupnp
>>> itself gets initialized/deinitialized. De-/initialization of PTW32 is
>>> explicitly done via pthread_win32_process_* when libupnp is compiled
>>> for WIN32 and PTW32 is used as a static library. This is still done
>>> that way in the latest libupnp release (1.6.18).
>>>
>>> One of my first tests was to repeatedly call UpnpInit()/UpnpFinish()
>>> to check for general stability, which is also always a good test for
>>> software cleanliness according to my experience; unfortunately, this
>>> immediately resulted in the access violation in PTW32.
>>>
>>> It looks like libupnp still has to adapt to the changed calling
>>> behavior of PTW32, although they had plenty of time for it (the latest
>>> libupnp release is from January 2013, while the latest PTW32 is from
>>> May 2012). They probably overlooked the change in calling
>>> pthread_win32_process_*; I will send them a note for it.
>>>
>>> Nevertheless, I will continue to use my locally patched version of
>>> PTW32 until either libupnp adapted to the changed PTW32 behavior or
>>> PTW32 is available with initialization of globals during the attaching
>>> phase. I've ran into the same problem in some of my libraries too, and
>>> decided to give my globals a proper initialization in an
>>> Init()-function to avoid such issues during repeated
>>> de-/initialization. It would be nice if you could consider doing this
>>> in PTW32 too, just for the sake of backwards-compatibility (if this
>>> actually worked in pre-2.9.0 versions).
>>>
>>> Best regards,
>>> Klaus
>>>
>>>
>>> > Hi,
>>> >
>>> > Applications statically linked with current versions of the library
>>> no longer need to call those routines explicitly. From
>>> README.NONPORTABLE:
>>> >
>>> >     These functions contain the code normally run via DllMain
>>> >     when the library is used as a dll. As of version 2.9.0 of the
>>> >     library, static builds using either MSC or GCC will call
>>> >     pthread_win32_process_* automatically at application startup and
>>> >     exit respectively.
>>> >
>>> > But you are also detaching and reattaching within the same process,
>>> which is unintended use. Is this how you expect to use the library or
>>> are you just analysing?
>>> >
>>> >> On 7/11/2013 4:53 AM, Klaus Fischer wrote:
>>> >> Dear pthreads-win32 developers,
>>> >>
>>> >> I have experienced a crash when building pthreads-win32 as static
>>> library and re-initializing it using the following sequence:
>>> >>
>>> >> - pthread_win32_process_attach_np()
>>> >> - pthread_win32_process_detach_np()
>>> >> - pthread_win32_process_attach_np()
>>> >>
>>> >> The global variable ptw32_threadReuseTop still points to memory
>>> used between the first attach/detach run, but this memory was already
>>> freed in function ptw32_processTerminate(), which was called during
>>> detaching.
>>> >>
>>> >> When using e.g. pthread_self() afterwards, the global
>>> ptw32_threadReuseTop now points to invalid memory, causing an access
>>> violation writing to that memory location.
>>> >>
>>> >> A simple code change fixed that problem by assigning the default
>>> value PTW32_THREAD_REUSE_EMPTY to that global variable at the end of
>>> function ptw32_processTerminate(), after the while loop freeing all
>>> still allocated thread handles.
>>> >>
>>> >> A better way to fix that would probably be to initialize all the
>>> library globals of global.c during the attaching stage. In a static
>>> library, those globals are only initialized once when the process
>>> starts, but _should_ be re-initialized on every attach.
>>> >>
>>> >> I know this is not a concern when using this library as dynamic
>>> library, but since there is an option to use it as static library and
>>> other people also use it that way according to the mailing list, it
>>> would be great if it could survive multiple re-initializations.
>>> >>
>>> >> Thanks in advance,
>>> >> Klaus
>>
>

--
_________________________________________________________

Dipl. Inf. Klaus FISCHER  -  Software Engineer

     mailto: [hidden email]
TARA Systems GmbH       |  Tel:      +49 89 747121 37
Gmunder Str. 53         |  Fax:      +49 89 747121 20
81379 Munich - Germany  |  http://www.tara-systems.de
_________________________________________________________

Registered as HRB 103149 at Munich Local Court
Managing Director: Dipl.Ing. A. Wass