Static libraries initialization

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Static libraries initialization

Ramiro Polla-2
Hello,

pthreads-win32 requires programs that use the library statically to call
some (de)initialization code, which would be the same code called by
DllMain(). There are plenty of messages regarding this on the mailinglist.

This makes the library not very developer friendly. It causes code
duplication for every program that wants to use the library statically,
#ifdef mess and harder configuration checks.

1- Why isn't this initialization done by pthreads-win32 itself? All
    pthread_* functions could have an if(!initialized) {...} block before
    running any code that depends on it.
2- Static pthreads-win32 libraries should at least check if they were
    properly initialized before allowing pthread_* functions to run.
    Returning an error is far better than having the program crash.

Ramiro Polla
Reply | Threaded
Open this post in threaded view
|

Re: Static libraries initialization

Ross Johnson-2
Hi Ramiro,

Re not replying to the list - sorry, I didn't notice that I'd hit the
wrong button.

Ramiro Polla wrote:

> Hello Ross,
>
> Ross Johnson wrote:
>> Hi Ramiro,
>>
>> I agree that requiring the attach/detach routines is clunky,
>> however...see my comments inline.
>>
>> Ramiro Polla wrote:
>>> pthreads-win32 requires programs that use the library statically to
>>> call some (de)initialization code, which would be the same code
>>> called by DllMain(). There are plenty of messages regarding this on
>>> the mailinglist.
>>>
>>> 1- Why isn't this initialization done by pthreads-win32 itself? All
>>>    pthread_* functions could have an if(!initialized) {...} block
>>> before
>>>    running any code that depends on it.
>> It could be done this way for initialisation, but you'd still need to
>> run the process detach routine explicitly before exiting
>
> Can't this be done with atexit()?
I wasn't aware of that function, or had forgotten that it exits. I don't
do a lot of C programming.

Now there is only the thread detach cleanup to deal with. I haven't
found an equivalent to atexit() for win32 native threads. Is there one,
or is there another way to make this automatic? The problem is as follows:-

Threads created via pthread_create() are cleaned up automatically, even
when the library is statically linked (something I keep forgetting was
added). However, one feature of pthreads-win32 is that it allows win32
native threads to call pthreads routines, which means fully symmetric
interaction is possible between win32 and POSIX threads. I don't want
this to be configurable or optional or dependent upon using the dll.

To make this feature possible, a [detached] POSIX thread id is
automatically created for the win32 thread the first time it's needed
and is retained for the life of the thread. When statically linked,
win32 threads must call pthread_win32_thread_detach_np() explicitly
before exiting in order to cleanup any "implicit" POSIX handles or other
POSIX resources such as TSD etc.

>> and I may have thought a long time ago that if you need one you may
>> as well have the other, if only to make it more obvious and "in your
>> face".
>
>> It also allows the developer to choose when initialisation occurs
>> and, although most may not care, some might.
>
> It could be configurable.
Ignore that issue, I was just brainstorming reasons to maintain the
status quo.
>
>> For some routines, the additional overhead is not negligible, and
>> they tend to be the most heavily used routines.
>
> The overhead of one if()? If the branch prediction sets the if() to be
> unlikely, there should be 1 memory access and 1 cmp function overhead.
> It shouldn't even disturb the instruction pipeline.
Yes, it's almost negligible and I use the same method to initialise
static POSIX mutexes, cond vars, etc., although this one is slightly
worse in that the data isn't necessarily in cache already or local.
IIRC, we need to use an Interlocked routine to compare the value, not
for atomicity necessarily but so that we get the memory barrier effects
right. And I've found that these calls can be relatively slow due to bus
locking. I'm also still assuming that every single routine needs this
overhead, but that may not be necessary in the end.

Nevertheless, if you want to make this overhead configurable as you've
described below that's ok with me.
>
>> There are a couple of other impediments but nothing too major - they
>> just all add up.
One was the thread detach already mentioned earlier and another, a
function defined as a macro, I no longer consider a problem because it
need not trigger a process attach.

>
>>> 2- Static pthreads-win32 libraries should at least check if they were
>>>    properly initialized before allowing pthread_* functions to run.
>>>    Returning an error is far better than having the program crash.
>> Ideally yes, I agree, but presumably the crash occurs pretty soon in
>> development and once fixed is fixed forever. So, in the interests of
>> keeping overheads within the library to a minimum I think it's
>> reasonable not to check. This is one of those APIs that absolutely
>> needs to be as efficient as possible.
>>
>> One last thing: the library was originally intended to be used only
>> as a dll, and static linking requirements have been given only
>> minimal attention.
>
> Would it be ok to add another build option such as GC-static-autoinit?
> Users should expect it to have a small overhead, at the price of not
> needing to alter their code nor the build system to work with static
> or shared libraries.
If you're offering to provide the patches I'll certainly try to include
them. Perhaps you could also add an option to build the benchmarks in
the tests folder using static linking and perhaps also compare with the
dll versions.
>
> Oh, and apparently you only replied to me. Could this be discussed on
> the mailinglist?
Oops.

Regards.
Ross
>
> Ramiro Polla

Reply | Threaded
Open this post in threaded view
|

Re: Static libraries initialization

Ross Johnson-2
Ross Johnson wrote:

>>
>> The overhead of one if()? If the branch prediction sets the if() to
>> be unlikely, there should be 1 memory access and 1 cmp function
>> overhead. It shouldn't even disturb the instruction pipeline.
> Yes, it's almost negligible and I use the same method to initialise
> static POSIX mutexes, cond vars, etc., although this one is slightly
> worse in that the data isn't necessarily in cache already or local.
> IIRC, we need to use an Interlocked routine to compare the value, not
> for atomicity necessarily but so that we get the memory barrier
> effects right. And I've found that these calls can be relatively slow
> due to bus locking. I'm also still assuming that every single routine
> needs this overhead, but that may not be necessary in the end.
I'll retract my interlocked/memory fence comments - they aren't needed
here. I keep forgetting it's just the first quick check, and the second
thorough check will only ever be performed once or twice.

Ross

Reply | Threaded
Open this post in threaded view
|

Re: Static libraries initialization

Roland Schwarz
In reply to this post by Ross Johnson-2


Ross Johnson wrote:
> Now there is only the thread detach cleanup to deal with. I haven't
> found an equivalent to atexit() for win32 native threads. Is there one,
> or is there another way to make this automatic? The problem is as follows:-

Sorry for dropping in. I did not follow this list for a while, but the
above question triggered me.

While I was solving the cleanup problem for the boost TLS implementation
for static linking I came over a not so well known feature of the PE
file format:

The operating system calls into a hook on every detaching thread similar
to DllMain, but also for statically linked libs.

Withou going into detail: This is compiler dependent and has been made
working for MSVC and gcc compilers. Borland unfortunately cannot
be made to work. :-(

You might look for the details in the boost threading lib, or contact
me if you need more details.

Ah, yes not to forget to mention: credits for this idea go to Aaron La
Framboise.

Regards,

--
_________________________________________
  _  _  | Roland Schwarz
 |_)(_  | aka. speedsnail
 | \__) | mailto:[hidden email]
________| http://www.blackspace.at