strange pthread_create cap

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

strange pthread_create cap

Laura Arhire-2
Hello

I'm having some trouble with pthreads-win and sockets, wondering if
anyone can help. I have a test setup with a loop which iterates a number
of times. Inside the loop, I create a thread on which I run an SSL
server socket. After the thread is created, I connect an SSL Client
socket to the server socket, then disconnect it, disconnect the server
socket, and issue a pthread_join.

Using this setup (one thread created with pthread_create at any time), I
cannot create anymore threads after 10-15 such iterations (in rare cases
- say if I run a 200-iteration loop, I might be able to create threads
again once I reach iteration 50 or so). I believe this to be an issue
with my SSL client socket implementation, because everything works well
if I don't connect a client socket during the iteration. It also works
well if I use a non-SSL server/client socket.

However, I thought I'd ask here: is there any reason why pthread_create
would return EAGAIN in such a setup? The handle count varies very
slightly during the iterations but does not increase over the run time.
The memory does increase due to my test setup, but at the maximum peak
it is no where near close to using up all system resources.

Can it be some sort of problem with the release of the socket (win
sockets are not guaranteed to be released as soon as closesocket returns) ?

Thank you in advance,
Laura
Reply | Threaded
Open this post in threaded view
|

Re: strange pthread_create cap

Ross Johnson-2
Hi Laura,

Pthread_create can return EAGAIN for several reasons, all to do with
lack of resources, such as memory allocation (unlikely), or failing to
start the underlying Windows native thread, e.g. due to a lack of
resources there as well.

I don't know Windows well enough but I'm wondering if your listening
server sockets are really closing down, or perhaps just not quickly
enough. What happens if you add a short delay between creating threads,
i.e. opening new sockets?

Also, are you testing on a Windows Server or Windows Workstation? I ask
because, IIRC and if my information is not out-of-date, workstations
have a limit (10) on the number of sockets that can be listening at the
same time.

Ross

Laura Arhire wrote:

> Hello
>
> I'm having some trouble with pthreads-win and sockets, wondering if
> anyone can help. I have a test setup with a loop which iterates a
> number of times. Inside the loop, I create a thread on which I run an
> SSL server socket. After the thread is created, I connect an SSL
> Client socket to the server socket, then disconnect it, disconnect the
> server socket, and issue a pthread_join.
>
> Using this setup (one thread created with pthread_create at any time),
> I cannot create anymore threads after 10-15 such iterations (in rare
> cases - say if I run a 200-iteration loop, I might be able to create
> threads again once I reach iteration 50 or so). I believe this to be
> an issue with my SSL client socket implementation, because everything
> works well if I don't connect a client socket during the iteration. It
> also works well if I use a non-SSL server/client socket.
>
> However, I thought I'd ask here: is there any reason why
> pthread_create would return EAGAIN in such a setup? The handle count
> varies very slightly during the iterations but does not increase over
> the run time. The memory does increase due to my test setup, but at
> the maximum peak it is no where near close to using up all system
> resources.
>
> Can it be some sort of problem with the release of the socket (win
> sockets are not guaranteed to be released as soon as closesocket
> returns) ?
>
> Thank you in advance,
> Laura

Reply | Threaded
Open this post in threaded view
|

Re: strange pthread_create cap

Ross Johnson-2
...but, in any case, failing to open sockets should not cause
pthread_create itself to return EAGAIN.

Ross Johnson wrote:

> Hi Laura,
>
> Pthread_create can return EAGAIN for several reasons, all to do with
> lack of resources, such as memory allocation (unlikely), or failing to
> start the underlying Windows native thread, e.g. due to a lack of
> resources there as well.
>
> I don't know Windows well enough but I'm wondering if your listening
> server sockets are really closing down, or perhaps just not quickly
> enough. What happens if you add a short delay between creating
> threads, i.e. opening new sockets?
>
> Also, are you testing on a Windows Server or Windows Workstation? I
> ask because, IIRC and if my information is not out-of-date,
> workstations have a limit (10) on the number of sockets that can be
> listening at the same time.
>
> Ross
>
> Laura Arhire wrote:
>> Hello
>>
>> I'm having some trouble with pthreads-win and sockets, wondering if
>> anyone can help. I have a test setup with a loop which iterates a
>> number of times. Inside the loop, I create a thread on which I run an
>> SSL server socket. After the thread is created, I connect an SSL
>> Client socket to the server socket, then disconnect it, disconnect
>> the server socket, and issue a pthread_join.
>>
>> Using this setup (one thread created with pthread_create at any
>> time), I cannot create anymore threads after 10-15 such iterations
>> (in rare cases - say if I run a 200-iteration loop, I might be able
>> to create threads again once I reach iteration 50 or so). I believe
>> this to be an issue with my SSL client socket implementation, because
>> everything works well if I don't connect a client socket during the
>> iteration. It also works well if I use a non-SSL server/client socket.
>>
>> However, I thought I'd ask here: is there any reason why
>> pthread_create would return EAGAIN in such a setup? The handle count
>> varies very slightly during the iterations but does not increase over
>> the run time. The memory does increase due to my test setup, but at
>> the maximum peak it is no where near close to using up all system
>> resources.
>>
>> Can it be some sort of problem with the release of the socket (win
>> sockets are not guaranteed to be released as soon as closesocket
>> returns) ?
>>
>> Thank you in advance,
>> Laura
>

Reply | Threaded
Open this post in threaded view
|

RE: strange pthread_create cap

robert kindred-2
In reply to this post by Ross Johnson-2
Hello, Ross:

[snip]
>
> Hi Laura,
>
[snip]
>
> Also, are you testing on a Windows Server or Windows Workstation? I ask
> because, IIRC and if my information is not out-of-date, workstations
> have a limit (10) on the number of sockets that can be listening at the
> same time.

I thought this was true also, but I have had more than 20 connections to my
Windows service via its listening sockets on my desktop.  This limit is
apparently only applied to Windows services (file shares, etc.).  My .02,
though I don't think it solves Laura's problem.

>
> Ross
>
> Laura Arhire wrote:
> > Hello
> >
> > I'm having some trouble with pthreads-win and sockets, wondering if
> > anyone can help. I have a test setup with a loop which iterates a
> > number of times. Inside the loop, I create a thread on which I run an
> > SSL server socket. After the thread is created, I connect an SSL
> > Client socket to the server socket, then disconnect it, disconnect
> the
> > server socket, and issue a pthread_join.

[snip]

> >
> > Thank you in advance,
> > Laura

Robert Kindred

Reply | Threaded
Open this post in threaded view
|

RE: strange pthread_create cap

John Bossom

If Laura is running on Windows XP... there is a built in limit on the
number of simultaneously openned sockets... This is on purpose...
to prevent you from turning Windows XP into a Server...

Cheers,

John E. Bossom

Quoting robert kindred <[hidden email]>:

> Hello, Ross:
>
> [snip]
>>
>> Hi Laura,
>>
> [snip]
>>
>> Also, are you testing on a Windows Server or Windows Workstation? I ask
>> because, IIRC and if my information is not out-of-date, workstations
>> have a limit (10) on the number of sockets that can be listening at the
>> same time.
>
> I thought this was true also, but I have had more than 20 connections to my
> Windows service via its listening sockets on my desktop.  This limit is
> apparently only applied to Windows services (file shares, etc.).  My .02,
> though I don't think it solves Laura's problem.
>
>>
>> Ross
>>
>> Laura Arhire wrote:
>> > Hello
>> >
>> > I'm having some trouble with pthreads-win and sockets, wondering if
>> > anyone can help. I have a test setup with a loop which iterates a
>> > number of times. Inside the loop, I create a thread on which I run an
>> > SSL server socket. After the thread is created, I connect an SSL
>> > Client socket to the server socket, then disconnect it, disconnect
>> the
>> > server socket, and issue a pthread_join.
>
> [snip]
>
>> >
>> > Thank you in advance,
>> > Laura
>
> Robert Kindred
>
>


Reply | Threaded
Open this post in threaded view
|

Re: strange pthread_create cap

Laura Arhire-2
In reply to this post by robert kindred-2
Hi

Thanks for the replies. I managed to figure out what was wrong, and will
post this here in case some else manages to do the same rookie mistake
in the future.

The root of the problem was that I was compiling with a memory
management library. This, in turn, caused the whole non-paged pool
available for my application to be allocated within the first few calls
(and I suspect didn't fully release the sockets either) - actually
starting threads made this obvious (simply opening/closing sockets
doesn't).

I had initially dismissed the memory management library as a cause
because when compiling without it I still saw random test failures - so
I assumed (wrongly) that there must be something I was missing in the
sockets. However, while trying to figure out what resources I did not
release, I fixed the bug which caused the tests to fail - so now when I
do a simple compile, everything works as it should (and all the tests
pass, which is a bonus!).

Laura