Joe Mason | 31 Jul 2012 21:10
Favicon
Gravatar

sws test server with --fork

I've found out how to run sws with the --fork option by calling "runtests.pl -f".  But when I do that, several
copies of sws are left running after runtests.pl finishes (so if I run runtests.pl again without killing
them, it complains the ports are already in use).

Is it a good idea to add code to runtests.pl to kill off all forked copies of sws that it started when it exists? 
(Or is there already a way to do this that I'm just missing?)

Is there a way I can have the test suite automatically add --fork for tests that require it?  (A keyword,
maybe.)  If not, is there any interest in my adding this?

The reason I want it right now is because I wrote a test of NTLM auth, and found that because NTLM is connection
oriented, curl will only reuse the same connection for requests with the exact same NTLM paramters. If you
try to make two NTLM requests to the same host but with different usernames or passwords, curl will use a
separate connection for each.  Which is fine, but without the --fork option to sws, the test suite only
listens on one connection at a time.  With --fork, my new test seems to work.

It seems to me that curl needs a lot more tests that exercise the mapping of requests to connections
(especially as we start improving pipelining) so it's important for the test server to support multiple
connections.  So --fork should be well supported by the test suite.  But the code is full of comments that say
things like, "This is NOT what you wanna do without knowing exactly why and for what," and that --fork is
for, "if you feel wild and crazy and want to setup some more exotic tests".  Honestly, it doesn't seem that
complicated to me, and a test involving several requests in a row (which may or may not use multiple
connections; impossible to tell without digging into the guts of curl, which shouldn't be necessary to
write a functionl test) don't seem that exotic... does fork mode h
 ave a lot of hidden gotchas?

Ideally the server would support requests from multiple connections by default, so that the test authors
wouldn't have to add "fork" keywords in a cargo-cult like way when writing tests that happen to create more
than one connection.  What would be the implications of turning --fork mode on by default for the test suite?

(Continue reading)

Daniel Stenberg | 1 Aug 2012 17:40
Picon
Favicon
Gravatar

Re: sws test server with --fork

On Tue, 31 Jul 2012, Joe Mason wrote:

> I've found out how to run sws with the --fork option by calling "runtests.pl 
> -f".  But when I do that, several copies of sws are left running after 
> runtests.pl finishes (so if I run runtests.pl again without killing them, it 
> complains the ports are already in use).

Yeah, I did the fork option once just to be able to do some testing with a 
large amount of connections (back when I created the multi_socket API). It was 
never used in the test suite "properly" so if we need to have something like 
that used, we need to add support for it.

> Is it a good idea to add code to runtests.pl to kill off all forked copies 
> of sws that it started when it exists?  (Or is there already a way to do 
> this that I'm just missing?)

If we need to have a server running that accepts multiple connections 
simultaneously, then using the --fork option might be a good idea. 
Alternatively, we make the server just handle more connections in the same 
process.

It probably make sense to make the sws server able to run with this option 
always instead for just a specific test case as otherwise we need to shut down 
and restart the sws server for the specific tests that need --fork.

> The reason I want it right now is because I wrote a test of NTLM auth, and 
> found that because NTLM is connection oriented, curl will only reuse the 
> same connection for requests with the exact same NTLM paramters. If you try 
> to make two NTLM requests to the same host but with different usernames or 
> passwords, curl will use a separate connection for each.  Which is fine, but 
(Continue reading)

Joe Mason | 1 Aug 2012 18:14
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Daniel 
> Stenberg [daniel <at> haxx.se]
> 
> > It seems to me that curl needs a lot more tests that exercise the mapping 
> > of
> > requests to connections (especially as we start improving pipelining) so
> > it's important for the test server to support multiple connections.  So
> > --fork should be well supported by the test suite.  But the code is full 
> > of
> > comments that say things like, "This is NOT what you wanna do without
> > knowing exactly why and for what," and that --fork is for, "if you feel 
> > wild
> > and crazy and want to setup some more exotic tests".
> 
> Right. I think I added those comments mostly because of what you already found
> out: that the test suite isn't yet modified to actually work with it. It makes
> the --fork option mostly usable only for custom special tests. I don't think a
> forking http server in particular needs a lof of magic to get working.

My main concern is if the server saves state between successive requests that it makes (including when a
connection is closed and it starts listening on the next, in serial). If it forks for each connection then
that state wouldn't be shared between the connection handlers.  Do you know if that happens?

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
(Continue reading)

Joe Mason | 1 Aug 2012 18:48
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Daniel 
> Stenberg [daniel <at> haxx.se]
> Sent: Wednesday, August 01, 2012 11:40 AM
> To: libcurl development
> Subject: Re: sws test server with --fork
> 
> > The reason I want it right now is because I wrote a test of NTLM auth, and
> > found that because NTLM is connection oriented, curl will only reuse the
> > same connection for requests with the exact same NTLM paramters. If you 
> > try
> > to make two NTLM requests to the same host but with different usernames or
> > passwords, curl will use a separate connection for each.  Which is fine, 
> > but
> > without the --fork option to sws, the test suite only listens on one
> > connection at a time.  With --fork, my new test seems to work.
> 
> Ah yes, makes sense. If we really need to have more than one conection to the
> server up, then we need to make the server handle it I'd say!

My new test just stopped working even with --fork.  Don't know why... I'm leaning towards extending the sws
server to use poll() and multiple sockets in the same process.  That will take care of concerns about
serialization of the log files, etc, and make sure we don't lose any state.

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
(Continue reading)

Daniel Stenberg | 1 Aug 2012 19:01
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Wed, 1 Aug 2012, Joe Mason wrote:

> My new test just stopped working even with --fork.  Don't know why... I'm 
> leaning towards extending the sws server to use poll() and multiple sockets 
> in the same process.  That will take care of concerns about serialization of 
> the log files, etc, and make sure we don't lose any state.

Yes I think it is a good idea. It will also be easier to kill the server etc.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Joe Mason | 3 Aug 2012 01:03
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Daniel 
> Stenberg [daniel <at> haxx.se]
> 
> On Wed, 1 Aug 2012, Joe Mason wrote:
> 
> > My new test just stopped working even with --fork.  Don't know why... I'm
> > leaning towards extending the sws server to use poll() and multiple 
> > sockets
> > in the same process.  That will take care of concerns about serialization 
> > of
> > the log files, etc, and make sure we don't lose any state.
> 
> Yes I think it is a good idea. It will also be easier to kill the server etc.

Done: see https://github.com/JoeNotCharles/curl/commits/sws_poll

Well, almost done.  I didn't change send_doc, so writing to a socket will still blast data at it as fast as
possible without polling for write. And I didn't touch http_connect, so tunnelled connections are still
limited to one socket.  Neither of these seems to cause a problem with existing tests.

There's still an intermittent bug: sometimes, if curl sends requests on socket A, then B, then A again, by
the time sws reads the second request from socket A, the socket is closed (apparently due to a timeout).  I
added SO_KEEPALIVE to the sockets and this got much rarer, but it still happens occasionally.

Mind taking a look at this to see if you can spot anything I'm missing? I broke the work up into a bunch of
commits with smaller changes that should be easier to follow (moving code into its own function in one
commit, changing function return values in another, etc).

Joe

(Continue reading)

Daniel Stenberg | 3 Aug 2012 01:25
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Thu, 2 Aug 2012, Joe Mason wrote:

> Mind taking a look at this to see if you can spot anything I'm missing? I 
> broke the work up into a bunch of commits with smaller changes that should 
> be easier to follow (moving code into its own function in one commit, 
> changing function return values in another, etc).

Sure! Any particular test case that trigger the problem or how would I go 
ahead and experience it the best way?

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Joe Mason | 3 Aug 2012 06:42
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Daniel 
> Stenberg [daniel <at> haxx.se]
> Sent: Thursday, August 02, 2012 7:25 PM
> To: libcurl development
> Subject: RE: sws test server with --fork
> 
> On Thu, 2 Aug 2012, Joe Mason wrote:
> 
> > Mind taking a look at this to see if you can spot anything I'm missing? I
> > broke the work up into a bunch of commits with smaller changes that should
> > be easier to follow (moving code into its own function in one commit,
> > changing function return values in another, etc).
> 
> Sure! Any particular test case that trigger the problem or how would I go
> ahead and experience it the best way?

Test 2025, 2028, 2029, 2030, and 2031 from
https://github.com/JoeNotCharles/curl/commits/reset_test (all added in the top commit) should do
it.  (The tests all have some mistakes in them, which I have fixed locally but not uploaded to github yet, so
they'll still fail with the new sws - but with the old sws, they'll loop infinitely, and with the new one
they'll fail cleanly.)

I'm 90% sure the versions in that branch show the infinite loop. If not, they're more out of date than I
realized.  Tomorrow I'm going to clean up my local modifications and upload new copies.

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
(Continue reading)

Daniel Stenberg | 3 Aug 2012 23:35
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Fri, 3 Aug 2012, Joe Mason wrote:

> Test 2025
>
> I'm 90% sure the versions in that branch show the infinite loop. If not, 
> they're more out of date than I realized.  Tomorrow I'm going to clean up my 
> local modifications and upload new copies.

I run 2025 on your sws_poll branch and yes I get the inifinite hang all the 
time. Now I'll just try to figure out what the test does and what the server 
is supposed to do... =)

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Joe Mason | 3 Aug 2012 23:46
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Joe Mason 
> [jmason <at> rim.com]
> 
> Test 2025, 2028, 2029, 2030, and 2031 from 
> https://github.com/JoeNotCharles/curl/commits/reset_test (all added in the top
>  commit) should do it.  (The tests all have some mistakes in them, which I 
> have fixed locally but not uploaded to github yet, so they'll still fail with 
> the new sws - but with the old sws, they'll loop infinitely, and with the new 
> one they'll fail cleanly.)
> 
> I'm 90% sure the versions in that branch show the infinite loop. If not, 
> they're more out of date than I realized.  Tomorrow I'm going to clean up my 
> local modifications and upload new copies.

I've updated the branch with another patch fixing all the errors in the tests.

Also, good news!  I have a fix for bug 3545398 - at least, covering basic, digest, and ntlm auth.  (I don't have
curl built with NTLM_WB so I couldn't test if it has any specific bugs other than the one I fixed for regular
NTLM. And I don't know where to begin setting up the test suite to work with GSSNEGOTIATE - I understand it's
a real pain to set up.  I notice there are no existing tests covering it.)  See the sourceforge page for details.

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

(Continue reading)

Daniel Stenberg | 5 Aug 2012 23:25
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Fri, 3 Aug 2012, Joe Mason wrote:

> Also, good news!  I have a fix for bug 3545398 - at least, covering basic, 
> digest, and ntlm auth.

Lovely work, thanks a lot!

I'm about to merge and push those commits to the master branch.

I do however still get a hang with the new NTLM tests (2025, 2028 and 2031) so 
unless I manage to fix those problems first, I might push them as DISABLED so 
that we can work out the problems first before the autobuilders get to run 
them.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Daniel Stenberg | 6 Aug 2012 00:37
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Sun, 5 Aug 2012, Daniel Stenberg wrote:

> I do however still get a hang with the new NTLM tests (2025, 2028 and 2031) 
> so unless I manage to fix those problems first, I might push them as 
> DISABLED so that we can work out the problems first before the autobuilders 
> get to run them.

Okey, I ended up doing it like that. The changes and test cases are now pushed 
and I'll continue to work on the remaining error that seems to be a sws server 
bug from what I've seen so far.

Joe, if you get a chance to try this out, do these tests work for you?

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Daniel Stenberg | 6 Aug 2012 00:40
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Mon, 6 Aug 2012, Daniel Stenberg wrote:

> Joe, if you get a chance to try this out, do these tests work for you?

Oh, and here's a little nit/request for me for future commits: please wrap the 
commit messages at column 72 so that they appear nice with 'git log' and web 
UIs etc.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Guenter | 6 Aug 2012 14:29

Re: sws test server with --fork

Hi,
Am 01.08.2012 19:01, schrieb Daniel Stenberg:
> On Wed, 1 Aug 2012, Joe Mason wrote:
>
>> My new test just stopped working even with --fork. Don't know why...
>> I'm leaning towards extending the sws server to use poll() and
>> multiple sockets in the same process. That will take care of concerns
>> about serialization of the log files, etc, and make sure we don't lose
>> any state.
>
> Yes I think it is a good idea. It will also be easier to kill the server
> etc.
unless we also provide an alternate code path (with select?) for 
platforms that lack poll() the recent changes break the testsuite on 
Windows because:
1) WSAPoll() as equivalent to poll() is only available on Vista and 
later [1]
2) WSAPoll() seems to have bugs [2]
3) MSYS has serious problems running on anything later than WinXP [3]
4) WSAPoll() does currently not yet appear in any MinGW headers [4]

therefore I have stopped all my recently added MSYS/MinGW autobuids 
because they are now useless since the testsuite cant run anymore, and 
for verifying compilation the Linux-based cross-builds are good enough.

Gün.

[1] 
http://msdn.microsoft.com/en-us/library/windows/desktop/ms741669%28v=vs.85%29.aspx
[2] http://curl.haxx.se/mail/lib-2012-07/0310.html
(Continue reading)

Daniel Stenberg | 6 Aug 2012 15:15
Picon
Favicon
Gravatar

Re: sws test server with --fork

On Mon, 6 Aug 2012, Guenter wrote:

> unless we also provide an alternate code path (with select?) for platforms 
> that lack poll() the recent changes break the testsuite on Windows because:

I can't think of any reason we can't use select() instead.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Joe Mason | 6 Aug 2012 16:34
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Daniel 
> Stenberg [daniel <at> haxx.se]
> 
> On Mon, 6 Aug 2012, Guenter wrote:
> 
> > unless we also provide an alternate code path (with select?) for platforms
> > that lack poll() the recent changes break the testsuite on Windows 
> > because:
> 
> I can't think of any reason we can't use select() instead.

Yes, I just used poll because I prefer the API.  I'm not familiar with Windows development so I didn't realize
select was better supported there.

I noticed that all the existing includes in sws.c used autoconf "HAVE_FOO" checks, which I didn't bother
including while I was developing this - are there any features that need to be wrapped for other platforms?

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

(Continue reading)

Joe Mason | 6 Aug 2012 19:50
Favicon
Gravatar

RE: sws test server with --fork

> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Joe Mason 
> [jmason <at> rim.com]
> 
> I noticed that all the existing includes in sws.c used autoconf "HAVE_FOO" 
> checks, which I didn't bother including while I was developing this - are 
> there any features that need to be wrapped for other platforms?

In fact, there was one - fcntl(O_NONBLOCK - and there's an existing curlx_nonblock wrapper for it.

I've updated to use curlx_nonblock and select - sws should be cross-platform again now.

I also fixed the last 3 tests that were locking up, and reenabled them!  The select version segfaulted
instead of locking up, which made it easy to find.

Patches are in https://github.com/JoeNotCharles/curl/commits/reset_fix (top 4 commits)

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

(Continue reading)

Guenter | 7 Aug 2012 00:03

Re: sws test server with --fork

Hi Joe,
Am 06.08.2012 19:50, schrieb Joe Mason:
>> From: curl-library [curl-library-bounces <at> cool.haxx.se] on behalf of Joe Mason
>> [jmason <at> rim.com]
>>
>> I noticed that all the existing includes in sws.c used autoconf "HAVE_FOO"
>> checks, which I didn't bother including while I was developing this - are
>> there any features that need to be wrapped for other platforms?
>
> In fact, there was one - fcntl(O_NONBLOCK - and there's an existing curlx_nonblock wrapper for it.
>
> I've updated to use curlx_nonblock and select - sws should be cross-platform again now.
>
> I also fixed the last 3 tests that were locking up, and reenabled them!  The select version segfaulted
instead of locking up, which made it easy to find.
>
> Patches are in https://github.com/JoeNotCharles/curl/commits/reset_fix (top 4 commits)
thanks very much for your hard work on this!

Gün.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Daniel Stenberg | 7 Aug 2012 00:30
Picon
Favicon
Gravatar

RE: sws test server with --fork

On Mon, 6 Aug 2012, Joe Mason wrote:

> Patches are in https://github.com/JoeNotCharles/curl/commits/reset_fix (top 
> 4 commits)

Merged and pushed. Again, nice work Joe!

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html


Gmane