Andrei Cipu | 30 Mar 13:10 2012

POST after the first POST returned 303

Hi,

Currently libCURL assumes that if a POST request returns HTTP code 303, the follow-up must be a GET request.
It even has a funny comment in the code:  " doing a second POST when following isn't what anyone would want!".
However, RFC 2616 states [1] that " a different URI [...]  SHOULD be retrieved using a GET method on that
resource" (only SHOULD, not MUST). As it turns out, there are servers out there that want a POST after they
returned 303 to a previous POST request. 

The patch in the following email adds another value (CURL_REDIR_POST_303) for the CURLOPT_POSTREDIR
option, allowing the user to send 2 consecutive POSTs.

Regards,
  Andrei Cipu

[1] http://tools.ietf.org/html/rfc2616#section-10.3.4

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Andrei Cipu | 30 Mar 13:19 2012

RE: POST after the first POST returned 303


> Hi,

> Currently libCURL assumes that if a POST request returns HTTP code 303, the follow-up must be a GET
request. It even has a funny comment in the code:  " doing a second POST when following isn't 
> what anyone would want!". However, RFC 2616 states [1] that " a different URI [...]  SHOULD be retrieved
using a GET method on that resource" (only SHOULD, not MUST). As it turns out, there are 
> servers out there that want a POST after they returned 303 to a previous POST request. 

> The patch in the following email adds another value (CURL_REDIR_POST_303) for the CURLOPT_POSTREDIR
option, allowing the user to send 2 consecutive POSTs.

Git send-email fail :) Attaching by hand...

Andrei
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Daniel Stenberg | 1 Apr 15:16 2012
Picon

Re: POST after the first POST returned 303

On Fri, 30 Mar 2012, Andrei Cipu wrote:

> Currently libCURL assumes that if a POST request returns HTTP code 303, the 
> follow-up must be a GET request. It even has a funny comment in the code: 
> " doing a second POST when following isn't what anyone would want!".

Yes, the comment is indeed funny and should be removed.

> However, RFC 2616 states [1] that " a different URI [...] SHOULD be 
> retrieved using a GET method on that resource" (only SHOULD, not MUST). As 
> it turns out, there are servers out there that want a POST after they 
> returned 303 to a previous POST request.

And there are browsers doing that as well? The "bad" 301 and 302 behaviors 
were due to how things were in the early HTTP 1.0 days. 303 wasn't introduced 
until HTTP 1.1 and the phrasing was quite clear IMHO. Why would anyone to 
blatantly and clearly violate the spec and not obey the SHOULD? It doesn't 
really make sense to me. Do you know?

A server that *REALLY* wants the client to use the same request again after a 
HTTP redirect should use 307. And if they're somewhat modern and what the 
"permanent" state for the redirect then 308 is your friend:

   http://tools.ietf.org/html/draft-reschke-http-status-308-07

Your patch is small and unintroducive, but I would rather first try to 
understand why 303 is used here and what browsers facing this response code 
do.

--

-- 
(Continue reading)

Andrei Cipu | 2 Apr 15:24 2012

RE: POST after the first POST returned 303

> -----Original Message-----
> From: curl-library-bounces <at> cool.haxx.se [mailto:curl-library-
> bounces <at> cool.haxx.se] On Behalf Of Daniel Stenberg
> Sent: Sunday, April 01, 2012 4:17 PM
> To: libcurl development
> Subject: Re: POST after the first POST returned 303
>
> > However, RFC 2616 states [1] that " a different URI [...] SHOULD be
> > retrieved using a GET method on that resource" (only SHOULD, not
> > MUST). As it turns out, there are servers out there that want a POST
> > after they returned 303 to a previous POST request.
> 
> And there are browsers doing that as well? The "bad" 301 and 302 behaviors
> were due to how things were in the early HTTP 1.0 days. 303 wasn't
> introduced until HTTP 1.1 and the phrasing was quite clear IMHO. Why would
> anyone to blatantly and clearly violate the spec and not obey the SHOULD? It
> doesn't really make sense to me. Do you know?

First of all, I don't think of this as a blatant violation, but rather a bending of the spec. That's why I
believe the server I'm working with might not be the only one implementing it, thus making the patch
potentially useful to other libcurl users (but I have no proof for this).

To answer your question, the behavior appears when the server is talking (HTTP) to a dedicated client, so I
believe the answer is "because they can". I cannot test that particular server using a full browser or give
you any other data points, but I suspect the browser will follow the rfc. Very likely, the message exchange
with the browser will be slightly different.

Thanks,
   Andrei

(Continue reading)

Daniel Stenberg | 2 Apr 15:48 2012
Picon

RE: POST after the first POST returned 303

On Mon, 2 Apr 2012, Andrei Cipu wrote:

> First of all, I don't think of this as a blatant violation, but rather a 
> bending of the spec. That's why I believe the server I'm working with might 
> not be the only one implementing it, thus making the patch potentially 
> useful to other libcurl users (but I have no proof for this).

In my view, a SHOULD is short for "this is the way to do it unless you have a 
really really good reason not to", and I can't think of such a reason here. 
But OK, I'll give in since I understand this is a server and another client 
that made it like this.

> To answer your question, the behavior appears when the server is talking 
> (HTTP) to a dedicated client, so I believe the answer is "because they can". 
> I cannot test that particular server using a full browser or give you any 
> other data points, but I suspect the browser will follow the rfc. Very 
> likely, the message exchange with the browser will be slightly different.

So okay, let's add the option for 303 as well - after all it is very easy and 
comes with hardly any cost. But then I have some minor additional requests for 
the patch:

1 - please also update the documentation (see docs/libcurl/symbols-in-versions
     too)

2 - I think we can switch to using a bitmask internally now since having three
     different booleans for this turns inconvenient I think.

3 - Not strictly required, but please consider adding a test case that
     verifies this new bit. There should be existing 301/302 test cases to
(Continue reading)

Andrei Cipu | 2 Apr 17:25 2012

RE: POST after the first POST returned 303

> 
> 1 - please also update the documentation (see docs/libcurl/symbols-in-
> versions
>      too)
> 
> 2 - I think we can switch to using a bitmask internally now since having three
>      different booleans for this turns inconvenient I think.

I attached v2.

> 
> 3 - Not strictly required, but please consider adding a test case that
>      verifies this new bit. There should be existing 301/302 test cases to
>      get inspiration from.

I'm still digging through libcurl's testing framework, I find it very convoluted compared with the rest of
the code. Could you please give some more precise indications on where to look? A search for "302" and
"post302" hasn't found anything useful.

If I finally make it, I will provide a separate patch for that.

Andrei.

Attachment (redir_post_303.patch): application/octet-stream, 8 KiB
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Daniel Stenberg | 2 Apr 20:08 2012
Picon

RE: POST after the first POST returned 303

On Mon, 2 Apr 2012, Andrei Cipu wrote:

> I attached v2.

Thanks! I'll check it out soonish.

> I'm still digging through libcurl's testing framework, I find it very 
> convoluted compared with the rest of the code.

tests/README describes the general test setup and concepts.

> Could you please give some more precise indications on where to look? A 
> search for "302" and "post302" hasn't found anything useful.

$ grep post302 tests/data/test*
tests/data/test1076:HTTP POST with 302 redirect and --post302
tests/data/test1076:http://%HOSTIP:%HTTPPORT/blah/1076 -L -d "moo" --post302

... but as you will see, the only tests we have for these features are using 
the command line tool curl so for you to easily duplicate and slightly modify 
for example test1076 you need to make this a command line option first! =)

Adding a libcurl-using test wouldn't be a lot harder, but you'd have to write 
a tiny test code like lib599.c and create a new test, perhaps named 1500 
(since the 5xx range is full).

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
(Continue reading)

Daniel Stenberg | 5 Apr 23:40 2012
Picon

RE: POST after the first POST returned 303

On Mon, 2 Apr 2012, Daniel Stenberg wrote:

> ... but as you will see, the only tests we have for these features are using 
> the command line tool curl so for you to easily duplicate and slightly 
> modify for example test1076 you need to make this a command line option 
> first! =)

Thanks Andrei, I've merged and pushed your patch.

As a bonus, I first added the --post303 command line option:

   https://github.com/bagder/curl/commit/29e68b200c

and then I showed how easy it is to add a new test for something like this 
with the new test case 1332:

   https://github.com/bagder/curl/commit/1dd69921d

Enjoy!

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html


Gmane