Everything that concerns curl. Questions, bug reports, scripts and related tools and subjects welcome

Yura Kaidalov | 24 May 2012 15:21

cURL process hangs on ftp upload/download operations.

Hello

cURL process hangs on ftp upload/download operations on Windows. Such hangs 
happens several times per day. Here's details:
curl version is:
curl 7.23.1 (i386-pc-win32) libcurl/7.23.1 OpenSSL/1.0.0 libssh2/1.0
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp 
scp sftp smtp smtps telnet tftp
Features: AsynchDNS Largefile NTLM SSL

command line is: curl.exe -u uname:password --connect-timeout 180 --trace 
__trace.out -T source.dat ftp://hostname/target.dat

content of __trace.out:
== Info: About to connect() to hostname port 21 (#0)
== Info:   Trying XX.XX.XX.XX... == Info: connected
<= Recv header, 27 bytes (0x1b)
0000: 32 32 30 20 4d 69 63 72 6f 73 6f 66 74 20 46 54 220 Microsoft FT
0010: 50 20 53 65 72 76 69 63 65 0d 0a                P Service..
=> Send header, 12 bytes (0xc)
0000: 55 53 45 52 20 65 73 74 61 72 0d 0a             USER uname..
<= Recv header, 34 bytes (0x22)
0000: 33 33 31 20 50 61 73 73 77 6f 72 64 20 72 65 71 331 Password req
0010: 75 69 72 65 64 20 66 6f 72 20 65 73 74 61 72 2e uired for uname.
0020: 0d 0a                                           ..
=> Send header, 19 bytes (0x13)
0000: 50 41 53 53 20 53 74 61 72 45 61 67 6c 65 21 21 PASS XXXXXXXXXXX
0010: 21 0d 0a                                        X..
<= Recv header, 27 bytes (0x1b)
0000: 32 33 30 20 55 73 65 72 20 65 73 74 61 72 20 6c 230 User uname l

(Continue reading)

headers

Daniel Stenberg | 24 May 2012 22:33

Re: cURL process hangs on ftp upload/download operations.

On Thu, 24 May 2012, Yura Kaidalov wrote:

> cURL process hangs on ftp upload/download operations on Windows. Such hangs 
> happens several times per day.

Out of how may tries?

> command line is: curl.exe -u uname:password --connect-timeout 180 --trace 
> __trace.out -T source.dat ftp://hostname/target.dat

I don't remember off-hand how good windows is at keepalives, but 
--keepalive-time might be a good option to append to that command line

> 0000: 31 32 35 20 44 61 74 61 20 63 6f 6e 6e 65 63 74 125 Data connect
> 0010: 69 6f 6e 20 61 6c 72 65 61 64 79 20 6f 70 65 6e ion already open
> 0020: 3b 20 54 72 61 6e 73 66 65 72 20 73 74 61 72 74 ; Transfer start
> 0030: 69 6e 67 2e 0d 0a                               ing...
> => Send data, 4 bytes (0x4)
> 0000: 74 65 73 74                                     data
>
> My questions are: why does cURL hangs? and how can I prevent these hangs?

libcurl will sit waiting for the "connection" to accept more data to send and 
until that happens it won't do anything. The mystery to me is why your 
network/server somehow dies or otherwise makes the connection stall like this.

I don't think we had any such bug in that curl version, but I can of course be 
wrong.

Another option combo to use to work around the hang is

(Continue reading)

headers

Yura Kaidalov | 25 May 2012 12:41

Re: cURL process hangs on ftp upload/download operations.

> Out of how may tries?
I noticed 92 hangs per 3541 requests.

> I don't remember off-hand how good windows is at keepalives, 
> but --keepalive-time might be a good option to append to that command line
OK - thank you, I'm going to append this option to the command.

>The mystery to me is why your network/server somehow dies or otherwise 
>makes the connection stall like this.
Do you mean, that such hangs can happen, if a ftp server is going down while 
cURL request is in progress? 

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 25 May 2012 14:38

Re: cURL process hangs on ftp upload/download operations.

On Fri, 25 May 2012, Yura Kaidalov wrote:

>> The mystery to me is why your network/server somehow dies or otherwise 
>> makes the connection stall like this.

> Do you mean, that such hangs can happen, if a ftp server is going down while 
> cURL request is in progress?

I'm thinking this:

   http://curl.haxx.se/docs/faq.html#Why_doesn_t_cURL_return_an_error

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Gokhan Sengun | 25 May 2012 14:44

Re: cURL process hangs on ftp upload/download operations.

Hi Yura,

The mystery to me is why your network/server somehow dies or otherwise makes the connection stall like this.

Do you mean, that such hangs can happen, if a ftp server is going down while cURL request is in progress?

I'm thinking this:

http://curl.haxx.se/docs/faq.html#Why_doesn_t_cURL_return_an_error

I think

1- Trying with another FTP server
2- Capturing network traces with the original FTP server and observing what is going on during hang period

are two musts to diagnose this issue.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html