headers
Felix E. Klee | 20 Jun 2012 18:56
Picon
Gravatar

curl_multi_socket_action: second argument?

What is the difference between calling `curl_multi_socket_action` with a
socket as second argument and with `CURL_SOCKET_TIMEOUT`?

My assumption:

* With socket as second argument:

  a) Calls `CURLOPT_WRITEFUNCTION` with data retrieved via that socket
    since the last call.

  b) Checks status of the socket. If status has changed, then calls:

      CURLMOPT_SOCKETFUNCTION

    I assume that when/after retrieving data, then the only status
    change that can happen here can be:

      CURL_POLL_REMOVE

    After all, the socket does already exist.

* With `CURL_SOCKET_TIMEOUT`:

  Checks statuses of *all sockets*, and for each socket whose status has
  changed, calls:

    CURLMOPT_SOCKETFUNCTION

Is that correct?
-------------------------------------------------------------------
(Continue reading)

Permalink | Reply |
headers
Daniel Stenberg | 21 Jun 2012 15:12
Picon
Favicon
Gravatar

Re: curl_multi_socket_action: second argument?

On Wed, 20 Jun 2012, Felix E. Klee wrote:

> What is the difference between calling `curl_multi_socket_action` with a 
> socket as second argument and with `CURL_SOCKET_TIMEOUT`?

In the first case you're telling libcurl some activity is happening on the 
given socket, and in the other case you're saying that the timeout has 
triggered.

> My assumption:
>
> * With socket as second argument:
>
>  a) Calls `CURLOPT_WRITEFUNCTION` with data retrieved via that socket
>    since the last call.

If data gets received, yes. Socket activity can mean several other things as 
well.

>  b) Checks status of the socket. If status has changed, then calls:
>
>      CURLMOPT_SOCKETFUNCTION

If something internally has changed so that your program should wait for 
another activity on the socket than what it already has been told to, then yes 
it will happen.

> * With `CURL_SOCKET_TIMEOUT`:
>
>  Checks statuses of *all sockets*, and for each socket whose status has
(Continue reading)

Permalink | Reply |
headers
Felix E. Klee | 21 Jun 2012 16:46
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

Thanks for the explanation!

That makes me understand a little more of the details. But, to be
honest, I don't get the big picture. I know that sockets are being used
for inter process communication, but I don't see that here. There is
only one process after all: the application linked with cURL

Is there some general overview concerning how cURL multi socket works?

Helpful would be an example list of steps for a typical retrieval,
something like:

1. Function X is called.

2. Callback Y gets triggered.

3. Data is retrieved.

etc. Or just some very coarse pseudo code.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |
headers
Felix E. Klee | 23 Jun 2012 18:53
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

On Thu, Jun 21, 2012 at 4:46 PM, Felix E. Klee <felix.klee <at> inka.de>
wrote:
> Helpful would be an example list of steps for a typical retrieval
> [...]

What follows is the coarse flow of a program that watches a database and
downloads data from requested URLs.

Key events for exactly one request (query), which is successful:

1. Program start

  The event loop is started.

2. Timer event (database-watcher)
  Timer event (database-watcher)
  ...

  The database is checked every half a second until finally a request
  (URL) is available.

  Once a request is available, the event handler constructs a query
  based on the URL. The associated cURL easy handle is passed to
  curl_multi_add_handle().

  curl_multi_add_handle() calls CURLMOPT_TIMERFUNCTION with a timeout of
  just 1 ms. CURLMOPT_TIMERFUNCTION adjust the timeout for the
  check-cURL-handles-watcher.

3. Timer event (check-cURL-handles-watcher)
(Continue reading)

Permalink | Reply |
headers
Daniel Stenberg | 23 Jun 2012 22:55
Picon
Favicon
Gravatar

Re: curl_multi_socket_action: second argument?

On Thu, 21 Jun 2012, Felix E. Klee wrote:

> That makes me understand a little more of the details. But, to be honest, I 
> don't get the big picture. I know that sockets are being used for inter 
> process communication, but I don't see that here. There is only one process 
> after all: the application linked with cURL

sockets are used for the TCP connection(s) and sometimes for name resolving - 
depending on your particular libcurl build.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |
headers
Felix E. Klee | 24 Jun 2012 14:52
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

On Sat, Jun 23, 2012 at 10:55 PM, Daniel Stenberg <daniel <at> haxx.se>
wrote:
> sockets are used for the TCP connection(s) and sometimes for name
> resolving - depending on your particular libcurl build.

Thanks! As far as I understand it, the OS's TCP/IP stack is writing
directly into the sockets.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |
headers
Felix E. Klee | 21 Jun 2012 17:02
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

On Thu, Jun 21, 2012 at 3:12 PM, Daniel Stenberg <daniel <at> haxx.se> wrote:
>> [curl_multi_socket_action] With `CURL_SOCKET_TIMEOUT`:
>
> It goes through all easy-handles that have reached the timeout time
> and checks if there's anything to do for them due to the timeout. It
> can lead to calls to the CURLMOPT_SOCKETFUNCTION sure, but also to
> CURLMOPT_TIMERFUNCTION or none at all, depending on the exact
> circumstances.

Concerning the timer / timeouts, some more questions:

* Why is it necessary to repeatedly call `curl_multi_socket_action` with
  `CURL_SOCKET_TIMEOUT`? After a socket has been created, isn't it
  sufficient to only watch actions on the socket?

  After all, each socket is associated with an easy handle.

* How is the timeout value determined internally?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |
headers
Felix E. Klee | 23 Jun 2012 19:59
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

On Thu, Jun 21, 2012 at 5:02 PM, Felix E. Klee <felix.klee <at> inka.de>
wrote:
> * How is the timeout value determined internally?

The 40 seconds value that I've seen (see my other message), is that
configurable?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |
headers
Daniel Stenberg | 23 Jun 2012 22:54
Picon
Favicon
Gravatar

Re: curl_multi_socket_action: second argument?

On Thu, 21 Jun 2012, Felix E. Klee wrote:

> Concerning the timer / timeouts, some more questions:
>
> * Why is it necessary to repeatedly call `curl_multi_socket_action` with
>  `CURL_SOCKET_TIMEOUT`? After a socket has been created, isn't it
>  sufficient to only watch actions on the socket?
>
>  After all, each socket is associated with an easy handle.

Because things in libcurl can trigger on time and not just on socket activity! 
Things like retries or transfer-speed checks and whatever. Lots of operations 
also have a maximum time so that a handle won't stick in a state forever. This 
is especially true if you set a timeout with the API.

> * How is the timeout value determined internally?

The code sets a timeout value where applicable. See 
lib/multi.c:multi_timeout() for the specific function that handles it 
internally. libcurl can hold multiple timeouts for each handle, but only the 
nearest in time value is what is told to the timer callback.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
(Continue reading)

Permalink | Reply |
headers
Felix E. Klee | 24 Jun 2012 14:53
Picon
Gravatar

Re: curl_multi_socket_action: second argument?

Thanks for the explanation - very helpful!
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Permalink | Reply |

Gmane