headers
martin f krafft | 13 Jun 2011 10:02

Timeout during APPEND

Dear list,

I am running dovecot 1.2.15 on a Debian server.

One user reports continuous problems synchronising her mailbox via
IMAP (offlineimap, via SSH tunnel or SSL socket). It seems that she
has a large, locally-created message, but the uplink bandwidth seems
to be not enough to push it before dovecot times out the APPEND
command.

The error/exception happens inside offlineimap's Python imaplib2.py
file:

  APPEND => no response after 30.0 secs

I do not know the IMAP protocol all that well, but it seems to me
like this is broken somewhere.

Could you please help me figure out the problem?

--

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"man sagt nicht 'nichts!', man sagt dafür 'jenseits' oder 'gott'."
                                                 - friedrich nietzsche

spamtraps: madduck.bogus <at> madduck.net
Permalink | Reply |
headers
martin f krafft | 13 Jun 2011 10:11

Re: Timeout during APPEND

also sprach martin f krafft <madduck <at> madduck.net> [2011.06.13.1002 +0200]:
> One user reports continuous problems synchronising her mailbox via
> IMAP (offlineimap, via SSH tunnel or SSL socket). It seems that she
> has a large, locally-created message, but the uplink bandwidth seems
> to be not enough to push it before dovecot times out the APPEND
> command.

Upon further inspection, we found that the message *does* get saved
remotely. Hence, this seems like an offlineimap problem, timing out
because it receives no responses to APPEND (because the transfer
takes so long). The transfer actually completes, but offlineimap
will have given up by then already.

Has anyone else seen this?

Can you confirm this behaviour?

What should offlineimap be doing differently?

Thanks,

--

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

because light travels faster than sound,
some people appear to be intelligent,
until you hear them speak.

spamtraps: madduck.bogus <at> madduck.net
(Continue reading)

Permalink | Reply |
headers
Timo Sirainen | 13 Jun 2011 14:44
Picon
Picon
Favicon

Re: Timeout during APPEND

On Mon, 2011-06-13 at 10:11 +0200, martin f krafft wrote:
> also sprach martin f krafft <madduck <at> madduck.net> [2011.06.13.1002 +0200]:
> > One user reports continuous problems synchronising her mailbox via
> > IMAP (offlineimap, via SSH tunnel or SSL socket). It seems that she
> > has a large, locally-created message, but the uplink bandwidth seems
> > to be not enough to push it before dovecot times out the APPEND
> > command.
> 
> Upon further inspection, we found that the message *does* get saved
> remotely. Hence, this seems like an offlineimap problem, timing out
> because it receives no responses to APPEND (because the transfer
> takes so long). The transfer actually completes, but offlineimap
> will have given up by then already.
> 
> Has anyone else seen this?
> 
> Can you confirm this behaviour?
> 
> What should offlineimap be doing differently?

Timing out after only 30 seconds seems a bit aggressive to me,
especially if you're uploading a large message over a slow network
connection. Isn't it configurable?
Permalink | Reply |
headers
martin f krafft | 13 Jun 2011 15:11

Re: Timeout during APPEND

also sprach Timo Sirainen <tss <at> iki.fi> [2011.06.13.1444 +0200]:
> Timing out after only 30 seconds seems a bit aggressive to me,
> especially if you're uploading a large message over a slow network
> connection. Isn't it configurable?

Not that I can see, but I will check out the code later too.

The question is whether IMAP really limits us to using something
silly as timeouts. Couldn't the server keep sending BUSY messages,
or the like?

How could the client distinguish between an upload progressing, and
the connection having stalled. Does it look at the flow rate of
data, or how does IMAP cater for this requirement?

--

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

an egg has the shortest sex-life of all: if gets laid once; it gets
eaten once. it also has to come in a box with 11 others, and the
only person who will sit on its face is its mother.

spamtraps: madduck.bogus <at> madduck.net
Permalink | Reply |
headers
Timo Sirainen | 13 Jun 2011 16:23
Picon
Picon
Favicon

Re: Timeout during APPEND

On Mon, 2011-06-13 at 15:11 +0200, martin f krafft wrote:
> also sprach Timo Sirainen <tss <at> iki.fi> [2011.06.13.1444 +0200]:
> > Timing out after only 30 seconds seems a bit aggressive to me,
> > especially if you're uploading a large message over a slow network
> > connection. Isn't it configurable?
> 
> Not that I can see, but I will check out the code later too.
> 
> The question is whether IMAP really limits us to using something
> silly as timeouts. Couldn't the server keep sending BUSY messages,
> or the like?

It could, and Dovecot does that for several commands. But I'm a bit
afraid of adding such code for APPEND, because it could easily break
some clients. I know an old version of Evolution broke if it got any
extra data during APPEND.

> How could the client distinguish between an upload progressing, and
> the connection having stalled. Does it look at the flow rate of
> data, or how does IMAP cater for this requirement?

If your router/whatever swallows the entire 10 MB at once and starts
uploading it for the next 60 seconds, I guess there's nothing that a
client can do.
Permalink | Reply |
headers
martin f krafft | 14 Jun 2011 06:56

Re: Timeout during APPEND

also sprach Timo Sirainen <tss <at> iki.fi> [2011.06.13.1623 +0200]:
> It could, and Dovecot does that for several commands. But I'm a bit
> afraid of adding such code for APPEND, because it could easily break
> some clients. I know an old version of Evolution broke if it got any
> extra data during APPEND.

Couldn't the client signal to the server that it wants/expects such
data, and only then does dovecot send such pings?

--

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"...the prevailing catholic odor - incense, wax, centuries of mild
 bleating from the lips of the flock."
                                -- thomas pynchon, gravity's rainbow

spamtraps: madduck.bogus <at> madduck.net
Permalink | Reply |
headers
Timo Sirainen | 14 Jun 2011 14:54
Picon
Picon
Favicon

Re: Timeout during APPEND

On Tue, 2011-06-14 at 06:56 +0200, martin f krafft wrote:
> also sprach Timo Sirainen <tss <at> iki.fi> [2011.06.13.1623 +0200]:
> > It could, and Dovecot does that for several commands. But I'm a bit
> > afraid of adding such code for APPEND, because it could easily break
> > some clients. I know an old version of Evolution broke if it got any
> > extra data during APPEND.
> 
> Couldn't the client signal to the server that it wants/expects such
> data, and only then does dovecot send such pings?

Good luck getting any client to implement something like that.
Permalink | Reply |
headers
martin f krafft | 14 Jun 2011 14:59

Re: Timeout during APPEND

also sprach Timo Sirainen <tss <at> iki.fi> [2011.06.14.1454 +0200]:
> > Couldn't the client signal to the server that it wants/expects
> > such data, and only then does dovecot send such pings?
> 
> Good luck getting any client to implement something like that.

FYI: http://bugs.debian.org/630444

--

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

no cat has eight tails.
a cat has one tail more than no cat.
therefore, a cat has nine tails.

spamtraps: madduck.bogus <at> madduck.net
Permalink | Reply |

Gmane