Bill Ryder | 9 Aug 2012 22:04
Picon

Question about 2.4.x mailbox rename/locking behaviour

Hi all,

We've been running cyrus for over a decade now but I have management that really wants to put in dovecot.

We have two problems with cyrus in our environment which is constantly used as ammunition against it.

We are running 2.3.16 right now.

The problems are

1. Can not unreserve a mailbox without restarting cyrus - these can be caused by number 2 below
2. Mailbox renames - or deletes - can take a LONG time - and delivery stops eventually.

The renames (and mailbox deletes which for us are renames into Trash) are the killer since the mailbox is
locked during a move. 
We  have about a 1,000 users many of whom with 10 years of email. 100,000 emails in a folder is not uncommon.

A lot of our traffic is mailling lists and we deliver in batches of 50 to get a balance between hardlinking and
parallelism, So 
one locked mailbox in that batch hangs the entire batch.

Depending on fileserver load etc it can take 30 minutes to do one of these moves (artists often have wacom
email folder drag and 
drop events which can start huge unexpected mailbox renames) - in these cases if the mailbox is large I have
to kill the imapd 
doing the move and clean it up. This leads to a reserved mailbox that I can't clear without a reboot.

During these moves all dellivers into that folder lock up because cyrus.header is locked and the lmtp's try
to get a lock on that 
folder and they stop until postfix times out the delivery and defers it. Eventually we end up with a massive
(Continue reading)

Greg Banks | 13 Aug 2012 04:34
Gravatar

Re: Question about 2.4.x mailbox rename/locking behaviour


On Fri, Aug 10, 2012, at 06:04 AM, Bill Ryder wrote:
> Hi all,

Hi Bill,

Sorry for the slow reply, I was away Friday.

> We have two problems with cyrus in our environment which is constantly
> used as ammunition against it.
> 
> We are running 2.3.16 right now.
> 
> The problems are
> 
> 1. Can not unreserve a mailbox without restarting cyrus - these can be
> caused by number 2 below
> 2. Mailbox renames - or deletes - can take a LONG time - and delivery
> stops eventually.
> [...]
> Has anything changed in 2.4 that will stop this kind of thing from
> happening?

So I joined the project at the point when 2.3 was already in the past,
but on examining the source I see that both 2.4 and the development
branch that will be 2.5 both have the same behaviour you describe.  Each
individual file in a folder in linked to its new location and unlinked
again from it's old location, while keeping the mailbox lock.  This is a
long way south of optimal for the common case where an entire user is on
the same filesystem, but that code path is designed to handle all the
(Continue reading)


Gmane