gmane.mail.imap.uw.c-client

millay | 4 May 06:01

Hard filesystem quota interaction with uw-imap

Does uw-imap create or use temporary files owned by the user connecting to
imapd? I'm unable to find any documentation or mailing list threads
related to this topic.
I run two FreeBSD mail servers (one 6.1 the other 6.3). The 6.1 server is
running imap-uw-2004g while the 6.3 server is running imap-uw-2006j (both
compiled from the ports tree with the same options to check mail in /home
instead of  /var/mail). Both servers are using soft and hard filesystem
quotas. Clients are connecting with squirrelmail web mail client on a
third server.
The problem I am experiencing is that when a user who's mail is stored on
the server running imap-uw-2006j and they are close to their hard quota
the get the following message reported by squirrelmail:

ERROR : Could not complete request.
Query: SELECT "INBOX"
Reason Given: Unable to extend mailbox: Disc quota exceeded

The last line being what is returned by uw-imap. The user's quota usage
looks like this on the file system:

[root <at> leopard /home/r]# quota -u rhetoric
Disk quotas for user rhetoric (uid 23788):
     Filesystem   usage   quota   limit   grace   files   quota   limit  
grace
          /home   12284*  10240   12288 7days      10       0       0

This is a problem because users cannot see their list of messages to be
able to delete them. Interestingly enough, if I rsync the user's home
folder over to the 6.1 server running imap-uw-2004g and check that account
with squirrel mail I can see the list of the user's messages just fine.

(Continue reading)

headers

Mark Crispin | 4 May 07:09

Re: Hard filesystem quota interaction with uw-imap

First, you should upgrade from 2006j to 2007a immediately.  2006j was an 
intermediate version.

The only temporary files are small (a few bytes) files on /tmp used to aid 
in locking.

Assuming that you are using the traditional UNIX mailbox format, note that 
imap-2006 implemented UIDPLUS functionality, and as a result requires the 
immediate assignment of UIDs to the message metadata in new messages. 
That probably accounts for the difference; it needs to update in the new 
messages at select time, rather than a lazy update that is deferred until 
the user deletes and expunges messages.

I strongly urge that you not have hard quotas set so close to the soft 
quotas.  Hard quotas are bad juju, especially in the traditional UNIX 
mailbox format which can actually *increase* in size as a result of being 
opened due to the need to insert message metadata.

You should, however, have some policy mechanism to put teeth into the soft 
quotas.  At some point, I will make a public distribution of how UW does 
internal quota enforcement as sample code.  Briefly, UW allows a user to 
exceed soft quota for up to 7 days or a threshold (which is still much 
less than any hard limit); if either is exceeded then mail delivery is 
shut off until the user gets below the soft quota.  It's not impossible, 
but it requires considerable talent and effort to reach the hard quota.

On Sun, 4 May 2008, millay <at> sover.net wrote:
> Does uw-imap create or use temporary files owned by the user connecting to
> imapd? I'm unable to find any documentation or mailing list threads
> related to this topic.

(Continue reading)

headers

millay | 4 May 07:32

Re: Hard filesystem quota interaction with uw-imap

Mark,
Thanks for the quick reply. I think you may have nailed the problem with
the  change in when UIDs are written and yes I will look to possibly
upgrade ASAP. One thing I noticed shortly after writing this message to
the list however was that the server running 2006j is also creating a
dot-lock file in the user's home directory (INBOX.lock) while the server
running 2004g is not. Is this a function of uw-imap or c-client and is it
needed in addition to the locking mechanism in /tmp? If not, is there any
way to disable that functionality?

I agree that implementing a policy to not allow a user to get much further
than their soft quota would be ideal but at this point its not something I
can do right away at my place of employment. In the mean time is there an
easy way to disable UIDPLUS or should I just try and roll the 2006j box
back to 2004g? I'm really just looking for a quick fix until I can
implement a more elegant solution with quotas. Anything I can do to make
the server not add anything to the size of a user's home directory when
they check their mail that close to their soft quota.

Your help is greatly appreciated. Thanks again for the quick reply and the
knowledgeable answers you are providing!
Jeremiah

> First, you should upgrade from 2006j to 2007a immediately.  2006j was an
> intermediate version.
>
> The only temporary files are small (a few bytes) files on /tmp used to aid
> in locking.
>
> Assuming that you are using the traditional UNIX mailbox format, note that

(Continue reading)

headers

Mark Crispin | 4 May 07:42

Re: Hard filesystem quota interaction with uw-imap

UW imapd always has written the .lock file.  If your old version does not, 
then it was hacked not to do so.  I do not recommend making such a hack; 
the .lock files are written for a reason.  If you do so anyway, you're on 
your own.

The /tmp lock files accomplish a completely different purpose which 
operates simultaneously with the .lock files.  There are four levels of 
locking; two having to do with "having the mailbox open for message 
access" and two having to do with the physical act of reading/writing 
(including appending) to the mailbox.  These are independent of each 
other; otherwise it would not be possible to have new mail delivery why a 
mailbox was open.

There is no easy way to "disable UIDPLUS".  It isn't just a matter of 
UIDPLUS as a new capability; the entire update code in the drivers was 
rewritten to accomodate the demands of UIDPLUS.  So, you would in effect 
have to rewrite the code back.

It's your choice how you want to deal with this; but I will note that the 
mbx and mix formats preallocate the metadata space and thus are not 
subject to these issues.  If you have users with large mailboxes you 
should think about switching them to use one of the other formats.

On Sun, 4 May 2008, millay <at> sover.net wrote:
> Mark,
> Thanks for the quick reply. I think you may have nailed the problem with
> the  change in when UIDs are written and yes I will look to possibly
> upgrade ASAP. One thing I noticed shortly after writing this message to
> the list however was that the server running 2006j is also creating a
> dot-lock file in the user's home directory (INBOX.lock) while the server

(Continue reading)

headers

Jeremiah Millay | 4 May 07:56

Re: Hard filesystem quota interaction with uw-imap

Yes this would be a temporary solution. I appreciate your input on the 
topic. I think I will roll my newer box back to 2004g for now. I 
definitely agree with you that this is not a uw-imap problem but rather 
the way that quotas are enforced on my mail system... something I inherited.
Jeremiah

Mark Crispin wrote:
> UW imapd always has written the .lock file.  If your old version does 
> not, then it was hacked not to do so.  I do not recommend making such 
> a hack; the .lock files are written for a reason.  If you do so 
> anyway, you're on your own.
>
> The /tmp lock files accomplish a completely different purpose which 
> operates simultaneously with the .lock files.  There are four levels 
> of locking; two having to do with "having the mailbox open for message 
> access" and two having to do with the physical act of reading/writing 
> (including appending) to the mailbox.  These are independent of each 
> other; otherwise it would not be possible to have new mail delivery 
> why a mailbox was open.
>
> There is no easy way to "disable UIDPLUS".  It isn't just a matter of 
> UIDPLUS as a new capability; the entire update code in the drivers was 
> rewritten to accomodate the demands of UIDPLUS.  So, you would in 
> effect have to rewrite the code back.
>
> It's your choice how you want to deal with this; but I will note that 
> the mbx and mix formats preallocate the metadata space and thus are 
> not subject to these issues.  If you have users with large mailboxes 
> you should think about switching them to use one of the other formats.
>

(Continue reading)