Discussions on the Cyrus IMAP server.

Juris Orlovs | 9 Feb 2006 11:26

slow IMAP data transfer

Hello

I'm trying to set up Cyrus IMAP on freebsd 6.0, but i have problems with performance of it. There is a big issue
with message writing to IMAP folders and also with reading a large quantities of messages. Reading of 8MB
message takes a bit less than a second, but in that time 8MB message write to IMAP folder over 100Mbit/s
connection takes ~6 seconds longer. Also, as I am only one by now who uses this server there could be no
chance of big resouce utilization.

Reading/writing ~650 messages, each 1KB) from/to imap folder sends huge amounts of data to RAID 1 arrays.
It can be seen using an iostat in that process.

Hardware (2x 3GHx Xeon, 2GB RAM, Adaptec 2410SA,
2x WD RAPTOR 74GB (aacd0)(system), 2x SEAGATE 300GB (aacd1)(mail storage))

Before that I was setting up Cyrus IMAP mail servers on Linux and there was no such problems. Maybe it's OS
issue? I also have set up such system on Celeron 800MHz, 256MB RAM and 80GB SEAGATE (FreeBSD 4.11), the
results are aproximately the same.
For SMTP i'm using Postfix, and it also shows bad results on data transfer. Seems that the biggest problems I
have with mail software, because other services like FTP doesn't show such bad results.

--------------------------------------------------------------------
READING 650 messages
--------------------------------------------------------------------
mx# iostat -Iw 3
      tty           aacd0            aacd1             cpu
 tin tout  KB/t xfrs   MB   KB/t xfrs   MB  us ni sy in id
   0   50 17.54 390334 6687.51  22.46 296070 6492.93   0  0  0  0 100
   0   64 16.00  41  0.64  16.00   1  0.02   0  0  0  0 100
   0   20 13.00   8  0.10   0.00   0  0.00   1  0  0  0 99
   0   20  0.00   0  0.00   0.00   0  0.00   0  0  0  0 100

(Continue reading)

headers

David Carter | 9 Feb 2006 11:46

Re: slow IMAP data transfer

On Thu, 9 Feb 2006, Juris Orlovs wrote:

> Before that I was setting up Cyrus IMAP mail servers on Linux and there 
> was no such problems. Maybe it's OS issue? I also have set up such 
> system on Celeron 800MHz, 256MB RAM and 80GB SEAGATE (FreeBSD 4.11), the 
> results are aproximately the same. For SMTP i'm using Postfix, and it 
> also shows bad results on data transfer.

Cyrus and Postfix both make very heavy use of fsync() to flush data to 
disk before they confirm actions back to the sending system. RAID 
controllers or disk boxes with battery backed writeback cache help lots.

--

-- 
David Carter                             Email: David.Carter <at> ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.
----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

headers

Michael Loftis | 9 Feb 2006 22:21

Re: slow IMAP data transfer

--On February 9, 2006 10:46:29 AM +0000 David Carter <dpc22 <at> cam.ac.uk> 
wrote:

> Cyrus and Postfix both make very heavy use of fsync() to flush data to
> disk before they confirm actions back to the sending system. RAID
> controllers or disk boxes with battery backed writeback cache help lots.

I'll confirm what David and Henrique both posted.  Cyrus, postfix, and 
friends, make a LOT of use of fsync.  This will cause a lot of I/O, esp if 
your RAID doesn't have a write cache.  I'd also suggest only using a 
battery backed write cache.

We see similar behavior on Linux with Cyrus, Postfix, and MailScanner (also 
running SpamAssassin).  If/when we get into a situation where our AX100 or 
CX200 disable their write caches the mailserver usually pretty much pukes 
since it's constantly causing sync's.

----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

headers

Henrique de Moraes Holschuh | 9 Feb 2006 11:43

Re: slow IMAP data transfer

On Thu, 09 Feb 2006, Juris Orlovs wrote:
> For SMTP i'm using Postfix, and it also shows bad results on data
> transfer. Seems that the biggest problems I have with mail software,
> because other services like FTP doesn't show such bad results.

I am by no means a FreeBSD expert, but I would check fsync() behaviour for
the filesystem you are using. ftp and other "normal" software doesn't
fsync() anything.  All proper mail-handling software, including Cyrus and
postfix, use fsync() a lot.

--

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh
----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html