Farkas Levente | 17 Jun 17:02

dovecot as gmail imap proxy

hi,
may be this is a bit of topic, but i hope not. it's always a big 
question which is the better to choose a hosting provider to keep and 
manage you mail or setup your own mail server, hosting, virus and spam 
filter etc.. now as more and more people and company move to gmail or 
google apps mail service it seems google can't handle the load and it's 
getting more and more slower. what's more not just the imap interface 
but nowadays the web interface used to be hang or even stop working. 
it's getting more and more annoying that the speed is worst than in a 10 
years ago. i thing about how can this be solved without totally give up 
gmail as a mail service provider (and may be they can solve it in a few 
months/years:-).
would it be possible to create a dovecot server as an imap proxy for 
gmail and google apps?
imho many people/company would be love to use such a setup where he can 
use gmail in vacation or any other time, but when the email is real 
critical apps (not acceptable to wait 1 minutes for an email to load or 
to open a folder/label) than he can use a proxy server eg. dovecot which 
act as a imap proxy to gmail and can be hosted on closer, faster server 
and this speed problem never happend. what's more it can be used as a 
mail backup server too.
anybody has any such setup, idea? would it be possible with dovecot.
any other suggestion?
yours.

--

-- 
   Levente                               "Si vis pacem para bellum!"

Alexander Prinsier | 17 Jun 17:13
Favicon

Re: dovecot as gmail imap proxy

To have a speed increase, dovecot needs direct access to the emails. So
you'll need them stored locally. You can't use it as a proxy, just to
speed up things, because dovecot would just need to wait for gmail too,
to load an email (except if dovecot would cache the emails, but then
again, it's just a cache, and you're talking about 'critical applications').

I'm not sure, but I don't think there's even a way to run dovecot as a
proxy.

Why don't you forward mail from your gmail account to your own server?
That's so much easier and more common. You get the benefit of good
filtering, and you have your own server to access your mail. If your own
server goes down, you still have gmail as backup.

Alexander

Farkas Levente wrote:
> hi,
> may be this is a bit of topic, but i hope not. it's always a big
> question which is the better to choose a hosting provider to keep and
> manage you mail or setup your own mail server, hosting, virus and spam
> filter etc.. now as more and more people and company move to gmail or
> google apps mail service it seems google can't handle the load and it's
> getting more and more slower. what's more not just the imap interface
> but nowadays the web interface used to be hang or even stop working.
> it's getting more and more annoying that the speed is worst than in a 10
> years ago. i thing about how can this be solved without totally give up
> gmail as a mail service provider (and may be they can solve it in a few
> months/years:-).
> would it be possible to create a dovecot server as an imap proxy for
(Continue reading)

Peter Hessler | 17 Jun 21:48

Re: dovecot as gmail imap proxy

I do this exact thing, and it is faster than gmail itself.  With IDLE, I 
get the msg alert on my dovecot server before the web interface shows a 
new message.

Also, far more reliable, with a superior UI.

On 2008 Jun 17 (Tue) at 17:13:32 +0200 (+0200), Alexander Prinsier wrote:
:Why don't you forward mail from your gmail account to your own server?
:That's so much easier and more common. You get the benefit of good
:filtering, and you have your own server to access your mail. If your own
:server goes down, you still have gmail as backup.

--

-- 
New Hampshire law forbids you to tap your feet, nod your head, or in
any way keep time to the music in a tavern, restaurant, or cafe.

Farkas Levente | 17 Jun 22:22

Re: dovecot as gmail imap proxy

Peter Hessler wrote:
> I do this exact thing, and it is faster than gmail itself.  With IDLE, I 
> get the msg alert on my dovecot server before the web interface shows a 
> new message.

which exact thing?
and what does the "with idle" means?

> Also, far more reliable, with a superior UI.
> 
> 
> On 2008 Jun 17 (Tue) at 17:13:32 +0200 (+0200), Alexander Prinsier wrote:
> :Why don't you forward mail from your gmail account to your own server?
> :That's so much easier and more common. You get the benefit of good
> :filtering, and you have your own server to access your mail. If your own
> :server goes down, you still have gmail as backup.
> 
	

--

-- 
   Levente                               "Si vis pacem para bellum!"

Asheesh Laroia | 17 Jun 23:18

Re: dovecot as gmail imap proxy

On Tue, 17 Jun 2008, Peter Hessler wrote:

> I do this exact thing, and it is faster than gmail itself.  With IDLE, I 
> get the msg alert on my dovecot server before the web interface shows a 
> new message.
>
> Also, far more reliable, with a superior UI.

Which is the superior UI?  Honest question - I'm shopping around for nice 
IMAP interfaces, if that's what you mean.  I'd love to know what you're 
using.

-- Asheesh.

--

-- 
Marriage is the triumph of imagination over intelligence.  Second marriage is
the triumph of hope over experience.

Peter Hessler | 18 Jun 00:37

Re: dovecot as gmail imap proxy

On 2008 Jun 17 (Tue) at 17:18:20 -0400 (-0400), Asheesh Laroia wrote:
> On Tue, 17 Jun 2008, Peter Hessler wrote:
>
>> I do this exact thing, and it is faster than gmail itself.  With IDLE, 
>> I get the msg alert on my dovecot server before the web interface shows 
>> a new message.
>>
>> Also, far more reliable, with a superior UI.
>
> Which is the superior UI?  Honest question - I'm shopping around for nice 
> IMAP interfaces, if that's what you mean.  I'd love to know what you're  
> using.
>
> -- Asheesh.
>
> -- 
> Marriage is the triumph of imagination over intelligence.  Second marriage is
> the triumph of hope over experience.
>

I <3 mutt.  Easy enough, but lots of power is available.

I haven't found any webmail interfaces that I like, but then again, I have yet 
to find *any* web interface that I like.

--

-- 
Marriage is the only adventure open to the cowardly.
		-- Voltaire

(Continue reading)

Geert Hendrickx | 18 Jun 10:32
Favicon

Re: dovecot as gmail imap proxy

On Tue, Jun 17, 2008 at 03:37:30PM -0700, Peter Hessler wrote:
> I <3 mutt.  Easy enough, but lots of power is available.

Yes, mutt is great! :-)

Btw, you can use mutt with dovecot without running dovecot as a daemon/server,
by putting this in your .muttrc:

set tunnel="MAIL=/home/geert/mbox /usr/pkg/libexec/dovecot/imap"

(adapt paths for your own system)

This way mutt will talk to the dovecot imap binary via stdin/stdout.  This
will also work over ssh.  This way you don't need to run dovecot as a server
and you still benefit from eg. the cache etc.

	Geert

Timo Sirainen | 18 Jun 10:44
Favicon

Re: dovecot as gmail imap proxy

On Wed, 2008-06-18 at 10:32 +0200, Geert Hendrickx wrote:
> On Tue, Jun 17, 2008 at 03:37:30PM -0700, Peter Hessler wrote:
> > I <3 mutt.  Easy enough, but lots of power is available.
> 
> 
> Yes, mutt is great! :-)
> 
> Btw, you can use mutt with dovecot without running dovecot as a daemon/server,
> by putting this in your .muttrc:
> 
> set tunnel="MAIL=/home/geert/mbox /usr/pkg/libexec/dovecot/imap"

"dovecot --exec-mail imap" is better. Then it uses the settings from
dovecot.conf as well as the proper default settings. With the above
you're for example using mbox_lazy_writes=no

Timo Sirainen | 17 Jun 17:14
Favicon

Re: dovecot as gmail imap proxy

On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:

> would it be possible to create a dovecot server as an imap proxy for  
> gmail and google apps?
> imho many people/company would be love to use such a setup where he  
> can use gmail in vacation or any other time, but when the email is  
> real critical apps (not acceptable to wait 1 minutes for an email to  
> load or to open a folder/label) than he can use a proxy server eg.  
> dovecot which act as a imap proxy to gmail and can be hosted on  
> closer, faster server and this speed problem never happend. what's  
> more it can be used as a mail backup server too.

I've thought about a caching IMAP mail storage backend for Dovecot and  
even started implementing it once, but it didn't get very far. If  
someone wants to try to continue it, I can send what I've written so  
far. Although it's mostly a copy of cydir backend with a tiny bit of  
it replaced with IMAP code.

Timo Sirainen | 17 Jun 17:21
Favicon

Re: dovecot as gmail imap proxy

On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:

> On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
>
>> would it be possible to create a dovecot server as an imap proxy  
>> for gmail and google apps?
>> imho many people/company would be love to use such a setup where he  
>> can use gmail in vacation or any other time, but when the email is  
>> real critical apps (not acceptable to wait 1 minutes for an email  
>> to load or to open a folder/label) than he can use a proxy server  
>> eg. dovecot which act as a imap proxy to gmail and can be hosted on  
>> closer, faster server and this speed problem never happend. what's  
>> more it can be used as a mail backup server too.
>
> I've thought about a caching IMAP mail storage backend for Dovecot  
> and even started implementing it once, but it didn't get very far.  
> If someone wants to try to continue it, I can send what I've written  
> so far. Although it's mostly a copy of cydir backend with a tiny bit  
> of it replaced with IMAP code.

Now that I think of it, the IMAP backend should do only direct IMAP  
backend accessing and if caching is wanted it could be a generic  
middle layer. Might be also useful for other things like caching mails  
from NFS server. Although Linux already has a cachefs for that.
Farkas Levente | 17 Jun 18:17

Re: dovecot as gmail imap proxy

Timo Sirainen wrote:
> On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:
> 
>> On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
>>
>>> would it be possible to create a dovecot server as an imap proxy for 
>>> gmail and google apps?
>>> imho many people/company would be love to use such a setup where he 
>>> can use gmail in vacation or any other time, but when the email is 
>>> real critical apps (not acceptable to wait 1 minutes for an email to 
>>> load or to open a folder/label) than he can use a proxy server eg. 
>>> dovecot which act as a imap proxy to gmail and can be hosted on 
>>> closer, faster server and this speed problem never happend. what's 
>>> more it can be used as a mail backup server too.
>>
>> I've thought about a caching IMAP mail storage backend for Dovecot and 
>> even started implementing it once, but it didn't get very far. If 
>> someone wants to try to continue it, I can send what I've written so 
>> far. Although it's mostly a copy of cydir backend with a tiny bit of 
>> it replaced with IMAP code.
> 
> Now that I think of it, the IMAP backend should do only direct IMAP 
> backend accessing and if caching is wanted it could be a generic middle 
> layer. Might be also useful for other things like caching mails from NFS 
> server. Although Linux already has a cachefs for that.

yes, that's exactly what i'm thinking about. all imap command which 
issued by the client to the middle layer should have to propagate back 
to the backed (in this case gmail). but imho the hard part that we've to 
assume that the backend (in this case gmail) is slow, so the middle 
(Continue reading)

Farkas Levente | 17 Jun 22:24

Re: dovecot as gmail imap proxy

Timo Sirainen wrote:
> On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:
> 
>> On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
>>
>>> would it be possible to create a dovecot server as an imap proxy for 
>>> gmail and google apps?
>>> imho many people/company would be love to use such a setup where he 
>>> can use gmail in vacation or any other time, but when the email is 
>>> real critical apps (not acceptable to wait 1 minutes for an email to 
>>> load or to open a folder/label) than he can use a proxy server eg. 
>>> dovecot which act as a imap proxy to gmail and can be hosted on 
>>> closer, faster server and this speed problem never happend. what's 
>>> more it can be used as a mail backup server too.
>>
>> I've thought about a caching IMAP mail storage backend for Dovecot and 
>> even started implementing it once, but it didn't get very far. If 
>> someone wants to try to continue it, I can send what I've written so 
>> far. Although it's mostly a copy of cydir backend with a tiny bit of 
>> it replaced with IMAP code.
> 
> Now that I think of it, the IMAP backend should do only direct IMAP 
> backend accessing and if caching is wanted it could be a generic middle 
> layer. Might be also useful for other things like caching mails from NFS 
> server. Although Linux already has a cachefs for that.

it seems other people also thing about this:
http://community.livejournal.com/linux/1748995.html
so it'd be useful thing for many of us:-)

(Continue reading)

Asheesh Laroia | 17 Jun 23:21

Re: dovecot as gmail imap proxy

On Tue, 17 Jun 2008, Farkas Levente wrote:

> hi,
> may be this is a bit of topic, but i hope not. it's always a big question 
> which is the better to choose a hosting provider to keep and manage you mail 
> or setup your own mail server, hosting, virus and spam filter etc.. now as 
> more and more people and company move to gmail or google apps mail service it 
> seems google can't handle the load and it's getting more and more slower. 
> what's more not just the imap interface but nowadays the web interface used 
> to be hang or even stop working. it's getting more and more annoying that the 
> speed is worst than in a 10 years ago. i thing about how can this be solved 
> without totally give up gmail as a mail service provider (and may be they can 
> solve it in a few months/years:-).
> would it be possible to create a dovecot server as an imap proxy for gmail 
> and google apps?

What I have is:

* rose, my primary server (but far away)
* supercore, another server at my parents' house

rose is the primary MX.

I run offlineimap on supercore to sync a local Maildir on supercore with 
rose's IMAP - that way, my parents can use a server with <1ms ping, and as 
they use Dovecot to modify the local Maildir on supercore, a minute 
later offlineimap will wake up and create IMAP operations to synchronize 
rose with the state on supercore.

FWIW.
(Continue reading)

Farkas Levente | 18 Jun 13:59

Re: dovecot as gmail imap proxy

Asheesh Laroia wrote:
> On Tue, 17 Jun 2008, Farkas Levente wrote:
> 
>> hi,
>> may be this is a bit of topic, but i hope not. it's always a big 
>> question which is the better to choose a hosting provider to keep and 
>> manage you mail or setup your own mail server, hosting, virus and spam 
>> filter etc.. now as more and more people and company move to gmail or 
>> google apps mail service it seems google can't handle the load and 
>> it's getting more and more slower. what's more not just the imap 
>> interface but nowadays the web interface used to be hang or even stop 
>> working. it's getting more and more annoying that the speed is worst 
>> than in a 10 years ago. i thing about how can this be solved without 
>> totally give up gmail as a mail service provider (and may be they can 
>> solve it in a few months/years:-).
>> would it be possible to create a dovecot server as an imap proxy for 
>> gmail and google apps?
> 
> What I have is:
> 
> * rose, my primary server (but far away)
> * supercore, another server at my parents' house
> 
> rose is the primary MX.
> 
> I run offlineimap on supercore to sync a local Maildir on supercore with 
> rose's IMAP - that way, my parents can use a server with <1ms ping, and 
> as they use Dovecot to modify the local Maildir on supercore, a minute 
> later offlineimap will wake up and create IMAP operations to synchronize 
> rose with the state on supercore.
(Continue reading)

Timo Sirainen | 18 Jun 22:05
Favicon

Re: dovecot as gmail imap proxy

On Jun 18, 2008, at 2:59 PM, Farkas Levente wrote:

> Timo what do you think how much work would be add such a middleware  
> imap server capabilities (ie. imap backend) to dovecot?

I think it would be quite a lot of work. Definitely a lot more than I  
have time for currently. The most difficult thing is probably writing  
IMAP parsing and event handling for it. I've written such code for my  
imaptest tool already though, maybe at some point they could be more  
or less merged to a common lib-imap-client.


Gmane