Postfix MTA user discussion

Gábor Lénárt | 3 Jul 12:19

Picon

MTA or SMTP proxy?

Hei,

Recently I am thinking of reimplementing our MX servers. Of course rcpt
check is a must, also I should not generate NDRs later, I should only accept
mails from outside which won't cause bounce on the mail store MTA later. My
problem: there are some conditions cannot be forecasted easily, I can't be
sure that mail can be delivered (over quota, user is deleted between the
moments of mail at MX and the mail storage etc). Some people would say, that
this issue is minor, and quite rare so I should not care about this. However
I think even rcpt checking can be eliminated on MX servers if I have some
kind of SMTP proxy like thing (instead of "real" MTA with queue) at the MX
servers. Since mail store won't accept the mail for non-existing rcpt
address, an SMTP proxy like solution does not even need rcpt check itself.
Also the over quota problem is solved. Also, I hate when there is some rare
problem with the backend mail servers (mail stores) and we have huge queues
filled up with waiting mails, I would say, that MX servers shouldn't accept
anything if they are not 100% sure that they can transfer the mail to the
backend. Since MTAs should have queue anyway, I am not sure this kind of
SMTP proxy server can be implemented with postfix (it would be great since I
would be able to use its policy server and maybe milter too), the key
feature that it shouldn't use any queue, when sender MTA issues the
corresponding SMTP commands, it should do that for the backend and giving
back the result. What do you think, is it a bad idea? I know this is really
not like a traditional MTA, should I use some kind of SMTP proxy solution
instead then? Any idea? Thanks a lot.

--

-- 
- Gábor

mouss | 4 Jul 00:05

Re: MTA or SMTP proxy?

Gábor Lénárt a écrit :
*

> Hei,
> 
> Recently I am thinking of reimplementing our MX servers. Of course rcpt
> check is a must, also I should not generate NDRs later, I should only accept
> mails from outside which won't cause bounce on the mail store MTA later. My
> problem: there are some conditions cannot be forecasted easily, I can't be
> sure that mail can be delivered (over quota, user is deleted between the
> moments of mail at MX and the mail storage etc). Some people would say, that
> this issue is minor, and quite rare so I should not care about this. However
> I think even rcpt checking can be eliminated on MX servers if I have some
> kind of SMTP proxy like thing (instead of "real" MTA with queue) at the MX
> servers. Since mail store won't accept the mail for non-existing rcpt
> address, an SMTP proxy like solution does not even need rcpt check itself.
> Also the over quota problem is solved. Also, I hate when there is some rare
> problem with the backend mail servers (mail stores) and we have huge queues
> filled up with waiting mails, I would say, that MX servers shouldn't accept
> anything if they are not 100% sure that they can transfer the mail to the
> backend. Since MTAs should have queue anyway, I am not sure this kind of
> SMTP proxy server can be implemented with postfix (it would be great since I
> would be able to use its policy server and maybe milter too), the key
> feature that it shouldn't use any queue, when sender MTA issues the
> corresponding SMTP commands, it should do that for the backend and giving
> back the result. What do you think, is it a bad idea? I know this is really
> not like a traditional MTA, should I use some kind of SMTP proxy solution
> instead then? Any idea? Thanks a lot.
> 

if you use a "proxy", then you expose the final MTA. you lose the
possibility to "aggregate" connections (connection caching). at one
time, you will need to check why you need a proxy at all. if it is to
protect a robust MTA such as postfix, then this is plain waste. if it is
to protect a "weak" MTA, then you don't want to expose the MTA at all.

queuing has a benefit: "channel attacks" don't succeed in real time. the
attacker needs to guess when the message is delivered. the old smap (TIS
fwtk) initially had no other benefit than isolating the client and the
server by adding an intermediary queue (so that attacks that could cause
sendmail to open a hole could no more work, since the sendmail process
is not accessible via the internet).

I am not saying that a "queue-less" proxy doesn't have a place, but you
need to justify its use. In particular, I wouldn't use assp (despite its
good "things").

Barney Desmond | 3 Jul 13:55

Picon

Re: MTA or SMTP proxy?

2009/7/3 Gábor Lénárt <lgb <at> lgb.hu>:
*

> Hei,
>
> Recently I am thinking of reimplementing our MX servers. Of course rcpt
> check is a must, also I should not generate NDRs later, I should only accept
<snip>

Huh?

This is all very standard behaviour for an MTA. Recipient checking is
a very common task for most Postfix users. Regardless of what you
expect or want, SMTP is a store-and-forward protocol - queues are a
strong component for an MTA.

You say you don't want to accept mail unless you're certain that the
backend can accept it, but the only way to know is to try. If it
succeeds then you might as well take it, you've already succeeded. In
any case, this amounts to simply collapsing all the layers of your
mail system, from receipt to storage. This of course depends on how
you expect to implement the systems, which you've not described.

Aaron Wolfe | 3 Jul 14:14

Picon

Re: MTA or SMTP proxy?

*

On Fri, Jul 3, 2009 at 7:55 AM, Barney Desmond<barneydesmond <at> gmail.com> wrote:
> 2009/7/3 Gábor Lénárt <lgb <at> lgb.hu>:
>> Hei,
>>
>> Recently I am thinking of reimplementing our MX servers. Of course rcpt
>> check is a must, also I should not generate NDRs later, I should only accept
> <snip>
>
> Huh?
>
> This is all very standard behaviour for an MTA. Recipient checking is
> a very common task for most Postfix users. Regardless of what you
> expect or want, SMTP is a store-and-forward protocol - queues are a
> strong component for an MTA.
>
> You say you don't want to accept mail unless you're certain that the
> backend can accept it, but the only way to know is to try. If it
> succeeds then you might as well take it, you've already succeeded. In
> any case, this amounts to simply collapsing all the layers of your
> mail system, from receipt to storage. This of course depends on how
> you expect to implement the systems, which you've not described.
>

Right, NDRs are not always bad.  Generating millions of the for dumb
reasons (in response to dictionary attacks for example) is bad.
Sending one when there is an unforeseeable problem such as those you
mention is really ok and a good thing.  They serve an important
purpose.

Gábor Lénárt | 3 Jul 14:12

Picon

Re: MTA or SMTP proxy?

On Fri, Jul 03, 2009 at 09:55:40PM +1000, Barney Desmond wrote:
> > Recently I am thinking of reimplementing our MX servers. Of course rcpt
> > check is a must, also I should not generate NDRs later, I should only accept
> <snip>
> 
> This is all very standard behaviour for an MTA. Recipient checking is
> a very common task for most Postfix users. Regardless of what you
> expect or want, SMTP is a store-and-forward protocol - queues are a
> strong component for an MTA.

I don't think so, SMTP itself is just a protocol, you can use it without
having and storage too. It's another question that how useful it is then ...

And rcpt check was just a question. I just say it's quite useless to do a
separated rcpt check on the MX servers, I mean not with some database lookup
with SMTP which is used to transfer the mail further too, so no other
infrastructure needed then (and that connection will be kept open to do the
transfer as well not just the mail "acceptance" check)

> 
> You say you don't want to accept mail unless you're certain that the
> backend can accept it, but the only way to know is to try. If it
> succeeds then you might as well take it, you've already succeeded. In
> any case, this amounts to simply collapsing all the layers of your   
> mail system, from receipt to storage. This of course depends on how
> you expect to implement the systems, which you've not described.

Yes I would try, at the same time as SMTP commands arriving to the MX
servers. That's why I said, it's more like an SMTP proxy function and not an
MTA - maybe. So, for example, I can do, after receiving the MAIL FROM and
RCPT TO commands, but before giving answer for DATA from the sender, I would
open an SMTP connection towards the backand and "play back" the SMTP
transaction, if it accepts, it's ok. Surely, if OK I will continue to
transfer the mail itself (after DATA). It's a proxy like functionality,
that's why I think it's not so postfix oriented question, but I asked that
here, because otherwise I would like to do extra checkings with very complex
rules implemented by a postfix policy server for example, because we have
already experience to do that. Just if I try to think about this more, it
turns out that postfix secure and multi daemon design does not allow this
kind of processing, what I described here. So in nutshell: this solution
would work like (for example) the squid http cache (but without caching), so
the HTTP request itself is got, then the server does not close the
connection with the client of course, and same time it tries to serve the
request to send a request to a server, and (if everything is OK) answering
to the client then. Also maybe it's a stupid example, but as with this
squid, you can implement extra ACLs, and complex rules as well to controll
the stuff, of course it's not a simple 1:1 repeater, it wouldn't make too
much sense then to have it.

--

-- 
- Gábor

Jozsef Kadlecsik | 3 Jul 14:30

Picon

Re: MTA or SMTP proxy?

*

On Fri, 3 Jul 2009, Gábor Lénárt wrote:

> On Fri, Jul 03, 2009 at 09:55:40PM +1000, Barney Desmond wrote:
> > > Recently I am thinking of reimplementing our MX servers. Of course rcpt
> > > check is a must, also I should not generate NDRs later, I should only accept
> > <snip>
> > 
> > This is all very standard behaviour for an MTA. Recipient checking is
> > a very common task for most Postfix users. Regardless of what you
> > expect or want, SMTP is a store-and-forward protocol - queues are a
> > strong component for an MTA.
> 
> I don't think so, SMTP itself is just a protocol, you can use it without
> having and storage too. It's another question that how useful it is then ...

SMTP is a store-and-forward protocol, it does require queues. You cannot 
simply wave aside this requirement.

I think it cannot be solved with an SMTP proxy "without queues" either: 
the final destination must queue the message first. After that it can 
attempt a local delivery, which may fail, for example due to the user 
being over quota. Therefore at the proxy side you also have to maintain 
the mail queue.

Best regards,
Jozsef
-
E-mail  : kadlec <at> blackhole.kfki.hu, kadlec <at> mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

mouss | 3 Jul 23:50

Re: MTA or SMTP proxy?

Jozsef Kadlecsik a écrit :
> On Fri, 3 Jul 2009, Gábor Lénárt wrote:
> 
>> On Fri, Jul 03, 2009 at 09:55:40PM +1000, Barney Desmond wrote:
>>>> Recently I am thinking of reimplementing our MX servers. Of course rcpt
>>>> check is a must, also I should not generate NDRs later, I should only accept
>>> <snip>
>>>
>>> This is all very standard behaviour for an MTA. Recipient checking is
>>> a very common task for most Postfix users. Regardless of what you
>>> expect or want, SMTP is a store-and-forward protocol - queues are a
>>> strong component for an MTA.
>> I don't think so, SMTP itself is just a protocol, you can use it without
>> having and storage too. It's another question that how useful it is then ...
> 
> SMTP is a store-and-forward protocol, it does require queues. You cannot 
> simply wave aside this requirement.
> 

This is wrong. an smtp proxy (without a queue) is more than feasible.
some people even use it (assp, ...). smtp does not mandate a queue.

> I think it cannot be solved with an SMTP proxy "without queues" either: 
> the final destination must queue the message first. After that it can 
> attempt a local delivery, which may fail, 

In theory at least, a final delivery MTA could deliver directly, without
a queue. of course, there is the problem of privileges. but for example,
on a system where the whole mailstore belongs to a single (virtual)
uid:gid, then the final MTA can deliver directly to a maildir. some
people even think this should be the way to deliver mail ("immediate
delivery").

> for example due to the user 
> being over quota. Therefore at the proxy side you also have to maintain 
> the mail queue.

Victor Duchovni | 3 Jul 23:54

Re: MTA or SMTP proxy?

*

On Fri, Jul 03, 2009 at 11:50:26PM +0200, mouss wrote:

> 
> In theory at least, a final delivery MTA could deliver directly, without
> a queue. of course, there is the problem of privileges. but for example,
> on a system where the whole mailstore belongs to a single (virtual)
> uid:gid, then the final MTA can deliver directly to a maildir. some
> people even think this should be the way to deliver mail ("immediate
> delivery").

This cannot be atomically for multi-recipient mail. SMTP requires
a single response for all recipients after ".", this requires either:

	- a Queue

    OR

	- an atomic multi-recipient final delivery mechanism.

So indeed SMTP (in the general case) does impose some constraints on
MTA queueing behaviour.

--

-- 
	Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majordomo <at> postfix.org?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.

Gábor Lénárt | 3 Jul 14:47

Picon

Re: MTA or SMTP proxy?

On Fri, Jul 03, 2009 at 02:30:10PM +0200, Jozsef Kadlecsik wrote:
> > > This is all very standard behaviour for an MTA. Recipient checking is
> > > a very common task for most Postfix users. Regardless of what you
> > > expect or want, SMTP is a store-and-forward protocol - queues are a
> > > strong component for an MTA.
> > 
> > I don't think so, SMTP itself is just a protocol, you can use it without
> > having and storage too. It's another question that how useful it is then ...
> 
> SMTP is a store-and-forward protocol, it does require queues. You cannot 
> simply wave aside this requirement.

I think, protocol itself is just the communication between MTAs (it does not
store anything itself, MTAs which uses the SMTP can/must store things while
using SMTP to communicate with other MTAs).As proxy level firewalls has got
proxies (like Zorp) for SMTP without having any queue.  I would like to do
this, and I have the suspect I am a bit off-topic now, because it's not so
postfix related question then.

> I think it cannot be solved with an SMTP proxy "without queues" either: 
> the final destination must queue the message first. After that it can 
> attempt a local delivery, which may fail, for example due to the user 
> being over quota. Therefore at the proxy side you also have to maintain 
> the mail queue.

For example PowerMail do this. It hasn't got queue by design. It's a
"mailstore" MTA, it can only accept mail. It simply does store mail to the
final destination (user maildir for example) instead of having a separated
queue file - which will be read then again to store into the user's mailbox
or whatever even on the same system - it does the job directly. Of course
PowerMail is not a general purpose MTA like Postfix, its usability is
limitid to implement the "final destination" of mails on the "mail store"
systems. For me it does not make sense that an MTA can accept mail which
won't be able to store that mail then, it's better to reject then.
Especially because generating NDRs needs more work, using large queues
requires more resources, and also a generated NDR is better to be generated
as near to the sender as possible: I simply does not accept mail if it would
be over quota, so the sender's MTA will generate NDR if needed. It's better
for the user too: maybe the sender's MTA uses the language of the sender
user, or they are more comfortable with their own "style" and so on.

--

-- 
- Gábor

Barney Desmond | 3 Jul 14:58

Picon

Re: MTA or SMTP proxy?

2009/7/3 Gábor Lénárt <lgb <at> lgb.hu>:
*

>> SMTP is a store-and-forward protocol, it does require queues. You cannot
>> simply wave aside this requirement.
>
> I think, protocol itself is just the communication between MTAs (it does not
> store anything itself, MTAs which uses the SMTP can/must store things while
> using SMTP to communicate with other MTAs).As proxy level firewalls has got

Okay, this is getting to semantics. SMTP is your transport for pushing
messages between entities that can store messages. It sounds like you
want some sort of SMTP protocol-level filter. Postfix can't do this in
a comprehensive way such as you describe, but it achieves a lot of
basic, related, important functionality (eg. recipient verification,
because Postfix often performs the mailstore function as well.

At this stage I need to ask: what is your question here? I haven't
tried it, but you may be interested in ASSP, I believe it fits the
definition of an SMTP proxy:
http://assp.sourceforge.net/

Gábor Lénárt | 3 Jul 15:14

Picon

Re: MTA or SMTP proxy?

On Fri, Jul 03, 2009 at 10:58:34PM +1000, Barney Desmond wrote:
> > I think, protocol itself is just the communication between MTAs (it does not
> > store anything itself, MTAs which uses the SMTP can/must store things while
> > using SMTP to communicate with other MTAs).As proxy level firewalls has got
> 
> Okay, this is getting to semantics. SMTP is your transport for pushing
> messages between entities that can store messages. It sounds like you
> want some sort of SMTP protocol-level filter. Postfix can't do this in

Yes, yes, that's what I said, and I used the $subject just because to make
my question clear, I was quite sure that postfix is an MTA and not an SMTP
proxy :)  [or filter, or how we would like it to be called] If it went to be
offtopic too much, I'm sorry.

> a comprehensive way such as you describe, but it achieves a lot of
> basic, related, important functionality (eg. recipient verification,
> because Postfix often performs the mailstore function as well.
> 
> At this stage I need to ask: what is your question here? I haven't

First I had the hope that there is some chance to make postfix work like
this, but yes, it's an MTA, not a filter/proxy. Just I was confused with
some "before queue" things I had heared before, so I thought it's possible
somehow to "rape" it to work this way even with some external patches or who
knows ... And I was curious about the opinions of people here in general.

> tried it, but you may be interested in ASSP, I believe it fits the
> definition of an SMTP proxy:
> http://assp.sourceforge.net/

Hmm, thanks a lot, I will check it out!

--

-- 
- Gábor

Victor Duchovni | 3 Jul 15:25

Re: MTA or SMTP proxy?

*

On Fri, Jul 03, 2009 at 03:14:00PM +0200, G?bor L?n?rt wrote:

> First I had the hope that there is some chance to make postfix work like
> this, but yes, it's an MTA, not a filter/proxy. Just I was confused with
> some "before queue" things I had heared before, so I thought it's possible
> somehow to "rape" it to work this way even with some external patches or who
> knows ... And I was curious about the opinions of people here in general.
> 
> > tried it, but you may be interested in ASSP, I believe it fits the
> > definition of an SMTP proxy:
> > http://assp.sourceforge.net/
> 
> Hmm, thanks a lot, I will check it out!

Feel free, but my advice is that your requirements are misguided.

Postfix can be a non-queueing proxy in front of another MTA by the
way, if you make sure that the ESMTP feature-set of the back-end
MTA are is the same as that advertised by Postfix.

I've done this to add TLS support for non-TLS capabable MTAs without
introducing an intermediate queue.

    http://www.postfix.org/postconf.5.html#smtpd_proxy_filter

Still not a good idea in general.

--

-- 
	Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majordomo <at> postfix.org?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.

Wietse Venema | 3 Jul 14:57

Re: MTA or SMTP proxy?

G�bor L�n�rt:
*

> On Fri, Jul 03, 2009 at 02:30:10PM +0200, Jozsef Kadlecsik wrote:
> > > > This is all very standard behaviour for an MTA. Recipient checking is
> > > > a very common task for most Postfix users. Regardless of what you
> > > > expect or want, SMTP is a store-and-forward protocol - queues are a
> > > > strong component for an MTA.
> > > 
> > > I don't think so, SMTP itself is just a protocol, you can use it without
> > > having and storage too. It's another question that how useful it is then ...
> > 
> > SMTP is a store-and-forward protocol, it does require queues. You cannot 
> > simply wave aside this requirement.

Reminds me of DECNET. When sending mail, the mail command would
not accept the message while the destination was down.

	Wietse

Wietse Venema | 3 Jul 13:49

Re: MTA or SMTP proxy?

G�bor L�n�rt:
> Hei,
> 
> Recently I am thinking of reimplementing our MX servers. Of course rcpt
> check is a must, also I should not generate NDRs later, I should only accept
> mails from outside which won't cause bounce on the mail store MTA later. My
> problem: there are some conditions cannot be forecasted easily, I can't be
> sure that mail can be delivered (over quota, user is deleted between the
> moments of mail at MX and the mail storage etc). Some people would say, that
> this issue is minor, and quite rare so I should not care about this. However

Perhaps you can use an access map (plain file or perhaps *SQL) with
"reject" actions for over-quote users. These tables don't need to
be updated yby the minute.

/etc/postfix/main.cf:
    smtpd_recipient_restrictions = 
	... hash:/etc/postfix/recipient_access ...

/etc/postfix/recipient_access:
    user1 <at> example.com 	450 4.2.2 User mailbox over quota
    user2 <at> example.com 	450 4.2.2 User mailbox over quota

It could also done with "reject_unverified_recipient" provided that
the mailbox delivery system reports over-quote conditions.

> Also the over quota problem is solved. Also, I hate when there is some rare
> problem with the backend mail servers (mail stores) and we have huge queues
> filled up with waiting mails, I would say, that MX servers shouldn't accept
> anything if they are not 100% sure that they can transfer the mail to the

This could be done with a wild-card
access action:

/etc/postfix/main.cf:
    smtpd_recipient_restrictions = 
	regexp:/etc/postfix/down.regexp

/etc/postfix/down.regexp:
    /./	450 4.3.2 System not accepting network messages

It could also done with "reject_unverified_recipient" if the bottle
neck is directly down-stream. reject_unverified_recipient does not
work across multiple hops.

	Wietse

Return

Return to gmane.mail.postfix.user.

Advertisement

Project Web Page

Postfix MTA user discussion

Search Archive

Language

Change language

Options

Current view: All messages / Showing whole messages / Hiding cited text.
Change to Threads only, shortened messages, or show cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane