Jeremy Chadwick | 2 Jun 21:33

Newbie-esque SPF deployment questions

Greetings.

I've deployed SPF DNS records for our domains, but one of our users
today informed me of something somewhat bizarre which I still can't
seem to make sense of.

Basically, our outgoing setup (mail from user <at> parodius.com to
someplace <at> somedomain.com) is as follows:

Client MUA --> mx1.parodius.com:587 --> mx1.parodius.com:* --> someplace <at> somedomain.com
                 {sendmail}               {sendmail}

And for incoming mail (mail destined to user <at> parodius.com):

Internet host --> mx1.parodius.com:25 --> procmail (MTA) --> SpamAssassin + SPF --> Local mailbox
                    {sendmail}

Important to note: sendmail is bound to mx1.parodius.com (64.62.145.229),
which is an IP alias on the same physical machine as the A record you
see for the actual domain itself (parodius.com).  All SMTP traffic
(incoming and outgoing) is done over 64.62.145.229; and yes, I am 100%
sure of this.  I just wanted to make that crystal clear.  :-)

The SPF records I deployed were the following:

parodius.com.		IN A    64.62.145.226
parodius.com.		IN MX 10 mx1.parodius.com.
parodius.com.		IN TXT  "v=spf1 mx ~all"

mx1.parodius.com.	IN A    64.62.145.229
mx1.parodius.com.	IN TXT  "v=spf1 a -all"

Users of our service must use our mail server (mx1.parodius.com) to
send mail from their user <at> parodius.com addresses.  I chose ~all for
our domain since there are probably a few stragglers who still use
their own local ISPs mail servers to send their mail, so I wanted
something in-between lenient and strict.

So, question 1: are these SPF records correct for what we want?  I
realise this is quite an ignorant question for a UNIX administrator
to ask, but the documentation of SPF is -- despite what others may
claim -- quite rhetorical and confusing.

Question 2: Do I really need the SPF record for mx1.parodius.com?

Now onto the actual problem one of our users found today:

The user, who's located in Canada, mails numerous friends of his (similar
to a mailing list) by placing their addresses in the Bcc: field.  He uses
username <at> parodius.com as his From: -- and also in the To: field (yes,
he gets a copy of the mail himself).  Bcc is used so that the users
don't know of other peoples' Email addresses, in the case that one of
them gets a virus/zombie and starts spamming all the addresses it can
find, yadda yadda... you know the routine.

However, since doing this, his own Emails are getting marked with a +0.5
in SpamAssassin score due to SPF lookups claiming SOFTFAIL:


>> pts rule name description >> ---- ---------------------- -------------------------------------------------- >> 0.5 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) >> [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com]
This confuses me greatly, as pentarou.parodius.com == 64.62.145.226, which has nothing to do with our SMTP setup. Can someone shed some light on what all is going on here, why this is breaking, and what can be done to fix it properly? Thanks. :-) -- -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. |
Steve Yates | 3 Jun 06:38
Favicon

Re: Newbie-esque SPF deployment questions

On Thu, 2 Jun 2005 12:33:06 -0700
Jeremy Chadwick <spf <at> jdc.parodius.com> wrote:


> >> 0.5 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) > >> [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com] > > This confuses me greatly, as pentarou.parodius.com == 64.62.145.226, > which has nothing to do with our SMTP setup.
Did you go to that URL? "pentarou.parodius.com rejected a message claiming to be from username <at> parodius.com. pentarou.parodius.com saw a message coming from the IP address 65.95.32.147 which is Toronto-HSE-ppp3714352.sympatico.ca; the sender claimed to be username <at> parodius.com" It looks like the sending server attempted delivery to your .226 address. - Steve Yates - ITS, Inc. - When someone asks you, "A penny for your thoughts," and you put your two cents in, what happens to the other penny? ~ Taglines by Taglinator 4 - www.srtware.com ~
Jeremy Chadwick | 3 Jun 09:55

Re: Newbie-esque SPF deployment questions

On Thu, Jun 02, 2005 at 11:38:54PM -0500, Steve Yates wrote:
> > >> 0.5  SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
> > >>                             [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com]
> > 
> > This confuses me greatly, as pentarou.parodius.com == 64.62.145.226,
> > which has nothing to do with our SMTP setup.
> 
> 	Did you go to that URL?

Yes, and the content made no sense; pentarou.parodius.com has nothing
to do with our SMTP setup.

> "pentarou.parodius.com rejected a message claiming to be from username <at> parodius.com.
> 
> pentarou.parodius.com saw a message coming from the IP address 65.95.32.147 which is
Toronto-HSE-ppp3714352.sympatico.ca; the sender claimed to be username <at> parodius.com"
> 
> 	It looks like the sending server attempted delivery to your .226
> address.

Which is incorrect, since sendmail isn't bound to .226:

bash-2.03$ telnet pentarou.parodius.com smtp
Trying 64.62.145.226...
^C
bash-2.03$ telnet pentarou.parodius.com 587
Trying 64.62.145.226...
^C
bash-2.03$ telnet mx1.parodius.com smtp
Trying 64.62.145.229...
Connected to mx1.parodius.com.
Escape character is '^]'.
220 mx1.parodius.com ESMTP Sendmail 8.13.3/8.13.3; Fri, 3 Jun 2005 00:53:28 -0700 (PDT)
{...}
bash-2.03$ telnet mx1.parodius.com 587
Trying 64.62.145.229...
Connected to mx1.parodius.com.
Escape character is '^]'.
220 mx1.parodius.com ESMTP Sendmail 8.13.3/8.13.3; Fri, 3 Jun 2005 00:53:31 -0700 (PDT)
{...}

Instead, I think SpamAssassin is using gethostname() blindly, since the
actual machine name (i.e. result from `hostname`) is pentarou.parodius.com.

I'm going to try binding spamd+spamc to mx1.parodius.com (and apply
appropriate IP filters) to see if that solves the problem.  Otherwise,
this looks to be a fairly large oversight when it comes to machines which
use IP aliases for individual services...

--

-- 
| Jeremy Chadwick                                 jdc at parodius.com |
| Parodius Networking                        http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, USA |
| Making life hard for others since 1977.                             |

Jeremy Chadwick | 3 Jun 10:20

Re: Newbie-esque SPF deployment questions


> I'm going to try binding spamd+spamc to mx1.parodius.com (and apply > appropriate IP filters) to see if that solves the problem. Otherwise, > this looks to be a fairly large oversight when it comes to machines which > use IP aliases for individual services...
This made no difference. -- -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. |
Steve Yates | 3 Jun 06:35
Favicon

Re: Newbie-esque SPF deployment questions

On Thu, 2 Jun 2005 12:33:06 -0700
Jeremy Chadwick <spf <at> jdc.parodius.com> wrote:


> Question 2: Do I really need the SPF record for mx1.parodius.com?
This SPF record (as defined) tells the world that mail arriving addressed "from user <at> mx1.parodius.com" is forged. It's not relevant to "user <at> parodius.com" mail. - Steve Yates - ITS, Inc. - The number? Just dial randomly...you'll get us eventually. ~ Taglines by Taglinator 4 - www.srtware.com ~
Jeremy Chadwick | 3 Jun 09:39

Re: Newbie-esque SPF deployment questions


On Thu, Jun 02, 2005 at 11:35:28PM -0500, Steve Yates wrote: > > Question 2: Do I really need the SPF record for mx1.parodius.com? > > This SPF record (as defined) tells the world that mail arriving > addressed "from user <at> mx1.parodius.com" is forged. It's not relevant to > "user <at> parodius.com" mail.
Thanks Steve, this clears up a lot. I'll be removing that SPF record in a moment... -- -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. |

Re: Newbie-esque SPF deployment questions

On Thu, Jun 02, 2005 at 12:33:06PM -0700, Jeremy Chadwick wrote:

> And for incoming mail (mail destined to user <at> parodius.com):
> 
> Internet host --> mx1.parodius.com:25 --> procmail (MTA) --> SpamAssassin + SPF --> Local mailbox
>                     {sendmail}
> 
> Important to note: sendmail is bound to mx1.parodius.com (64.62.145.229),
> which is an IP alias on the same physical machine as the A record you
> see for the actual domain itself (parodius.com).  All SMTP traffic
> (incoming and outgoing) is done over 64.62.145.229; and yes, I am 100%
> sure of this.  I just wanted to make that crystal clear.  :-)

That does include SpamAssassin+SPF ?  You're sure that part
isn't handled by pentarou.parodius.com ?

> However, since doing this, his own Emails are getting marked with a +0.5
> in SpamAssassin score due to SPF lookups claiming SOFTFAIL:
> 
> >> pts  rule name              description
> >> ---- ---------------------- --------------------------------------------------
> >> 0.5  SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
> >>                             [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com]

According to _your_ SpamAssassin, the host delivering this message
you _your_ hosts is 65.95.32.147

If the user is submitting the message, don't run it through SPF (or
make sure he's allowed, whitelisted, whatever).

If the SpamAssassin is called on the receiving end, the user is
delivering the message directly from home in stead of via your
infrastructure.

user -> your infra (incl. SA+SPF) -> remote user (incl. himself)
user -> your infra (no filter) -> remote user (himself, via SA+SPF)

Maybe you can find out which of these two possibilities is the case.

HTH
Alex

Jeremy Chadwick | 3 Jun 05:23

Re: Newbie-esque SPF deployment questions

On Fri, Jun 03, 2005 at 01:38:53AM +0200, Alex van den Bogaerdt wrote:
> > Important to note: sendmail is bound to mx1.parodius.com (64.62.145.229),
> > which is an IP alias on the same physical machine as the A record you
> > see for the actual domain itself (parodius.com).  All SMTP traffic
> > (incoming and outgoing) is done over 64.62.145.229; and yes, I am 100%
> > sure of this.  I just wanted to make that crystal clear.  :-)
> 
> That does include SpamAssassin+SPF ?  You're sure that part
> isn't handled by pentarou.parodius.com ?

Well, technically it's all the same physical machine.  But yes, I
suppose that statement would be true.  SA (with SPF support) is run
under procmail using using spamc + spamd.  spamd is bound to 127.0.0.1,
and spamc connects to 127.0.0.1 (when procmail is spawned).  All of
these probably just blindly use gethostname(3) to look up the local
machine name...

Would binding spamd to mx1.parodius.com (and making spamc connect to
mx1.parodius.com) fix this situation?  This is more of a SpamAssassin
question, I suppose.  The problem I have with binding spamd to
mx1.parodius.com is that that's a public network interface, which means
more IP filters (for port 783).

> > However, since doing this, his own Emails are getting marked with a +0.5
> > in SpamAssassin score due to SPF lookups claiming SOFTFAIL:
> > 
> > >> pts  rule name              description
> > >> ---- ---------------------- --------------------------------------------------
> > >> 0.5  SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
> > >>                             [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com]
> 
> According to _your_ SpamAssassin, the host delivering this message
> you _your_ hosts is 65.95.32.147
> 
> If the user is submitting the message, don't run it through SPF (or
> make sure he's allowed, whitelisted, whatever).
> 
> If the SpamAssassin is called on the receiving end, the user is
> delivering the message directly from home in stead of via your
> infrastructure.
> 
> user -> your infra (incl. SA+SPF) -> remote user (incl. himself)
> user -> your infra (no filter) -> remote user (himself, via SA+SPF)
> 
> Maybe you can find out which of these two possibilities is the case.

Hmm, I'm slightly confused by your later comments, but here's how the
pathing works for what I call "outgoing" mail:

1. End-user (65.95.32.147) sends mail using mail client.  Client is
   configured to use mx1.parodius.com:587 as their SMTP server.
   Client is sending mail to foobar <at> someplace.com.
2. mx1.parodius.com:587 receives connection from 65.95.32.147.  This
   IP is in the sendmail access_db list, and is permitted to do
   relaying (i.e. send mail through us).
3. mx1.parodius.com accepts the message.
4. mx1.parodius.com does standard lookups (MX record, A record) on
   someplace.com, and attempts to deliver the message to
   foobar <at> someplace.com.

For "incoming" mail, the pathing works as follows:

1. Some SMTP server on the Internet (1.2.3.4) connects to
   mx1.parodius.com:25.  Mail is destined for username <at> parodius.com.
2. 1.2.3.4 isn't blocked by SBL+XBL, or listed as a spammer in our
   access_db list, so the mail is accepted.
3. sendmail uses procmail as the LDA (local delivery agent).
4. procmail is spawned, and reads /etc/procmailrc, which requests
   that the entire mail message by piped through spamc.
5. spamc connects to spamd (running on 127.0.0.1:783).
6. spamd does numerous spam checks (Bayes learning, rules matching,
   and SPF lookups).  spamd is configured with trusted_networks
   64.62.145.224/27 (our network block) and internal_networks
   64.62.145.229 (which is mx1.parodius.com).
7. spamd responds to spamc with the appropriate data, i.e. is the
   mail spam, is it not spam, etc., does the appropriate changes
   (modification to Subject: line to contain [spam], SpamAssassin
   X-* headers, etc.)
8. spamc returns the modified content of the message to procmail
   via the pipe in Step #4.
9. procmail delivers the mail to the mailbox /var/mail/username.

I'm still confused as to why SPF is claiming SOFTFAIL for the scenario
of when the user is sending mail to himself.

Is the problem caused by the fact that SA is doing the SPF lookup "too
late" in the mail delivery stage, via spamc+spamd across a procmail
pipe, rather than, for example, using a sendmail SPF milter?

Would adding an additional "a:" entry (for pentarou.parodius.com) to
our SPF record be the Right Thing(tm) to do?

Thanks so much, so far everyone here has been very helpful.  :-)

--

-- 
| Jeremy Chadwick                                 jdc at parodius.com |
| Parodius Networking                        http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, USA |
| Making life hard for others since 1977.                             |

Re: Newbie-esque SPF deployment questions

On Thu, Jun 02, 2005 at 08:23:10PM -0700, Jeremy Chadwick wrote:

> > If the user is submitting the message, don't run it through SPF (or
> > make sure he's allowed, whitelisted, whatever).
> > 
> > If the SpamAssassin is called on the receiving end, the user is
> > delivering the message directly from home in stead of via your
> > infrastructure.
> > 
> > user -> your infra (incl. SA+SPF) -> remote user (incl. himself)
> > user -> your infra (no filter) -> remote user (himself, via SA+SPF)
> > 
> > Maybe you can find out which of these two possibilities is the case.
> 
> Hmm, I'm slightly confused by your later comments, but here's how the
> pathing works for what I call "outgoing" mail:
> 
> 1. End-user (65.95.32.147) sends mail using mail client.  Client is
>    configured to use mx1.parodius.com:587 as their SMTP server.
>    Client is sending mail to foobar <at> someplace.com.
> 2. mx1.parodius.com:587 receives connection from 65.95.32.147.  This
>    IP is in the sendmail access_db list, and is permitted to do
>    relaying (i.e. send mail through us).
> 3. mx1.parodius.com accepts the message.
> 4. mx1.parodius.com does standard lookups (MX record, A record) on
>    someplace.com, and attempts to deliver the message to
>    foobar <at> someplace.com.

But before doing this, is it perhaps using SA+SPF ?  This would
be wrong, as the end user at 65.95.32.147 is not allowed to use
the domain in mail from.

At this point, you cannot use SPF in your setup yet it looks as
if you may be doing it.  This would explain the score.

> I'm still confused as to why SPF is claiming SOFTFAIL for the scenario
> of when the user is sending mail to himself.

Because the mail is coming from an unauthorized host.

Don't challenge the statement this SPF checker made.  Consider
it to be true and look for the mistake either in your infrastructure
or in your line of reasoning.

Remember: computers don't do what you want. They do what you ask them
to do which may be something different from what you want.

Alex

Scott Kitterman | 3 Jun 11:57

Re: Newbie-esque SPF deployment questions

...... Original Message .......
On Fri, 3 Jun 2005 00:39:23 -0700 Jeremy Chadwick <spf <at> jdc.parodius.com> 
wrote:

>On Thu, Jun 02, 2005 at 11:35:28PM -0500, Steve Yates wrote: >> > Question 2: Do I really need the SPF record for mx1.parodius.com? >> >> This SPF record (as defined) tells the world that mail arriving >> addressed "from user <at> mx1.parodius.com" is forged. It's not relevant to >> "user <at> parodius.com" mail. > >Thanks Steve, this clears up a lot. I'll be removing that SPF record >in a moment... > >--
Don't do that. You want that record for SPF's HELO check. Scott Kitterman
Picon

Re: Newbie-esque SPF deployment questions

Jeremy Chadwick schrieb:

Some remodle the entries:
-------
 > parodius.com.		IN TXT  "v=spf1 mx ~all"
 > parodius.com.		IN MX 10 mx1.parodius.com.
 > mx1.parodius.com.	IN A    64.62.145.229
------
these entries belong logocal together

SPF -> MX
MX -> mx1.parodius.com.
mx1.parodius.com. ->   64.62.145.229
Only Mails _from_ 64.62.145.229 are good.

--------
 > mx1.parodius.com.	IN TXT  "v=spf1 a -all"
 > mx1.parodius.com.	IN A    64.62.145.229
--------
these entries belongs logical together

Mails from 64.62.145.229 are good when "@mx1.parodius.com" (Old Mail-Adresses before the time of the MX-entries)
In the RFCs a Mailhost must rechable also per postmaster <at> mx1.parodius.com. and postmaster@[64.62.145.229].
I Think therefore is the SPF-entry for mx1.
and for mails from <> (in DSNs) which is postmaster <at> mx1.parodius.com

-----
 > parodius.com.		IN A    64.62.145.226
------
Is not nessesary for Mailing, it has nothing to do with SPF

 >>>pts  rule name              description
 >>>---- ---------------------- --------------------------------------------------
 >>>0.5  SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 >>>                            [SPF failed: Please see http://spf.pobox.com/why.html?sender=username%40parodius.com&ip=65.95.32.147&receiver=pentarou.parodius.com]

Says:
the mail comes from  65.95.32.147
65.95.32.147 = Toronto-HSE-ppp3714352.sympatico.ca
which is not 64.62.145.229.
that makes it softfail

 > This confuses me greatly, as pentarou.parodius.com == 64.62.145.226,
 > which has nothing to do with our SMTP setup.

You have also:
-------
pentarou.parodius.com MX (Mail Exchanger) Priority: 10 mx1.parodius.com
-------
this mean you can have mailadresses with user <at> pentarou.parodius.com
have the user
user <at> parodius.com
send it to user <at> pentarou.parodius.com
and spamassin take the letters after the @ to make the message?

best reguards
Franz Gstaettner


Gmane