gmane.mail.spam.spf.help

John Kirkwood | 18 Jul 17:17

SPF and Google Groups (sending on behalf of)

Dear all at SPF-help,

Conundrum:

user <at> un.org posts a message to the email list server
geneva-web-group <at> googlegroups.
Google Groups then sends a group email, marked From: user <at> un.org, but sent
using a Google mailserver.
The SPF record at un.org does not designate Google as a permitted sender.
My ISP blocks the email (dotster.com / mail3.dotsterhost.com - quite strict
on RFC and SPF imperfections, for example will <fail> on an invalid SPF
record).

 Received-SPF: pass (googlegroups.com designates 209.85.146.244 as a trusted
SMTP server)
 Received-SPF: fail (un.org does not designate 209.85.146.244 as a permitted
sender)

Any ideas? (Full headers of sent mail below - with sender's name changed -
email retrieved from a Death2Spam mail relay server).

Best regards,
John Kirkwood

>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Date: Fri, 18 Jul 2008 02:49:21 -0700 (PDT)
From: User <user <at> un.org>
To: Geneva Web Group <geneva-web-group <at> googlegroups.com>
Subject: Web development job - paid this time...

(Continue reading)

headers

Steve Yates | 18 Jul 17:40

RE: SPF and Google Groups (sending on behalf of)

John Kirkwood wrote on 7/18/2008 10:17:35 AM:

> user <at> un.org posts a message to the email list server
> geneva-web-group <at> googlegroups. Google Groups then sends a group email,
> marked From: user <at> un.org, but sent using a Google mailserver. The SPF
> record at un.org does not designate Google as a permitted sender. My
ISP
> blocks the email (dotster.com / mail3.dotsterhost.com - quite strict
on
> RFC and SPF imperfections, for example will <fail> on an invalid SPF
> record).
> 
>  Received-SPF: pass (googlegroups.com designates 209.85.146.244 as a
> trusted
> SMTP server)
>  Received-SPF: fail (un.org does not designate 209.85.146.244 as a
permitted
> sender)
> 
> Any ideas? (Full headers of sent mail below - with sender's name
changed -
> email retrieved from a Death2Spam mail relay server).

	My first thought is to ask why the recipient is apparently
testing the From header?  SPF doesn't protect that.  Sender ID tries to
protect that.  Looks to me like Google has it right and the ISP is
wrong.

http://www.openspf.org/FAQ/Envelope_from_scope

(Continue reading)

headers

Frank Ellermann | 18 Jul 18:14

Re: SPF and Google Groups (sending on behalf of)

John Kirkwood wrote:

> Google Groups then sends a group email, marked 
> From: user <at> un.org, but sent using a Google mailserver.

Based on your header shown below this is an 2822-From,
the ordinary From header field.  SPF does not operate
on the mail header, it uses the mail envelope.

IOW there's no problem, in theory...  Back to reality:

> The SPF record at un.org does not designate Google
> as a permitted sender.

Yes, that's as it should be...

> My ISP blocks the email

...that's also as it should be IFF there is really an
SPF FAIL.  For that your ISP should look at the HELO
and the MAIL FROM (not the 2822-From mentioned above),
based on what you found that is:

| Received-SPF: pass (googlegroups.com designates
|  209.85.146.244 as a trusted SMTP server)

That's an SPF PASS for the HELO wa-out-0708.google.com
(you see that HELO name in the Received header field).

| Received-SPF: fail (un.org does not designate

(Continue reading)

headers

Alex van den Bogaerdt | 18 Jul 18:37

Re: SPF and Google Groups (sending on behalf of)

On Fri, Jul 18, 2008 at 05:17:35PM +0200, John Kirkwood wrote:

> user <at> un.org posts a message to the email list server
> geneva-web-group <at> googlegroups.
> Google Groups then sends a group email, marked From: user <at> un.org, but sent
> using a Google mailserver.

and, important, using "Sender: geneva-web-group <at> googlegroups.com".

> The SPF record at un.org does not designate Google as a permitted sender.

no problem. The sender is googlegroups.com

> My ISP blocks the email (dotster.com / mail3.dotsterhost.com - quite strict
> on RFC and SPF imperfections, for example will <fail> on an invalid SPF
> record).
> 
>  Received-SPF: pass (googlegroups.com designates 209.85.146.244 as a trusted
> SMTP server)
>  Received-SPF: fail (un.org does not designate 209.85.146.244 as a permitted
> sender)
> 
> Any ideas? (Full headers of sent mail below - with sender's name changed -
> email retrieved from a Death2Spam mail relay server).

Real true SPF will only look at 'mail from' in the SMTP transaction. This is
visible as the return path in the message's headers.

SPF-by-microsoft abuses SPF records and pretend they're SenderID records.
Instead of rejecting in the SMTP session before any data is sent, it will

(Continue reading)

headers

Frank Ellermann | 18 Jul 22:58

Re: SPF and Google Groups (sending on behalf of)

Alex van den Bogaerdt wrote:

>>  X-Sender: user <at> un.org

> If microsoft's protocol states that X-Sender is more
> important than Sender, then your ISP does the right thing.

LOL, no, Sender ID PRA does not use X-Sender.  And it also
wouldn't check the HELO, as shown in this example.  I think
it was a plain broken SPF check based on the 2822-From, not
on the "X-Sender".

Guessing:  Google wants to protect the mail from Sender ID
PRA confusions, therefore it adds its own Sender.  There
was already an old Sender, therefore Google renamed this 
to X-Sender.  Because the Sender ID PRA "technique" using
a Resent-From in such cases is FUBAR.  

Everybody and everything did exactly the right thing, but
the receiver checked 2822-From instead of MAIL FROM, and
that is obviously wrong.

 Frank

headers

Alex van den Bogaerdt | 19 Jul 00:02

Re: Re: SPF and Google Groups (sending on behalf of)

On Fri, Jul 18, 2008 at 10:58:45PM +0200, Frank Ellermann wrote:

> LOL, no, Sender ID PRA does not use X-Sender.  And it also
> wouldn't check the HELO, as shown in this example.  I think
> it was a plain broken SPF check based on the 2822-From, not
> on the "X-Sender".

Whatever.

Point is: SPF checks (2)821, not (2)822.

We did not cause this confusion. If implementations get this wrong,
go ask at the place causing the confusion.

That this is very hard to do, with high probability of falling
on deaf ears, does still not mean the discussion belongs here.

Cheers,
Alex

headers

Frank Ellermann | 19 Jul 18:36

Re: SPF and Google Groups (sending on behalf of)

Alex van den Bogaerdt wrote:

> If implementations get this wrong, go ask at the place
> causing the confusion.

That was the original question, is it an implementation
getting it wrong, or something else.  So far we know

* it's no problem at the old sender (un.org)
* it's no problem at the new sender (googlegroups)
* it's no problem with SPF HELO checks
* it's no problem with SPF MAIL FROM checks
* it's no problem with Sender ID confusions
* it's no problem with rejecting FAIL 
* it is a bogus FAIL checking the wrong address

What we don't know is if this a broken implementation,
as in "some code got it wrong", or a broken setup, as
in "code is fine, but operator arranged for the wrong
address to be checked".  

The latter case could turn out to be "code is dubious,
as it supports to do something stupid"...

> does still not mean the discussion belongs here.

IBTD because I'd like to know if this is dubious code.
If we know this we could put a warning on the SPF site.

SPF users should never have to worry about such issues,

(Continue reading)

headers

Alex van den Bogaerdt | 20 Jul 15:12

Re: Re: SPF and Google Groups (sending on behalf of)

On Sat, Jul 19, 2008 at 06:36:41PM +0200, Frank Ellermann wrote:

> > does still not mean the discussion belongs here.
> 
> IBTD because I'd like to know if this is dubious code.
> If we know this we could put a warning on the SPF site.
> 
> SPF users should never have to worry about such issues,
> and if an implementor confused MAIL FROM with 2822-From
> (s)he belongs into a "hall of shame" on the main page -
> hopefully convincing other implementors that this isn't
> how they want a link to their code.

I think you know very well what I was saying, but for the off chance
that you didn't: there's one specific entity which takes our carefully
crafted SPF records and then {ab|re}uses them for their own incompatible
protocol: SenderID.

If implementors get it wrong when parsing the various headers with all
their if-then-else decisions, that's indirectly the fault of this other
protocol, not ours.  Why should we provide support?

I strongly believe that anything looking at message headers (perhaps
with the exception of return-path) is SenderID and that questions on
this should be redirected to the appropriate place.

I urge all who do actually implement SenderID to mention SenderID in
their error messages/bounces, not SPF.

Right now I feel like we are an unpaid helpdesk for MS, something I

(Continue reading)

headers

Frank Ellermann | 20 Jul 16:51

Re: Re: SPF and Google Groups (sending on behalf of)

Alex van den Bogaerdt wrote on the help list:

> there's one specific entity which takes our carefully
> crafted SPF records and then {ab|re}uses them for their
> own incompatible protocol: SenderID.

From time to time a fresh SenderID bashing is good, and
the three or four folks answering that question on the
help list all checked if it *could* be realted to this
issue.  

But it clearly was not, and your idea that SenderID PRA
might do strange things with an X-Sender: header field
was, hm, strange.  IMO users confronted with wrong SPF
results are entitled to ask questions on the help list,
and sending them away claiming that it's no SPF problem
if SenderID does strange things can only make sense if
their problem clearly is an effect of SenderID.  

But this wasn't the case.  Even if it turns out that the
implementation did the less plausible thing, and checked
X-Sender instead of 2822-From, there was a good PRA for
a SenderID PASS, and a good MAIL FROM for an SPF PASS.

The implementation is broken, it got FAIL, based on the
2822-From or maybe the equivalent X-Sender.

> If implementors get it wrong when parsing the various
> headers with all their if-then-else decisions, that's
> indirectly the fault of this other protocol, not ours.

(Continue reading)

headers

Alex van den Bogaerdt | 20 Jul 17:09

Re: Re: Re: SPF and Google Groups (sending on behalf of)

On Sun, Jul 20, 2008 at 04:51:12PM +0200, Frank Ellermann wrote:

> But it clearly was not

Clearly it isn't as clear as you say it is.

I repeat:

SPF does not do anything with headers, with return-path being
the one and only possible exception.

If an implementation does do anything else with headers, it is doing
so because is doesn't know any better.  And that is because there is
this other protocol, not SPF, which does look at headers.

> The user who asked the question has likely never before
> heard of SenderID.  Apart from being rude it makes no
> sense to send users to "SenderID help" (if that exists)
> if their problem is a broken SPF implementation.

1: It is not a broken SPF implementation. It is a broken SenderID
implementation.

2: There's nothing rude about saying: 'Your SPF record is just fine.
The problem has nothing to do with SPF, eventhough the error message
says so.  Please turn to the receiver or to microsoft.'  Especially
not if accompanied by some links, e.g. to http://www.openspf.org/SPF_vs_Sender_ID

Anyway, I have made my point and will not continue this conversation.

(Continue reading)

headers

Scott Kitterman | 20 Jul 17:22

Re: Re: Re: SPF and Google Groups (sending on behalf of)

On Sun, 20 Jul 2008 17:09:53 +0200 Alex van den Bogaerdt 
<alex <at> ergens.op.het.net> wrote:
>On Sun, Jul 20, 2008 at 04:51:12PM +0200, Frank Ellermann wrote:
>
>> But it clearly was not
>
>Clearly it isn't as clear as you say it is.
>
>I repeat:
>
>SPF does not do anything with headers, with return-path being
>the one and only possible exception.
>
>If an implementation does do anything else with headers, it is doing
>so because is doesn't know any better.  And that is because there is
>this other protocol, not SPF, which does look at headers.
>
>> The user who asked the question has likely never before
>> heard of SenderID.  Apart from being rude it makes no
>> sense to send users to "SenderID help" (if that exists)
>> if their problem is a broken SPF implementation.
>
>1: It is not a broken SPF implementation. It is a broken SenderID
>implementation.
>

Not everything that misuses SPF records is SenderID.  There is a Mozilla 
Thunderbird plugin that does SPF checks against From that predates the 
existance of SenderID.

(Continue reading)

headers

Alex van den Bogaerdt | 20 Jul 17:48

Re: Re: Re: SPF and Google Groups (sending on behalf of)

On Sun, Jul 20, 2008 at 11:22:29AM -0400, Scott Kitterman wrote:

> Not everything that misuses SPF records is SenderID.  There is a Mozilla 
> Thunderbird plugin that does SPF checks against From that predates the 
> existance of SenderID.

This is new to me. If such cases do occur more often, I think it should
be mentioned on the website (or is it already?)

If such plugins are common in the field, I have to adjust my conclusion.
In such a case, point to the receiver only, not to SenderId.  Most of my
rant is still valid in that case. It is _not_ SPF.

If OTOH such plugins (which do not seem to be used in this particular
case by the way) have existed in the past, were rare then and even rarer
now, they are the exception to the rule and I stand by my original thoughts.

In either case: I feel we should draw a line.  The SPF policy is good? The
sending host is authorized?  Then it is not an SPF problem.  I'm willing
to appreciate the need to point people to that well known bandaid (adding
a "Sender" header) although I think that's already dubious.

Alex

headers

Scott Kitterman | 20 Jul 18:22

Re: Re: Re: SPF and Google Groups (sending on behalf of)

On Sunday 20 July 2008 11:48, Alex van den Bogaerdt wrote:
> On Sun, Jul 20, 2008 at 11:22:29AM -0400, Scott Kitterman wrote:
> > Not everything that misuses SPF records is SenderID.  There is a Mozilla
> > Thunderbird plugin that does SPF checks against From that predates the
> > existance of SenderID.
>
> This is new to me. If such cases do occur more often, I think it should
> be mentioned on the website (or is it already?)

http://razor.occams.info/code/spf/

Already mentioned on http://www.openspf.org/Implementations

Note that his site says, "The extension uses Sender Policy Framework (SPF) (in 
a nonstandard way) ...".  It didn't mention the non-standard part before I 
discussed it with him.

> If such plugins are common in the field, I have to adjust my conclusion.
> In such a case, point to the receiver only, not to SenderId.  Most of my
> rant is still valid in that case. It is _not_ SPF.

Agreed that its' not SPF.  I've seen fewer issues related to this is recent 
years.  I don't know if it's less used, working better, or his user base 
understands its limitations better.

> If OTOH such plugins (which do not seem to be used in this particular
> case by the way) have existed in the past, were rare then and even rarer
> now, they are the exception to the rule and I stand by my original
> thoughts.
>

(Continue reading)

headers

Frank Ellermann | 20 Jul 19:00

Re: SPF and Google Groups (sending on behalf of)

Alex van den Bogaerdt wrote:

> In either case: I feel we should draw a line.  The SPF
> policy is good? The sending host is authorized?  Then
> it is not an SPF problem.

Alex, the users don't care *whose fault* it is when they
have a question, they want to know what is wrong.  That
user actually wasn't sure if something is wrong, or if
this might be a problem on the side of googlegroups.com
or the original sender.

Analyzing header fields was not his profession or hobby,
otherwise he could have answered his own question.  If
you tell him "go away, this is a SenderID PRA problem,
ask the SenderID folks why they use X-Sender" it misses
his points - besides SenderID folks would tell him that
this is bullshit.  

I can't tell if "SenderID folks" talking with ordinary
users exist, and it's perfectly understandable that you
don't like to deal with SenderID issues.  Like I don't
deal with confidential help requests using "topic: the
SPF Web site", about one per week.  I've reported that
the contact form doesn't do what the source code says,
I've tried to approve SPF questions on the Webmaster
list, and got pushback for that.  Now I mostly ignore
help requests with "topic: SPF Web site", confidential
or otherwise.  See, I also draw a line, it is just not 
exactly the same line as your line.

(Continue reading)

headers

John Kirkwood | 24 Jul 08:31

RE: Re: SPF and Google Groups (sending on behalf of)

Dear all,

Many thanks for your clarifications and input. It turns out that the
SPF implementation was fine, but some separate processing had been
added temporarily, which interfered and has now been removed.
Apologies for the static.

However, the situation I now face is that my mail is MX relayed via
death2spam.net and then on to dotster.com, and dotster.com is
reapplying SPF as if death2spam.net were the originating MTA. (I was
assured that mail relay was allowed for the dotster.com mailservers,
although I am now told that it is not and that the only solution is to
remove the relay).

Before I go hunting for another mail provider, can anyone say whether
SPF would normally have any problems with mail relay? (I am loathe to
lose my death2spam mail filtering - it does a damn good job).

Many thanks,
John

-----Original Message-----
From: Alex van den Bogaerdt [mailto:alex <at> ergens.op.het.net]
Sent: 20 July 2008 15:13
To: spf-help <at> v2.listbox.com
Subject: Re: [spf-help] Re: SPF and Google Groups (sending on behalf
of)

On Sat, Jul 19, 2008 at 06:36:41PM +0200, Frank Ellermann wrote:

(Continue reading)

headers

Frank Ellermann | 24 Jul 10:23

death2spam.net (was: SPF and Google Groups (sending on behalf of))

John Kirkwood wrote:

> my mail is MX relayed via death2spam.net and then on to 
> dotster.com, and dotster.com is reapplying SPF as if
> death2spam.net were the originating MTA.

If death2spam.net forwards mail to third parties it *IS*
the originating MTA as far as SPF checkers at these 3rd
parties are concerned.

> can anyone say whether SPF would normally have any 
> problems with mail relay?

No problems with mail relays on the side of the sender or
on the side of the receiver.  SPF is in essence for the
one critical hop where two unknown strangers (sender and
receiver) meet.

Now if you introduce a second critical hop in the form of
"forwarding to 3rd party" this will nowhere work "as is":

S -> F -> R instead of S -> R

The sender S defines S-IPs permitted to send MAIL FROM S.
The sender S has no reason to permit sending IPs of any
forwarder F or forger F, in fact S has no idea who F is.

The receiver R checks that MAIL FROM S comes from one of
the S-IPs as defined in the sender policy of S.  Without
intervention sending IPs of forwarder F will FAIL, as for

(Continue reading)

headers

Rob MacGregor | 24 Jul 11:18

Re: Re: SPF and Google Groups (sending on behalf of)

On Thu, Jul 24, 2008 at 07:31, John Kirkwood <jkirkwood <at> kclinfo.com> wrote:
>
> Before I go hunting for another mail provider, can anyone say whether
> SPF would normally have any problems with mail relay? (I am loathe to
> lose my death2spam mail filtering - it does a damn good job).

See

http://www.openspf.org/FAQ/Forwarding

In short, yes, it can break forwarding and there are solutions.

--

-- 
 Please keep list traffic on the list.

Rob MacGregor
 Whoever fights monsters should see to it that in the process he
 doesn't become a monster. Friedrich Nietzsche