A spam filtering project

email builder | 9 Feb 05:06

Picon

SPF and DKIM tests by default?

Hello,

I have a server where I never customized any of the SA
rules/tests (SA v.3.3.1).  The server does run sa-update
every day.  Is this the right place to look to know what
tests the server should be running?

https://spamassassin.apache.org/tests_3_0_x.html

From that page, it seems that SPF checks are normal
but DKIM is not. Is this right?

Contrary to that, this page suggests that DKIM test are
enabled by default in version 3.3:

https://wiki.apache.org/spamassassin/Plugin/DKIM

Also, where can I look to verify the tests/rules currently
in place on the server?  (per-user rules are not implemented)

I looked in /usr/share/spamassassin and there are a few
files with "spf" and "dkim" in their names.  Does that
mean those tests are active?

ls *spf*
-rw-r--r-- 1 root root 3100 Mar 15  2010 25_spf.cf
-rw-r--r-- 1 root root 3584 Mar 15  2010 60_whitelist_spf.cf

ls *dkim*
-rw-r--r-- 1 root root 4407 Mar 15  2010 25_dkim.cf

(Continue reading)

darxus | 9 Feb 18:27

Re: SPF and DKIM tests by default?

On 02/08, email builder wrote:
> Hello,
> 
> I have a server where I never customized any of the SA
> rules/tests (SA v.3.3.1).  The server does run sa-update
> every day.  Is this the right place to look to know what
> tests the server should be running?
> 
> https://spamassassin.apache.org/tests_3_0_x.html

At the top of that page, it says "Tests Performed: v3.0.x" which is not the
version you are running.  https://spamassassin.apache.org/tests_3_3_x.html
contains tests for 3.3.  I don't know when they get updated, maybe only
when 3.3.0 was released.  I wouldn't trust it much.

Run: sa-update -D 2>&1| grep DIR

That will output something like:

Feb  9 12:08:49.609 [20855] dbg: generic: Perl 5.010001, PREFIX=/usr,
DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin

On this system, sa-update downloads rules to /var/lib/spamassassin, so I
guess you're looking for the LOCAL_STATE_DIR.

That directory will contain a directory related to your SA version,
something like 3.003001, which will contain updates_spamassassin_org, which
will contain the files defining all the rules.  

Although that doesn't necessarily tell you which are enabled by default.

(Continue reading)

email builder | 11 Feb 03:20

Picon

Re: SPF and DKIM tests by default?

Thanks a lot for your reply

> Run: sa-update -D 2>&1| grep DIR

> 
> That will output something like:
> 
> Feb  9 12:08:49.609 [20855] dbg: generic: Perl 5.010001, PREFIX=/usr, 
> DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, 
> LOCAL_STATE_DIR=/var/lib/spamassassin
> 
> On this system, sa-update downloads rules to /var/lib/spamassassin, so I
> guess you're looking for the LOCAL_STATE_DIR.

OK, makes sense.  Mine is the same as yours.

> That directory will contain a directory related to your SA version,
> something like 3.003001, which will contain updates_spamassassin_org, which
> will contain the files defining all the rules.  

Hmm, in there I find TWO directories:

 3.002005
 3.003001

Strangely, both have dates of today, but the *contents* of 3.002005 are from Apr 3 2011.  So I guess my
system uses 3.003001 since it's files are dated currently

Wonder if I can delete the older one

(Continue reading)

Kevin A. McGrail | 12 Feb 17:28

Re: SPF and DKIM tests by default?

On 2/10/2012 9:20 PM, email builder wrote:
> Wonder if I can delete the older one 
Sure.  Worst case just run sa-update again if you delete the wrong one.

> Hm, well is there a file or somewhere to look and see what rules are 
> active? 
Do you mean something like: With my configuration, what rules might 
possibly be triggered?

That's an interesting question.  Perhaps we could use a spamassassin 
parameter to run, parse config and dump all possible rules that would 
run (with scores) based on all plugins, etc. that are believed to be 
configured.  If that is what you want, please open a bug at 
https://issues.apache.org/SpamAssassin/ assuming no one knows a way this 
can occur now.
>> I believe for SPF you *should* be doing the detecting at your MTA
>> (mail server software) and inserting a header for spamassassin to use:
>> Received-SPF.  (Because SPF is supposed to use the "envelope from",
>> which is not necessarily included in a header.)
> I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing
this in postfix? Or during amavis processing?
Interesting thought though while the envelope sender is not in a header 
per se, it is in the From line for mbox format email, I believe.  If you 
are using procmail for delivery, for example, there shouldn't be an issue.

>
> Me too. I sent emails to myself from Yahoo and Gmail and got these in my X-Spam-Status:
>
> Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU
> Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID

(Continue reading)

email builder | 13 Feb 02:20

Picon

Re: SPF and DKIM tests by default?


> On 2/10/2012 9:20 PM, email builder wrote:

>>  Wonder if I can delete the older one 
> Sure.  Worst case just run sa-update again if you delete the wrong one.

OK, thank you. I'll report back if it causes any problems but I can't imagine it would.

>>  Hm, well is there a file or somewhere to look and see what rules are 
>> active? 
> Do you mean something like: With my configuration, what rules might possibly be 
> triggered?

yes

> That's an interesting question.  Perhaps we could use a spamassassin 
> parameter to run, parse config and dump all possible rules that would run (with 
> scores) based on all plugins, etc. that are believed to be configured.  If that 
> is what you want, please open a bug at https://issues.apache.org/SpamAssassin/ 
> assuming no one knows a way this can occur now.

OK it's a feature request then huh? I added it:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6757

>>>  I believe for SPF you *should* be doing the detecting at your MTA
>>>  (mail server software) and inserting a header for spamassassin to use:
>>>  Received-SPF.  (Because SPF is supposed to use the "envelope 
>>> from",
>>>  which is not necessarily included in a header.)
>>  I see. That makes sense. Is there a wiki page suggesting solutions for

(Continue reading)

darxus | 13 Feb 00:03

Re: SPF and DKIM tests by default?

On 02/10, email builder wrote:
> > I believe for SPF you *should* be doing the detecting at your MTA
> > (mail server software) and inserting a header for spamassassin to use:
> > Received-SPF.  (Because SPF is supposed to use the "envelope from",
> > which is not necessarily included in a header.)
> 
> I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing
this in postfix? Or during amavis processing?

I use postfix-policyd-spf-perl.
Which appears to currently be officially hosted at:
https://launchpad.net/postfix-policyd-spf-perl/

--

-- 
"For gasoline vapor, the explosive range is from 1.3 to 6.0% vapor
to air...useful against soft targets such as...armored vehicles...and
bunkers." - http://www.fas.org/man/dod-101/sys/dumb/fae.htm
http://www.ChaosReigns.com

email builder | 13 Feb 02:35

Picon

Re: SPF and DKIM tests by default?

 

> On 02/10, email builder wrote:
>>  > I believe for SPF you *should* be doing the detecting at your MTA
>>  > (mail server software) and inserting a header for spamassassin to use:
>>  > Received-SPF.  (Because SPF is supposed to use the "envelope 
>> > from",
>>  > which is not necessarily included in a header.)
>> 
>>  I see. That makes sense. Is there a wiki page suggesting solutions for 
>> this? Anyone know of tips for doing this in postfix? Or during amavis 
>> processing?
> 
> I use postfix-policyd-spf-perl.
> Which appears to currently be officially hosted at:
> https://launchpad.net/postfix-policyd-spf-perl/

Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that
Received-SPF header or can use the envelope sender if it's present?

Dave Funk | 13 Feb 03:19

Picon

Re: SPF and DKIM tests by default?

On Sun, 12 Feb 2012, email builder wrote:

>> On 02/10, email builder wrote:
>>>  > I believe for SPF you *should* be doing the detecting at your MTA
>>>  > (mail server software) and inserting a header for spamassassin to use:
>>>  > Received-SPF.  (Because SPF is supposed to use the "envelope 
>>> > from",
>>>  > which is not necessarily included in a header.)
>>>
>>>  I see. That makes sense. Is there a wiki page suggesting solutions for 
>>> this? Anyone know of tips for doing this in postfix? Or during amavis 
>>> processing?
>> 
>> I use postfix-policyd-spf-perl.
>> Which appears to currently be officially hosted at:
>> https://launchpad.net/postfix-policyd-spf-perl/
>
> Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that
Received-SPF header or can use the envelope sender if it's present?

If your MTA provides sufficient info for SA to determine the envelope 
sender that is enough. I've been using sendmail+milter+sa for years
with SPF & DKIM rules and never had any kind of special MTA added 
'Received-SPF' header.

One thing that -is- a factor; sa depends upon specific perl modules
for that functionality; DNS, SPF, & DKIM modules (EG Net::DNS, Mail::DKIM, 
Mail::SPF ), and 'loadplugin' statements in the correct ".pre" files.
Occasionally issues arise with problematic versions of those modules.
For example, search this list archive for disussions about problems caused

(Continue reading)

email builder | 13 Feb 03:54

Picon

Re: SPF and DKIM tests by default?

>>>  On 02/10, email builder wrote:

>>>>   > I believe for SPF you *should* be doing the detecting at your 
>>>>   > MTA
>>>>   > (mail server software) and inserting a header for 
>>>>   > spamassassin to use:
>>>>   > Received-SPF.  (Because SPF is supposed to use the 
>>>>   > "envelope from",
>>>>   > which is not necessarily included in a header.)
>>>> 
>>>>   I see. That makes sense. Is there a wiki page suggesting solutions 
>>>>   for this? Anyone know of tips for doing this in postfix? Or during amavis 
>>>>   processing?
>>> 
>>>  I use postfix-policyd-spf-perl.
>>>  Which appears to currently be officially hosted at:
>>>  https://launchpad.net/postfix-policyd-spf-perl/
>> 
>>  Thanks for that, although see my last post - do you know if the SPF tests 
>> only know how to look for that Received-SPF header or can use the envelope 
>> sender if it's present?
> 
> If your MTA provides sufficient info for SA to determine the envelope sender 
> that is enough.

I agree and I've done some more research and found that Postfix adds the envelope sender as a "Return-Path"
header (its pipe and virtual delivery agent at least do this). So I *do* have a header in my messages with the
envelope sender. Either the SPF rules don't know how to look for "Return-Path" (which would surprise me
given that it is quasi-standard and highly used) or I have some other problem.

(Continue reading)

Kevin A. McGrail | 13 Feb 16:20

Re: SPF and DKIM tests by default?

Q: Will some rules not fire if some condition exists based on other rules?

A: Correct.  There are plenty of rules that build on other rules.  We 
call these meta rules.

Regards,
KAM

email builder | 16 Feb 01:08

Picon

Re: SPF and DKIM tests by default?


> 
> Q: Will some rules not fire if some condition exists based on other rules?
> 
> A: Correct.  There are plenty of rules that build on other rules.  We call these 
> meta rules.
> 

OK, but:

Q: Are there any default rules as supplied by sa-update that would
prevent SPF rules from firing?

Q: Any other ideas on how to learn what rules are actually being used?

Q: Any suggestions as to why SPF rules would not fire on a
Gmail message where Gmail uses SPF, my SPF plugin and rule
initiation seem to be in place, and a Return-Path header with the
envelope from address exists?  (please see my previous messages
on this thread)

Matus UHLAR - fantomas | 16 Feb 09:40

Picon

Re: SPF and DKIM tests by default?

>> Q: Will some rules not fire if some condition exists based on other rules?
>>
>> A: Correct.  There are plenty of rules that build on other rules.  We call these
>> meta rules.

On 15.02.12 16:08, email builder wrote:
>Q: Are there any default rules as supplied by sa-update that would
>prevent SPF rules from firing?

you can disable SPF or clear all scores 

>Q: Any other ideas on how to learn what rules are actually being used?

huh?

>Q: Any suggestions as to why SPF rules would not fire on a
>Gmail message where Gmail uses SPF, my SPF plugin and rule
>initiation seem to be in place, and a Return-Path header with the
>envelope from address exists?  (please see my previous messages
>on this thread)

I haven't found the headers in apache archive, maybe I didn't search 
carefully enough, but it's misconfigured trusted_networks and 
internal_networks what causes SPF to misfire...
--

-- 
Matus UHLAR - fantomas, uhlar <at> fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows found: (R)emove, (E)rase, (D)elete

(Continue reading)

email builder | 16 Feb 22:54

Picon

Re: SPF and DKIM tests by default?

> 

>>>  Q: Will some rules not fire if some condition exists based on other 
>>> rules?
>>> 
>>>  A: Correct.  There are plenty of rules that build on other rules.  We 
>>> call these
>>>  meta rules.
>> 
>> Q: Are there any default rules as supplied by sa-update that would
>> prevent SPF rules from firing?
> 
> you can disable SPF or clear all scores 

The question was *as supplied by sa-update*

>> Q: Any other ideas on how to learn what rules are actually being used?
> 
> huh?

Please read the rest of this thread.

>> Q: Any suggestions as to why SPF rules would not fire on a
>> Gmail message where Gmail uses SPF, my SPF plugin and rule
>> initiation seem to be in place, and a Return-Path header with the
>> envelope from address exists?  (please see my previous messages
>> on this thread)
> 
> I haven't found the headers in apache archive, maybe I didn't search 
> carefully enough,

(Continue reading)

Bowie Bailey | 16 Feb 23:00

Re: SPF and DKIM tests by default?

On 2/16/2012 4:54 PM, email builder wrote:
>> but it's misconfigured trusted_networks and 
>> internal_networks what causes SPF to misfire...
> Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified
Google in one of those settings or had an error in one of them. In this case, those are at their defaults of
empty, so I'm hoping there are other suggestions. Thanks again..

Letting trusted_networks empty is not generally a good idea.  In
particular, if your SA server is using a private IP, it will default to
trusting too much.  Specify your local networks in trusted_networks and
see if that helps your problem.

Leaving trusted_networks empty does not mean "trust nothing";  it means
"let SA figure out what to trust".

--

-- 
Bowie

email builder | 16 Feb 23:50

Picon

Re: SPF and DKIM tests by default?

> On 2/16/2012 4:54 PM, email builder wrote:

>>>  but it's misconfigured trusted_networks and 
>>>  internal_networks what causes SPF to misfire...
>>  Thank you sincerely for your help. I can only imagine that SPF wouldn't 
>> fire if I accidentally specified Google in one of those settings or had an error 
>> in one of them. In this case, those are at their defaults of empty, so I'm 
>> hoping there are other suggestions. Thanks again..
> 
> Letting trusted_networks empty is not generally a good idea.  In
> particular, if your SA server is using a private IP, it will default to
> trusting too much.  Specify your local networks in trusted_networks and
> see if that helps your problem.
> 
> Leaving trusted_networks empty does not mean "trust nothing";  it 
> means "let SA figure out what to trust".

Makes sense, especially if my hunch about the "relayed through one or
more trusted relays, cannot use header-based Envelope-From, skipping"
part of the debug output I just sent to this list is on track.

Is there a way to set trusted_networks on the command line of the
spamassassin command just for testing?

email builder | 17 Feb 00:18

Picon

Re: SPF and DKIM tests by default?

> 

>>  On 2/16/2012 4:54 PM, email builder wrote:
> 
>>>>   but it's misconfigured trusted_networks and 
>>>>   internal_networks what causes SPF to misfire...
>>>   Thank you sincerely for your help. I can only imagine that SPF 
>>> wouldn't 
>>>  fire if I accidentally specified Google in one of those settings or had 
>>> an error 
>>>  in one of them. In this case, those are at their defaults of empty, so 
>>> I'm 
>>>  hoping there are other suggestions. Thanks again..
>> 
>>  Letting trusted_networks empty is not generally a good idea.  In
>>  particular, if your SA server is using a private IP, it will default to
>>  trusting too much.  Specify your local networks in trusted_networks and
>>  see if that helps your problem.
>> 
>>  Leaving trusted_networks empty does not mean "trust nothing";  it 
> 
>>  means "let SA figure out what to trust".
> 
> Makes sense, especially if my hunch about the "relayed through one or
> more trusted relays, cannot use header-based Envelope-From, skipping"
> part of the debug output I just sent to this list is on track.
> 
> Is there a way to set trusted_networks on the command line of the
> spamassassin command just for testing?

(Continue reading)

Bowie Bailey | 17 Feb 17:03

Re: SPF and DKIM tests by default?

On 2/16/2012 6:18 PM, email builder wrote:
>>
>>>  Letting trusted_networks empty is not generally a good idea.  In
>>>  particular, if your SA server is using a private IP, it will default to
>>>  trusting too much.  Specify your local networks in trusted_networks and
>>>  see if that helps your problem.
>>>
>>>  Leaving trusted_networks empty does not mean "trust nothing";  it 
>>>
>>>  means "let SA figure out what to trust".
>> Makes sense, especially if my hunch about the "relayed through one or
>> more trusted relays, cannot use header-based Envelope-From, skipping"
>> part of the debug output I just sent to this list is on track.
>>
>> Is there a way to set trusted_networks on the command line of the
>> spamassassin command just for testing?
> This didn't work:
>
> spamassassin -D --cf='trusted_networks 127.0.0.1' -t example_email_no_spf 2>&1 | grep -i SPF
>
> All my local handoffs are to localhost [127.0.0.1] so I wouldn't know what else to use (it's an all-in-one
single server simple system)

I'm not sure if that format will work or not.  If your normal process
uses Amavisd_new or spamd, you can just edit the config files for your
tests.  Changes to the config files will not affect the daemons until
they are restarted.

At some point, there has to be an external IP to accept mail from the
Internet.  That is what you need to add to trusted_networks.  127.0.0.1

(Continue reading)

Kevin A. McGrail | 16 Feb 14:11

Re: SPF and DKIM tests by default?

On 2/15/2012 7:08 PM, email builder wrote:
> OK, but: Q: Are there any default rules as supplied by sa-update that 
> would prevent SPF rules from firing?
Not that I can think of.
>
> Q: Any other ideas on how to learn what rules are actually being used?
What I would likely do is save the gmail message to an mbox format 
file.  Then I would run spamassassin -D -t /tmp/mboxfile 2>&1 | grep -i 
SPF and see what I find.

Regards,
KAM

email builder | 16 Feb 23:38

Picon

Re: SPF and DKIM tests by default?

> 

> On 2/15/2012 7:08 PM, email builder wrote:
>>  OK, but: Q: Are there any default rules as supplied by sa-update that would 
>> prevent SPF rules from firing?
> Not that I can think of.
>> 
>>  Q: Any other ideas on how to learn what rules are actually being used?
> What I would likely do is save the gmail message to an mbox format file.  Then I 
> would run spamassassin -D -t /tmp/mboxfile 2>&1 | grep -i SPF and see 
> what I find.

Well, that was actually the other more general question that
you kindly already offered your help for - how to determine
all rules currently in use at execution time. Short of other
opinions, we'll wait to see how the bugzilla item I created
progresses.

But your advice here is in fact quite useful and may do a
fine job at pointing to the issue. Keep in mind, all rules
are as given by sa-update. I copied in all the output below
but here are what I see as key points by line number:

Line 8: Someone earlier pointed out that SA uses this
Received-SPF header, but then I think it was you that
pointed out that this shouldn't be necessary, and I added
that it would seem odd to me if SA didn't also look for the
quasi-standard "Return-Path" header which for some mailers
such as Postfix will include the envelope from address. The
lack of this header doesn't seem to stop SPF execution though.

(Continue reading)

Return

Return to gmane.mail.spam.spamassassin.general.

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane