Alex | 10 Apr 09:09 2010
Picon

RBL Usage questions

Hi,

I'm trying to evaluate the block lists that are available to be used
at SMTP connection time with reject_maps_rbl, and wondered if someone
had any input. I have spent quite a bit of time researching many of
the block lists, but much of the information is somewhat old and I'm
concerned that it's outdated and unreliable.

Is there a "best practices" document that includes recommendations or
suggestions on which RBLs to use for which purpose? I'd also like to
be able to gather some stats on them, such as how many rejects,
queries, perhaps even the IPs that were rejected, so that I may
collect this information and create some historical data.

I'm currently considering multi.uribl.com and multi.surbl.org as the
top two, but even with that I've read that in the past there were
great concerns that they'd get knocked offline and what the
implications would be for the postfix server.

Does anyone know anything about these stats?

http://www.intra2net.com/en/support/antispam/index.php_sort=accuracy_order=desc.html

There certainly isn't a shortage of block lists to choose from:

http://www.dnsbl.info/dnsbl-list.php

The spamlinks site has done a great job of indexing the RBLs, but it
doesn't say anything about what kind of reputation they have, or if
they're really suitable for real-world use:
(Continue reading)

Ralf Hildebrandt | 10 Apr 09:21 2010
Picon

Re: RBL Usage questions

* Alex <mysqlstudent <at> gmail.com>:
> Hi,
> 
> I'm trying to evaluate the block lists that are available to be used
> at SMTP connection time with reject_maps_rbl, 

reject_maps_rbl is deprecated. Use reject_rbl_client et.al.

> Is there a "best practices" document that includes recommendations or
> suggestions on which RBLs to use for which purpose? I'd also like to
> be able to gather some stats on them, such as how many rejects,
> queries, perhaps even the IPs that were rejected, so that I may
> collect this information and create some historical data.

That doesn't help. Everybody's spam is different AND you forget the
false positives!

> I'm currently considering multi.uribl.com and multi.surbl.org as the
> top two, but even with that I've read that in the past there were
> great concerns that they'd get knocked offline and what the
> implications would be for the postfix server.

I'm using zen.spamhaus.org in postscreen and,

   reject_rbl_client           bl.spamcop.net
   reject_rbl_client           bogons.cymru.com
   reject_rhsbl_sender         dbl.spamhaus.org
   reject_rhsbl_reverse_client dbl.spamhaus.org

--

-- 
(Continue reading)

Stan Hoeppner | 11 Apr 00:49 2010

Re: RBL Usage questions

Ralf Hildebrandt put forth on 4/10/2010 2:21 AM:

> I'm using zen.spamhaus.org in postscreen and,
> 
>    reject_rbl_client           bl.spamcop.net
>    reject_rbl_client           bogons.cymru.com
>    reject_rhsbl_sender         dbl.spamhaus.org
>    reject_rhsbl_reverse_client dbl.spamhaus.org

Using these dnsbls here:

smtpd_recipient_restrictions =
	...
        reject_rbl_client   zen.spamhaus.org
        reject_rhsbl_client dbl.spamhaus.org
        reject_rhsbl_sender dbl.spamhaus.org
        reject_rhsbl_helo   dbl.spamhaus.org
	...

I reject most spam via other methods, mostly pcre/regex and cidr tables.  My
dnsbl queries reject less than 1% of my spam load.  Plug the following
dynamic/generic rdns regex table into your Postfix configuration and see if
it catches some spam for you.  It does a good job here.  Given its size I'd
recommend running it (and all your map files) via proxymap.  Ask here if
you're unsure or need help implementing proxymap.  It bit me the first time
I tried it.

smtpd_recipient_restrictions =
	...
        check_client_access regexp:/etc/postfix/fqrdns.regexp
(Continue reading)

Reinaldo de Carvalho | 11 Apr 00:56 2010
Picon

Re: RBL Usage questions

On Sat, Apr 10, 2010 at 7:49 PM, Stan Hoeppner <stan <at> hardwarefreak.com> wrote:
> smtpd_recipient_restrictions =
>        ...
>        check_client_access regexp:/etc/postfix/fqrdns.regexp
>        ...
>
> /etc/postfix/fqrdns.regexp
> http://www.hardwarefreak.com/fqrdns.regexp
>
> This regex file is free for anyone to use if you wish to.  The FP rate
> should be zero since it matches only dynamic/generic rdns names.
>
> --
> Stan
>

In other words:

/([0-9]{1,3}(\.|-)){3}.*\.[a-z]+/ reject generic hostname
/(^a?dsl|a?dsl(\.|-)|(\.|-)a?dsl|(\.|-)d(yn|ip|ial)(\.|-)|(\.|-)cable(\.|-)|(\.|-)user(\.|-)|^dynamic|(\.|-)dynamic|dynamic(\.|-)|(\.|-)ppp(oe)?(\.|-|)|^ppp)/
  reject generic hostname

--

-- 
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

"Don't try to adapt the software to the way you work, but rather
yourself to the way the software works" (myself)

(Continue reading)

Stan Hoeppner | 11 Apr 01:10 2010

Re: RBL Usage questions

Reinaldo de Carvalho put forth on 4/10/2010 5:56 PM:

> In other words:
> 
> /([0-9]{1,3}(\.|-)){3}.*\.[a-z]+/ reject generic hostname
> /(^a?dsl|a?dsl(\.|-)|(\.|-)a?dsl|(\.|-)d(yn|ip|ial)(\.|-)|(\.|-)cable(\.|-)|(\.|-)user(\.|-)|^dynamic|(\.|-)dynamic|dynamic(\.|-)|(\.|-)ppp(oe)?(\.|-|)|^ppp)/
>   reject generic hostname

Except these aren't fully qualified patterns, can generate FPs, and cause
other problems.  The patterns I shared are fully qualified, so the chance of
FPs is zero or near zero.  Also note the domain specific reject text in my
patterns.

Your patterns are what many people start out with.  They may work fine for a
while on low volume vanity servers for the family and the dog, but they
don't work well on real mail streams at decent sized organizations.  This
was discussed at length on spam-l not too long ago.  That's how I ended up
with the regexp file I shared here, because I was previously using something
generic like that above, and a seasoned OP took pity on me (and others).

--

-- 
Stan

Reinaldo de Carvalho | 11 Apr 01:28 2010
Picon

Re: RBL Usage questions

On Sat, Apr 10, 2010 at 8:10 PM, Stan Hoeppner <stan <at> hardwarefreak.com> wrote:
>
> Except these aren't fully qualified patterns, can generate FPs, and cause
> other problems.  The patterns I shared are fully qualified, so the chance of
> FPs is zero or near zero.  Also note the domain specific reject text in my
> patterns.
>
> Your patterns are what many people start out with.  They may work fine for a
> while on low volume vanity servers for the family and the dog, but they
> don't work well on real mail streams at decent sized organizations.

Please don't generalize. The organization size isn't the point.

>  This was discussed at length on spam-l not too long ago.  That's how I ended up
> with the regexp file I shared here, because I was previously using something
> generic like that above, and a seasoned OP took pity on me (and others).
>
> --
> Stan
>

Disclose the organization rules to receive emails on main website. Put
a URL in reject reason to inform the problem (need a patch for
build-in restrictions).

--

-- 
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

(Continue reading)

Alex | 11 Apr 02:28 2010
Picon

Re: RBL Usage questions

Hi,

>> I'm using zen.spamhaus.org in postscreen and,

Where can I find information on postscreen?

>>    reject_rbl_client           bl.spamcop.net
>>    reject_rbl_client           bogons.cymru.com

I would also be interested in info on using the bogons list here. How
does that apply here, considering it's not very effective to spoof the
source of an email that you'd actually like to be delivered in the
first place, no?

> smtpd_recipient_restrictions =
>        ...
>        reject_rbl_client   zen.spamhaus.org
>        reject_rhsbl_client dbl.spamhaus.org
>        reject_rhsbl_sender dbl.spamhaus.org
>        reject_rhsbl_helo   dbl.spamhaus.org

I'm familiar with zen, but I believe the dbl is relatively new,
correct? What other URI BL lists do people use?

Can these postfix restrictions be used with older versions of postfix?
I have a few systems with older versions and can't upgrade right now.

> I reject most spam via other methods, mostly pcre/regex and cidr tables.  My

Can you tell me more about rejecting using cidr tables? Do you mean
(Continue reading)

Sahil Tandon | 11 Apr 02:49 2010
Picon

Re: RBL Usage questions

On Sat, 10 Apr 2010, Alex wrote:

> >> I'm using zen.spamhaus.org in postscreen and,
> 
> Where can I find information on postscreen?

postscreen(8) is part of the 2.8 experimental release:

 http://www.postfix.org/postscreen.8.html

--

-- 
Sahil Tandon <sahil <at> tandon.net>

Stan Hoeppner | 11 Apr 06:16 2010

Re: RBL Usage questions

Alex put forth on 4/10/2010 7:28 PM:

>> smtpd_recipient_restrictions =
>>        ...
>>        reject_rbl_client   zen.spamhaus.org
>>        reject_rhsbl_client dbl.spamhaus.org
>>        reject_rhsbl_sender dbl.spamhaus.org
>>        reject_rhsbl_helo   dbl.spamhaus.org
> 
> I'm familiar with zen, but I believe the dbl is relatively new,
> correct? What other URI BL lists do people use?

The dbl is somewhat unique in that it's not just a uri domain list.  It also
contains domain names that directly send spam.  This is how I use the dbl.
It stops a few spam, and every little bit counts.

> Can these postfix restrictions be used with older versions of postfix?
> I have a few systems with older versions and can't upgrade right now.

I'm not sure about all of them.  Check yourself here:
http://www.postfix.org/postconf.5.html

>> I reject most spam via other methods, mostly pcre/regex and cidr tables.  My
> 
> Can you tell me more about rejecting using cidr tables? Do you mean
> the bogon list or ASN numbers? I seem to remember a downloadable list
> of the top 10 ASNs that could be used to add weight to an SA score.

I've built a cidr table over the last couple of years of mostly showshoe
networks that have spammed me.  I also use ipdeny.com cidr ranges to block
(Continue reading)

Ralf Hildebrandt | 11 Apr 12:53 2010
Picon

Re: RBL Usage questions

* Alex <mysqlstudent <at> gmail.com>:
> Hi,
> 
> >> I'm using zen.spamhaus.org in postscreen and,
> 
> Where can I find information on postscreen?

On this mailinglist? In the archives?

> >>    reject_rbl_client           bl.spamcop.net
> >>    reject_rbl_client           bogons.cymru.com
> 
> I would also be interested in info on using the bogons list here. How
> does that apply here, considering it's not very effective to spoof the
> source of an email that you'd actually like to be delivered in the
> first place, no?

I basically use it to block internal clients that should not connect to
my mailserver but use another server.

--

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebrandt <at> charite.de | http://www.charite.de
	    

(Continue reading)

Noel Jones | 11 Apr 03:16 2010

Re: RBL Usage questions

On 4/10/2010 5:49 PM, Stan Hoeppner wrote:
> I reject most spam via other methods, mostly pcre/regex and cidr tables.  My
> dnsbl queries reject less than 1% of my spam load.  Plug the following
> dynamic/generic rdns regex table into your Postfix configuration and see if
> it catches some spam for you.  It does a good job here.  Given its size I'd
> recommend running it (and all your map files) via proxymap.  Ask here if
> you're unsure or need help implementing proxymap.  It bit me the first time
> I tried it.
>
> smtpd_recipient_restrictions =
> 	...
>          check_client_access regexp:/etc/postfix/fqrdns.regexp
> 	...

You'll probably get more hits using 
check_reverse_client_hostname_access.  That prevents some 
clients from sneaking through as "unknown" when they don't 
have a matching A record.

http://www.postfix.org/postconf.5.html#check_reverse_client_hostname_access

   -- Noel Jones

>
> /etc/postfix/fqrdns.regexp
> http://www.hardwarefreak.com/fqrdns.regexp
>
> This regex file is free for anyone to use if you wish to.  The FP rate
> should be zero since it matches only dynamic/generic rdns names.
>
(Continue reading)

Stan Hoeppner | 11 Apr 04:55 2010

Re: RBL Usage questions

Noel Jones put forth on 4/10/2010 8:16 PM:
> On 4/10/2010 5:49 PM, Stan Hoeppner wrote:
>> smtpd_recipient_restrictions =
>>     ...
>>          check_client_access regexp:/etc/postfix/fqrdns.regexp
>>     ...
> 
> 
> You'll probably get more hits using
> check_reverse_client_hostname_access.  That prevents some clients from
> sneaking through as "unknown" when they don't have a matching A record.
> 
> http://www.postfix.org/postconf.5.html#check_reverse_client_hostname_access

I'm still on Debian Stable (Lenny) Postfix 2.5.5.  When Squeeze is rolled to
stable and I get Postfix 2.6.5 I'll be changing this, as well as some other
parameters that are only available in 2.6+

--

-- 
Stan

Michael Orlitzky | 11 Apr 02:27 2010

Re: RBL Usage questions

On 04/10/2010 03:21 AM, Ralf Hildebrandt wrote:
> 
> I'm using zen.spamhaus.org in postscreen and,
> 
>    reject_rbl_client           bl.spamcop.net
>    reject_rbl_client           bogons.cymru.com
>    reject_rhsbl_sender         dbl.spamhaus.org
>    reject_rhsbl_reverse_client dbl.spamhaus.org
> 

How much use do you get out of bogons.cymru.com at the SMTP stage? I was
considering it once, and talked myself out of it with flowcharts or
something.

Ralf Hildebrandt | 11 Apr 12:55 2010
Picon

Re: RBL Usage questions

* Michael Orlitzky <michael <at> orlitzky.com>:

> How much use do you get out of bogons.cymru.com at the SMTP stage? I was
> considering it once, and talked myself out of it with flowcharts or
> something.

Almost none. I use it for internal clients.

--

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebrandt <at> charite.de | http://www.charite.de
	    

Sean Reifschneider | 10 Apr 11:35 2010

Re: RBL Usage questions

On 04/10/2010 01:09 AM, Alex wrote:
> I'm trying to evaluate the block lists that are available to be used

I don't like allowing any blacklists to have serious power over blocking
e-mail.  I prefer using SpamAssassin, which will do lookups on many RBLs,
and then use the results to influence the score.  So if one RBL says
something bad, it doesn't influence it as much as if many of them do.

Sean
--

-- 
Sean Reifschneider, Member of Technical Staff <jafo <at> tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

mouss | 10 Apr 15:00 2010
Picon

Re: RBL Usage questions

Sean Reifschneider a écrit :
> On 04/10/2010 01:09 AM, Alex wrote:
>> I'm trying to evaluate the block lists that are available to be used
> 
> I don't like allowing any blacklists to have serious power over blocking
> e-mail.  I prefer using SpamAssassin, which will do lookups on many RBLs,
> and then use the results to influence the score.  So if one RBL says
> something bad, it doesn't influence it as much as if many of them do.
> 

That works for small sites who can afford to content filter all mail.
For other sites, this is no more an option.

and besides, I see more false positives with Spamassassin than with
zen.spamhaus.org. and the spam folder (or quarantine...) only works if
it's not full of junk.

Back to OP question: Use zen.spamhaus.org. then for other lists, use
them with warn_if_reject during some time and see if they bring value
without causing false positives.

Ansgar Wiechers | 10 Apr 17:36 2010
Picon

Re: RBL Usage questions

On 2010-04-10 mouss wrote:
> Sean Reifschneider a écrit :
>> I don't like allowing any blacklists to have serious power over
>> blocking e-mail.  I prefer using SpamAssassin, which will do lookups
>> on many RBLs, and then use the results to influence the score.  So if
>> one RBL says something bad, it doesn't influence it as much as if
>> many of them do.
> 
> That works for small sites who can afford to content filter all mail.
> For other sites, this is no more an option.

policyd-weight does the same without content filtering.

Regards
Ansgar Wiechers
--

-- 
"Abstractions save us time working, but they don't save us time learning."
--Joel Spolsky

mouss | 10 Apr 22:14 2010
Picon

Re: RBL Usage questions

Ansgar Wiechers a écrit :
> On 2010-04-10 mouss wrote:
>> Sean Reifschneider a écrit :
>>> I don't like allowing any blacklists to have serious power over
>>> blocking e-mail.  I prefer using SpamAssassin, which will do lookups
>>> on many RBLs, and then use the results to influence the score.  So if
>>> one RBL says something bad, it doesn't influence it as much as if
>>> many of them do.
>> That works for small sites who can afford to content filter all mail.
>> For other sites, this is no more an option.
> 
> policyd-weight does the same without content filtering.

Indeed. but here, zen is "reliable". so I use it directly.


Gmane