Bug 6212

Summary:	Evaluate Hostkarma JMF DNSBL's
Product:	Spamassassin	Reporter:	Warren Togami <wtogami>
Component:	Rules	Assignee:	SpamAssassin Developer Mailing List <dev>
Status:	NEW ---
Severity:	enhancement	CC:	apache, kmcgrail
Priority:	P5
Version:	SVN Trunk (Latest Devel Version)
Target Milestone:	Undefined
Hardware:	Other
OS:	All
Whiteboard:

Description Warren Togami 2009-09-28 11:18:42 UTC

Please add the following rules to the sandbox for testing so we can get some statistics from the weekly masschecks.

http://wiki.junkemailfilter.com/index.php/Spam_DNS_Lists#How_to_use_the_Lists
hostkarma.junkemailfilter.com responds to DNS queries with a few types of responses.  These three rules probably interest us the most.

header __RCVD_IN_JMF eval:check_rbl('JMF-lastexternal','hostkarma.junkemailfilter.com.')
describe __RCVD_IN_JMF Sender listed in JunkEmailFilter, ttp://hostkarma.junkemailfilter.com
tflags __RCVD_IN_JMF net
 
header RCVD_IN_JMF_W eval:check_rbl_sub('JMF-lastexternal', '127.0.0.1')
describe RCVD_IN_JMF_W Sender listed in JMF-WHITE, http://hostkarma.junkemailfilter.com
tflags RCVD_IN_JMF_W net nice
 
header RCVD_IN_JMF_BL eval:check_rbl_sub('JMF-lastexternal', '127.0.0.2')
describe RCVD_IN_JMF_BL Sender listed in JMF-BLACK, http://hostkarma.junkemailfilter.com
tflags RCVD_IN_JMF_BL net
 
header RCVD_IN_JMF_BR eval:check_rbl_sub('JMF-lastexternal', '127.0.0.4')
describe RCVD_IN_JMF_BR Sender listed in JMF-BROWN, http://hostkarma.junkemailfilter.com
tflags RCVD_IN_JMF_BR net

The "yellow" list claims to list IP's of Yahoo, GMail, Hotmail, etc.  It costs us nothing to trigger a sub-rule on it.  It might be interesting to see its results in masschecks.

header RCVD_IN_JMF_YL eval:check_rbl_sub('JMF-lastexternal', '127.0.0.3')
describe RCVD_IN_JMF_YL Sender listed in JMF-YELLOW, http://hostkarma.junkemailfilter.com
tflags RCVD_IN_JMF_YL net

Comment 1 Warren Togami 2009-09-29 19:25:37 UTC

Sigh, line wrapping mangled those rules.

There is some discussion on users@ list about the names of these rules.  For a while Marc had RCVD_IN_JMF_* on his wiki page and a few people use it copied from that.  But Marc prefers to have HOSTKARMA in the name.

RCVD_IN_HOSTKARMA_BL
RCVD_IN_HOSTKARMA_WL
RCVD_IN_HOSTKARMA_YL
RCVD_IN_HOSTKARMA_BR

I personally prefer these rule names because they are easier to read.  The only possible difficulty is people who might have manually configured their spamassassin using the old JMF rule names need to know to remove the JMF rules.

Do the developers have any strong opinions either way?

Comment 2 Henrik Krohns 2009-10-20 23:07:49 UTC

Might as well test these rules then..

urirhssub URIBL_HOSTKARMA_BL hostkarma.junkemailfilter.com. A 127.0.0.2
body      URIBL_HOSTKARMA_BL eval:check_uridnsbl('URIBL_HOSTKARMA_BL')
tflags    URIBL_HOSTKARMA_BL net nopublish

urirhssub URIBL_HOSTKARMA_BR hostkarma.junkemailfilter.com. A 127.0.0.4
body      URIBL_HOSTKARMA_BR eval:check_uridnsbl('URIBL_HOSTKARMA_BR')
tflags    URIBL_HOSTKARMA_BR net nopublish

urirhssub URIBL_HOSTKARMA_FRESH_2D hostkarma.junkemailfilter.com. A 127.0.2.1
body      URIBL_HOSTKARMA_FRESH_2D eval:check_uridnsbl('URIBL_HOSTKARMA_FRESH_2D')
tflags    URIBL_HOSTKARMA_FRESH_2D net nopublish

urirhssub URIBL_HOSTKARMA_FRESH_10D hostkarma.junkemailfilter.com. A 127.0.2.2
body      URIBL_HOSTKARMA_FRESH_10D eval:check_uridnsbl('URIBL_HOSTKARMA_FRESH_10D')
tflags    URIBL_HOSTKARMA_FRESH_10D net nopublish

BL and FRESH_2D actually work decend here, with 0.95+ S/O.

Comment 3 Warren Togami 2009-10-21 08:40:27 UTC

Where are these documented?  I don't see these on his wiki.

Comment 4 Henrik Krohns 2009-10-21 09:27:31 UTC

I suggest you read the wiki more closely then. All the information is there.

Comment 5 Warren Togami 2009-10-21 19:03:29 UTC

The wiki makes no mention of it being meant to be used as URIBL.  I asked Marc about this and he said it might give some interesting statistics though.  I'll add it to the sandbox.

Comment 6 AXB 2012-08-12 10:32:27 UTC

I'd like to propose removal of these test from sandboxes

( /trunk/rulesrc/sandbox/wtogami/20_bug_6212_hostkarma.cf )
 
They're set to nopublish (since 2009) and while it's nice to test new BLs, it puts an unnecessaary load on weekly masschecks to keep them there for such a long time.

comments, votes please!

Comment 7 Kevin A. McGrail 2012-08-12 15:33:06 UTC

(In reply to comment #6)
> I'd like to propose removal of these test from sandboxes
> 
> ( /trunk/rulesrc/sandbox/wtogami/20_bug_6212_hostkarma.cf )
>  
> They're set to nopublish (since 2009) and while it's nice to test new BLs,
> it puts an unnecessaary load on weekly masschecks to keep them there for
> such a long time.
> 
> comments, votes please!

+1 to comment them (not removal just to be clear).

However, should someone want to actively test and analyze them, I would immediately support that.  But we need mascheck to be quicker and this analysis isn't currently useful to the project.

Comment 8 AXB 2012-08-13 08:05:32 UTC

FTR: Rules commented out on Aug 12 2012