Bug 867 - Some "anti-ratware" USER_AGENT scores suspiciously high.
Summary: Some "anti-ratware" USER_AGENT scores suspiciously high.
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 1.5
Hardware: Other other
: P2 normal
Target Milestone: ---
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-09-09 21:36 UTC by Matt Kettler
Modified: 2002-12-18 05:36 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Patch to mod down aol and macoe user agent scores patch None Matt Kettler [HasCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Kettler 2002-09-09 21:36:12 UTC 
USER_AGENT_MACOE and USER_AGENT_AOL are shockingly high. 

MACOE admittedly mixes with "USER_AGENT" to produce a net score of +1.874, but
this is awfuly high of a net bonus for merely using an unusual MUA.

USER_AGENT_AOL is an X-Mailer type rule so does not mix with USER_AGENT, thus
all AOL users get a score of 2.304. Admittedly AOL is abused by spammers, but
there are better rules for hitting spam.

I'd like to suggest modding these scores down to produce a net-score of aprox
+0.5 for either one. This is in-line with the net score for KMAIL.

score USER_AGENT_MACOE 1.6
score USER_AGENT_AOL 0.5

Ultimately I think these should be at-worst 0, since they are "nice" rules and
poor spam-signs, but I think the GA leaned on them a bit so I'm hesitant to
drill them all the way down.
Comment 1 Matt Kettler 2002-09-09 21:38:22 UTC 
Created attachment 320 [details]
Patch to mod down aol and macoe user agent scores
Comment 2 Duncan Findlay 2002-09-20 21:15:19 UTC 
Both are fixed in CVS (both branches)
Comment 3 Daniel Quinlan 2002-09-21 00:02:49 UTC 
I'm reopening this bug.

I don't think we should be changing GA-evolved scores.  This will cause FNs
otherwise the GA would not have set the scores this high.  In addition, the
changes will be wiped out next time the GA is run.  If these scores are "too
high", there are several options:

1. decided it is not a problem so no fix is required
2. change the GA to score simple/questionable rules better
3. remove the rules

I would be content with 2 or 3 for these rules.  2 seems better since we have
a lot of the USER_AGENT rules.
Comment 4 Matt Kettler 2002-09-21 08:36:12 UTC 
I believe these were fixed by the newer GA run of scores, not by application of
the patch. This bug applied to the 2.40/2.41 GA run, but JM ran a new GA run and
rule pruning pass which fixes both of these. We're intending to use these scores
in 2.42, correct?

Also the AOL one had a second, extra entry already in 50_scores.cf forcing it
down to -0.3, that dupe was dealt with in another bug.

as far as editing GA scores.. those scores were pattently WRONG. I know modding
down GA scores can cause unwanted FN's, however those scores were extrordinarily
likely to cause unwanted FPs. IMO it's absurd not to fix such blatant errors in
the GA's output. 

If you don't see why a score of 3.02 on a rule with a S/O of 0.02 is wrong.. you
need help :)

  0.087    0.003    0.124    0.02    3.02  USER_AGENT_MACOE

from the latest GA scores in CVS:
score USER_AGENT_MACOE               -2.742 

And USER_AGENT_AOL has been pruned due to poor S/O.

I say call this bug fixed, or wontfix. The patch is now irrelevant since the bug
has been fixed elsewhere as a part of preping for 2.42.
Comment 5 Justin Mason 2002-12-18 14:36:32 UTC 
thoroughly obsoleted; the anti-ratware scores are now -ve and
we're working on new ones anyway