Bug 6086 - false positive FRT_VALIUM1/FRT_VALIUM2
Summary: false positive FRT_VALIUM1/FRT_VALIUM2
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 3.2.4
Hardware: All Linux
: P5 normal
Target Milestone: 3.3.0
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
: 6154 6300 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-03-13 13:23 UTC by Remo Schwaller
Modified: 2011-05-01 20:53 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
e-mail from PayPal false positive text/plain None Michael Monnerie [NoCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Remo Schwaller 2009-03-13 13:23:42 UTC
the german word "vollumfänglich" triggers (because of "vollum")
the valium rules which is a false positive.

X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_VALIUM2,
	FUZZY_VLIUM autolearn=disabled version=3.2.4
X-Spam-Report: *  1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2)
	*  0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam
	*  3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium
X-Spam-Level: ****X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_VALIUM2,
	FUZZY_VLIUM autolearn=disabled version=3.2.4
X-Spam-Report: *  1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2)
	*  0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam
	*  3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium
X-Spam-Level: ****
Comment 1 AXB 2009-03-13 13:43:00 UTC
while the score is a bit on the high side, your FP would have not been if you'd have autotrained Bayes in place and set your "required=" to 5.0
instead of "required=4.0" which may result in many more FPs than just this case.

In this case I feel chances of false negatives overweight such a FP case which only applies to one word, in one language, while the cases of obfuscation in spam seem notably higher.

--
Note: I'm not a SA developer.

Comment 2 Michael Monnerie 2009-05-03 22:54:44 UTC
I've got the same false positive on a HAM from PayPal, very annoying! And it even fires FUZZY_CREDIT, which is also a false positive!

Message is at http://zmi.at/x/frt_valium_fp.txt

 1.6 FRT_VALIUM1            BODY: ReplaceTags: Valium
 0.0 FUZZY_VLIUM            BODY: Attempt to obfuscate words in spam
 1.3 FRT_VALIUM2            BODY: ReplaceTags: Valium (2)
 2.0 TRACKER_ID             BODY: Beinhaltet eine Identitätsnummer zur
Nutzerbeobachtung
 1.2 FUZZY_CREDIT           BODY: Attempt to obfuscate words in spam
Comment 3 Michael Monnerie 2009-05-03 22:57:23 UTC
Created attachment 4444 [details]
e-mail from PayPal false positive

Definitely HAM and from PayPal, I verified with several other received messages!
Comment 4 Michael Monnerie 2009-05-03 23:00:34 UTC
I reveived an answer from John Hardin at 3.Mai 2009 saying:

On Sun, 3 May 2009, Michael Monnerie wrote:

> I've got a false positive with FRT_VALIUM1, FRT_VALIUM2 and FUZZY_VLIUM 
> with a german announcement from Paypal about changing their general 
> terms and conditions. Maybe those rules can be optimized?

This came up back in March. I'm a little surprised there hasn't been any 
action on it, as a bugzilla ticket was supposedly opened.

Here's the off-list discussion I had with Schwaller Remo, who originally
reported it (accents lost, sorry):

---------------------------------

On Fri, 13 Mar 2009, John Hardin wrote:

> On Fri, 13 Mar 2009, Schwaller Remo wrote:
>
>> >  Well, unfortunately it involves changes to the stock rule. If you 
>> >  send me a specific rule that's causing you problems, and the 
>> >  specific legitimate word it is matching on that's problematic, I'll 
>> >  send you an example of what you'd do.
>>
>>  thanks a lot for your help. the part "vollum" from the german word
>>  "vollumfnglich" triggers the Valium Rules.
>>
>>  X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_=VALIUM2,
>>  FUZZY_VLIUM autolearn=disabled version=3.2.4
>>  X-Spam-Report: *  1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2)
>>   *  0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam
>>   *  3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium
>
> Okay, here are the base rules:
>
> body FRT_VALIUM1 /<inter W0><post P2>\b(?!valium)<V><A><L><I><U><M>/i
>
> body FRT_VALIUM2 /<inter SP2><post P2>\b(?!valium)<V><A><L><I><U><M>/i
>
> You'd first turn off the base rules:
>
> score FRT_VALIUM1 0
> score FRT_VALIUM2 0
>
> Then create substitute versions. There are two ways you could do this.
>
> (1) Prohibit the specific FP:
>
> body FRT_VALIUM1_REMO /<inter W0><post P2>\b(?!valium|vollum)<V><A><L><I><U><M>/i
>
> body FRT_VALIUM2_REMO /<inter SP2><post P2>\b(?!valium|vollum)<V><A><L><I><U><M>/i
>
> or (2) ignore anything after the match:
>
> body FRT_VALIUM1_REMO /<inter W0><post P2>\b(?!valium)<V><A><L><I><U><M>\b/i
>
> body FRT_VALIUM2_REMO /<inter SP2><post P2>\b(?!valium)<V><A><L><I><U><M>\b/i
>
> Both have their advantages and disadvantages.
>
> Finally, score them and tell ReplaceTags to process them:
>
> score FRT_VALIUM1_REMO 3.0
> score FRT_VALIUM2_REMO 1.9
> replace_rules FRT_VALIUM1_REMO FRT_VALIUM2_REMO
>
> I suggest you put these in a separate .cf file in /etc/spamassassin

---------------------------------

On Thu, 19 Mar 2009, Schwaller Remo wrote:

> hi john
>
> sorry for my late reply.
>
> > ...
> > Finally, score them and tell ReplaceTags to process them:
> > score FRT_VALIUM1_REMO 3.0
> > score FRT_VALIUM2_REMO 1.9
> > replace_rules FRT_VALIUM1_REMO FRT_VALIUM2_REMO
> > I suggest you put these in a separate .cf file in /etc/spamassassin
>
> thanks a lot for your help. it's impletented and works like a charm :)
>
> kind regards + best wishes
> remo

Horray! I am pleased to hear it's working for you.

I wonder if the SA devs will adjust the base rule? You might want to put 
your override rule onto the bugzilla ticket as an attachment so they can 
see something that works.

---------------------------------


-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org
Comment 5 Michael Monnerie 2009-05-12 12:33:52 UTC
I got this report from Jens Schleusener today: He found that the VALIUM rules hit on messages with this links:
  http://www.comconsult-akademie.de/inxmail/url?vm0vq000000000iw62a3
                                            ^^^^^^

It seems to hit all URLs that have the string "url?" and then ReplaceTags <u> and <m>, so probably (?:[vu]|\\\/) and (?:m|rn). So the problem extends not only to german language but to all mails with URLs of such names. Needs a fix, really.
Comment 6 Karsten Bräckelmann 2009-07-14 17:04:36 UTC
*** Bug 6154 has been marked as a duplicate of this bug. ***
Comment 7 Justin Mason 2009-07-15 03:17:56 UTC
thanks guys; test case FPs added and replacement rule used.

svn commit -m "bug 6086: fix false positives on FRT_VALIUM1, FRT_VALIUM2, thanks to Schwaller Remo" 
Sending        rulesrc/sandbox/emailed/00_FVGT_File001.cf
Adding         t.rules/FRT_VALIUM1
Adding         t.rules/FRT_VALIUM1/fp-bug6086-1
Adding         t.rules/FRT_VALIUM1/fp-bug6086-2
Adding         t.rules/FRT_VALIUM1/fp-bug6086-zmi
Transmitting file data ....
Committed revision 794216.
Comment 8 Karsten Bräckelmann 2009-07-15 07:38:21 UTC
See bug 6040 for a related issue with FRT_VALIUM1, still valid after the recent rule change with 3.2.x replacement patterns.
Comment 9 John Hardin 2009-07-15 15:55:55 UTC
That doesn't detect:   _\/a1ium_   Is that worth reopening the bug?
Comment 10 Justin Mason 2009-07-16 02:39:42 UTC
(In reply to comment #9)
> That doesn't detect:   _\/a1ium_   Is that worth reopening the bug?

yeah.  We should probably collect a few test cases though :(
Comment 11 Henrik Krohns 2011-05-01 20:47:45 UTC
*** Bug 6300 has been marked as a duplicate of this bug. ***
Comment 12 Henrik Krohns 2011-05-01 20:53:52 UTC
Just to be sure, I updated stoplist to _all_ rules. Volturno from bug 6300. Just be done with it and lets close some buuugs..

(?!valium|verifiquem|volturno|vollum)

Trunk:
Sending        rules/25_replace.cf
Sending        rulesrc/sandbox/emailed/00_FVGT_File001.cf
Transmitting file data ..
Committed revision 1098409.