SA Bugzilla – Bug 6086
false positive FRT_VALIUM1/FRT_VALIUM2
Last modified: 2011-05-01 20:53:52 UTC
the german word "vollumfänglich" triggers (because of "vollum") the valium rules which is a false positive. X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_VALIUM2, FUZZY_VLIUM autolearn=disabled version=3.2.4 X-Spam-Report: * 1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2) * 0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam * 3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium X-Spam-Level: ****X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_VALIUM2, FUZZY_VLIUM autolearn=disabled version=3.2.4 X-Spam-Report: * 1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2) * 0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam * 3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium X-Spam-Level: ****
while the score is a bit on the high side, your FP would have not been if you'd have autotrained Bayes in place and set your "required=" to 5.0 instead of "required=4.0" which may result in many more FPs than just this case. In this case I feel chances of false negatives overweight such a FP case which only applies to one word, in one language, while the cases of obfuscation in spam seem notably higher. -- Note: I'm not a SA developer.
I've got the same false positive on a HAM from PayPal, very annoying! And it even fires FUZZY_CREDIT, which is also a false positive! Message is at http://zmi.at/x/frt_valium_fp.txt 1.6 FRT_VALIUM1 BODY: ReplaceTags: Valium 0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam 1.3 FRT_VALIUM2 BODY: ReplaceTags: Valium (2) 2.0 TRACKER_ID BODY: Beinhaltet eine Identitätsnummer zur Nutzerbeobachtung 1.2 FUZZY_CREDIT BODY: Attempt to obfuscate words in spam
Created attachment 4444 [details] e-mail from PayPal false positive Definitely HAM and from PayPal, I verified with several other received messages!
I reveived an answer from John Hardin at 3.Mai 2009 saying: On Sun, 3 May 2009, Michael Monnerie wrote: > I've got a false positive with FRT_VALIUM1, FRT_VALIUM2 and FUZZY_VLIUM > with a german announcement from Paypal about changing their general > terms and conditions. Maybe those rules can be optimized? This came up back in March. I'm a little surprised there hasn't been any action on it, as a bugzilla ticket was supposedly opened. Here's the off-list discussion I had with Schwaller Remo, who originally reported it (accents lost, sorry): --------------------------------- On Fri, 13 Mar 2009, John Hardin wrote: > On Fri, 13 Mar 2009, Schwaller Remo wrote: > >> > Well, unfortunately it involves changes to the stock rule. If you >> > send me a specific rule that's causing you problems, and the >> > specific legitimate word it is matching on that's problematic, I'll >> > send you an example of what you'd do. >> >> thanks a lot for your help. the part "vollum" from the german word >> "vollumfnglich" triggers the Valium Rules. >> >> X-Spam-Status: Yes, score=5.0 required=4.0 tests=FRT_VALIUM1,FRT_=VALIUM2, >> FUZZY_VLIUM autolearn=disabled version=3.2.4 >> X-Spam-Report: * 1.9 FRT_VALIUM2 BODY: ReplaceTags: Valium (2) >> * 0.0 FUZZY_VLIUM BODY: Attempt to obfuscate words in spam >> * 3.0 FRT_VALIUM1 BODY: ReplaceTags: Valium > > Okay, here are the base rules: > > body FRT_VALIUM1 /<inter W0><post P2>\b(?!valium)<V><A><L><I><U><M>/i > > body FRT_VALIUM2 /<inter SP2><post P2>\b(?!valium)<V><A><L><I><U><M>/i > > You'd first turn off the base rules: > > score FRT_VALIUM1 0 > score FRT_VALIUM2 0 > > Then create substitute versions. There are two ways you could do this. > > (1) Prohibit the specific FP: > > body FRT_VALIUM1_REMO /<inter W0><post P2>\b(?!valium|vollum)<V><A><L><I><U><M>/i > > body FRT_VALIUM2_REMO /<inter SP2><post P2>\b(?!valium|vollum)<V><A><L><I><U><M>/i > > or (2) ignore anything after the match: > > body FRT_VALIUM1_REMO /<inter W0><post P2>\b(?!valium)<V><A><L><I><U><M>\b/i > > body FRT_VALIUM2_REMO /<inter SP2><post P2>\b(?!valium)<V><A><L><I><U><M>\b/i > > Both have their advantages and disadvantages. > > Finally, score them and tell ReplaceTags to process them: > > score FRT_VALIUM1_REMO 3.0 > score FRT_VALIUM2_REMO 1.9 > replace_rules FRT_VALIUM1_REMO FRT_VALIUM2_REMO > > I suggest you put these in a separate .cf file in /etc/spamassassin --------------------------------- On Thu, 19 Mar 2009, Schwaller Remo wrote: > hi john > > sorry for my late reply. > > > ... > > Finally, score them and tell ReplaceTags to process them: > > score FRT_VALIUM1_REMO 3.0 > > score FRT_VALIUM2_REMO 1.9 > > replace_rules FRT_VALIUM1_REMO FRT_VALIUM2_REMO > > I suggest you put these in a separate .cf file in /etc/spamassassin > > thanks a lot for your help. it's impletented and works like a charm :) > > kind regards + best wishes > remo Horray! I am pleased to hear it's working for you. I wonder if the SA devs will adjust the base rule? You might want to put your override rule onto the bugzilla ticket as an attachment so they can see something that works. --------------------------------- -- John Hardin KA7OHZ http://www.impsec.org/~jhardin/ jhardin@impsec.org
I got this report from Jens Schleusener today: He found that the VALIUM rules hit on messages with this links: http://www.comconsult-akademie.de/inxmail/url?vm0vq000000000iw62a3 ^^^^^^ It seems to hit all URLs that have the string "url?" and then ReplaceTags <u> and <m>, so probably (?:[vu]|\\\/) and (?:m|rn). So the problem extends not only to german language but to all mails with URLs of such names. Needs a fix, really.
*** Bug 6154 has been marked as a duplicate of this bug. ***
thanks guys; test case FPs added and replacement rule used. svn commit -m "bug 6086: fix false positives on FRT_VALIUM1, FRT_VALIUM2, thanks to Schwaller Remo" Sending rulesrc/sandbox/emailed/00_FVGT_File001.cf Adding t.rules/FRT_VALIUM1 Adding t.rules/FRT_VALIUM1/fp-bug6086-1 Adding t.rules/FRT_VALIUM1/fp-bug6086-2 Adding t.rules/FRT_VALIUM1/fp-bug6086-zmi Transmitting file data .... Committed revision 794216.
See bug 6040 for a related issue with FRT_VALIUM1, still valid after the recent rule change with 3.2.x replacement patterns.
That doesn't detect: _\/a1ium_ Is that worth reopening the bug?
(In reply to comment #9) > That doesn't detect: _\/a1ium_ Is that worth reopening the bug? yeah. We should probably collect a few test cases though :(
*** Bug 6300 has been marked as a duplicate of this bug. ***
Just to be sure, I updated stoplist to _all_ rules. Volturno from bug 6300. Just be done with it and lets close some buuugs.. (?!valium|verifiquem|volturno|vollum) Trunk: Sending rules/25_replace.cf Sending rulesrc/sandbox/emailed/00_FVGT_File001.cf Transmitting file data .. Committed revision 1098409.