SA Bugzilla – Bug 4236
Check continent of envelope sender against continent of IP
Last modified: 2006-09-12 03:07:53 UTC
I'm not 100% sure but I don't think the current country checking (RelayCountry plugin) works this way. Apologies in advance if this duplicates an existing feature but I didn't spot it in bugzilla or wiki. A lot of spam has a forged envelope sender, often in a seemingly randomly-chosen part of the world. A lot of spam is relayed via hosts in various parts of the world. Many spammers make no effort to ensure that these two line up, so often spam will show up pretending to be from (for example) a European TLD but it's been relayed through a host in an IP block assigned to APNIC. It's possible that some legitimate email travels this way but that would be relatively rare, and a rule which assigns a moderate score to mail whose envelope sender is in a different part of the world than the IP address from which the mail arrived is helpful for me (I've been doing that for a while, as well as blocking such mail at the SMTP-receiver level on some systems).
Steve, Sounds like a reasonable enhancement request, but I wonder about its feasibility. Do I read you correctly that you're suggesting the capability to 1) determine the continent of the From TLD (eg: IT = europe), 2) identify the continents of all remote hosts listed in Received headers, and if there's no match (the email in this example didn't come from or through Europe), then report this via a "rule" hit? Since you're apparently doing that in your system, can you report what the overhead is for querying the network to identify the continents involved in an email? Thanks.
Adding provisional "dependency" -- not really dependent, but the patch provided in bug 3815 looks like it can help this enhancement development.
Yes, you understand the gist of it correctly. The way I've chosen to implement it can't detect the continent of 100% of emails but it is simple, should be efficient, and requires zero network traffic. I check the first octet of the IP address to see if it's one of the /8 (or larger) blocks assigned to regional registries such as RIPE; this is easy in that there's a complete published list of these delegations. Given such a list for smaller blocks (primarily the historical blocks which were delegated in olden days) it shouldn't be too difficult to expand that. As for detecting the nationality of the sender, that's also not bulletproof, as there's no foolproof way to know where in the world a .com is (for instance), particularly if it belongs to a multinational company, but for ccTLDs it's very simple. I also have rules for domains which, AFAIK, do not send mail from foreign (to me) areas; for instance, I don't believe aol.com sends from RIPE et al. and neither should a .ca (Canada) address. It may even be feasible to extend this approach to the point of checking the country of the IP address (via RelayCountry) against the ccTLD of the sender, but I don't know if any of the major multinational ISPs (e.g. wanadoo) share mail servers among multiple countries; if so, there would be some false positives. I'll attach samples of the rules I'm using. Some cautionary notes: I'm lazy, so I've hard-coded the list of servers which accept mail for my domain to simplify the parsing of the Received: line (but I know SA already has code to do this properly so it shouldn't be a big deal for the dev team to do a much better job). The scores I use are definitely not recommended for production! The lists of which countries belong on which continents is incomplete (for the most part, it's built from spam which has made it through SA and into my mailbox; if I've never received spam claiming to be from a country, that country won't be on my list).
Created attachment 2773 [details] Sample rules to implement this suggestion
Placing rules into body for mass-check {{{ # Check what part of the world the message came from and compare it # to the supposed sender's domain; list of mail servers for my domain # omitted. The parsing of Received: lines needs to be improved if # this is put into production! And the list of blocks delegated to # various registries is incomplete, as it only shows /8 or larger. # header __RCVD_APNIC Received =~ /(\[|\()(58|59|60|61|202|203|210|211|218|219|220|221|222)\.\d+\.\d+\.\d+.+by MAIL.SERVER.FOR.MY.DOMAIN/ header __RCVD_RIPE Received =~ /(\[|\()(62|80|81|82|83|84||85|86|87|88|193|194|195|212|213|217)\.\d+\.\d+\.\d+.+by MAIL.SERVER.FOR.MY.DOMAIN/ header __RCVD_LACNIC Received =~ /(\[|\()20(0|1)\.\d+\.\d+\.\d+.+by MAIL.SERVER.FOR.MY.DOMAIN/ meta __RCVD_FOREIGN (__RCVD_APNIC || __RCVD_RIPE || __RCVD_LACNIC) # Check for some domains which shouldn't send from foreign continents header __FROM_AOL From =~ /\@aol\.com/i meta SPAMMY_FOREIGN_AOL (__RCVD_FOREIGN && __FROM_AOL) score SPAMMY_FOREIGN_AOL 8 describe SPAMMY_FOREIGN_AOL Claims to be from AOL but is foreign # [I have similar rules for domains such as msn.com, hotmail.com, # and .ca; omitted for brevity] # Now, look for mail coming from the wrong continent for its # claimed origin # header __FROM_RIPE From:addr =~ /\.(at|ba|ch|cz|de|dk|eg|es|fi|fr|gr|hr|hu|ie|il|is|it|lt|lu|lv|ma|nl|no|pl|pt|ro|ru|se|sk|ua|uk|za)$/i meta SPAMMY_FOREIGN_RIPE (__FROM_RIPE && (__RCVD_APNIC || __RCVD_LACNIC)) score SPAMMY_FOREIGN_RIPE 10 describe SPAMMY_FOREIGN_RIPE Claims to be from RIPE but is not header __FROM_APNIC From:addr =~ /\.(au|cn|in|jp|kr|nz|ph|pk|sg|th)$/i meta SPAMMY_FOREIGN_APNIC (__FROM_APNIC && (__RCVD_RIPE || __RCVD_LACNIC)) score SPAMMY_FOREIGN_APNIC 8 describe SPAMMY_FOREIGN_APNIC Claims to be from APNIC but is not header __FROM_LACNIC From:addr =~ /\.(ar|br|cl|co|mx|pe|ve)$/i meta SPAMMY_FOREIGN_LACNIC (__FROM_LACNIC && (__RCVD_RIPE || __RCVD_APNIC)) score SPAMMY_FOREIGN_LACNIC 6 describe SPAMMY_FOREIGN_LACNIC Claims to be from LACNIC but is }}} NEEDSMC
# [automatically generated by automc: start] # DONEMC 5: completed request from comment 5 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_APNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_RIPE_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_LACNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_FOREIGN_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_AOL_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 8.00 T_MC_SPAMMY_FOREIGN_AOL_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_RIPE_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 10.00 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_APNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 8.00 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_LACNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 6.00 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c4 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_APNIC_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_RIPE_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_LACNIC_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __RCVD_FOREIGN_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_AOL_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 8.00 T_MC_SPAMMY_FOREIGN_AOL_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_RIPE_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 10.00 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_APNIC_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 8.00 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 1.00 __FROM_LACNIC_b4236_c5 0.000 0.0000 0.0000 0.500 0.46 6.00 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c5 above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Fri Apr 29 16:44:56 2005: __RCVD_APNIC_b4236_c4 = __RCVD_APNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c4&date=20050429 __RCVD_RIPE_b4236_c4 = __RCVD_RIPE from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c4&date=20050429 __RCVD_LACNIC_b4236_c4 = __RCVD_LACNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c4&date=20050429 __RCVD_FOREIGN_b4236_c4 = __RCVD_FOREIGN from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c4&date=20050429 __FROM_AOL_b4236_c4 = __FROM_AOL from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c4&date=20050429 T_MC_SPAMMY_FOREIGN_AOL_b4236_c4 = SPAMMY_FOREIGN_AOL from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c4&date=20050429 __FROM_RIPE_b4236_c4 = __FROM_RIPE from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c4&date=20050429 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c4 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c4&date=20050429 __FROM_APNIC_b4236_c4 = __FROM_APNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c4&date=20050429 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c4 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c4&date=20050429 __FROM_LACNIC_b4236_c4 = __FROM_LACNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c4&date=20050429 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c4 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 4 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c4&date=20050429 __RCVD_APNIC_b4236_c5 = __RCVD_APNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c5&date=20050429 __RCVD_RIPE_b4236_c5 = __RCVD_RIPE from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c5&date=20050429 __RCVD_LACNIC_b4236_c5 = __RCVD_LACNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c5&date=20050429 __RCVD_FOREIGN_b4236_c5 = __RCVD_FOREIGN from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c5&date=20050429 __FROM_AOL_b4236_c5 = __FROM_AOL from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c5&date=20050429 T_MC_SPAMMY_FOREIGN_AOL_b4236_c5 = SPAMMY_FOREIGN_AOL from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c5&date=20050429 __FROM_RIPE_b4236_c5 = __FROM_RIPE from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c5&date=20050429 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c5 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c5&date=20050429 __FROM_APNIC_b4236_c5 = __FROM_APNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c5&date=20050429 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c5 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c5&date=20050429 __FROM_LACNIC_b4236_c5 = __FROM_LACNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c5&date=20050429 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c5 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 5 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c5&date=20050429 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 452078 345485 106593 0.764 0.00 0.00 (all messages) 100.000 76.4215 23.5785 0.764 0.00 0.00 (all messages as %) # [automatically generated by automc: end]
Zero hits on > header __FROM_RIPE From:addr =~ /\.(at|ba|ch|cz|de|... I should be able to tweak the rule so it actually hits emails...
New mass-check attempt. My results: Section 3 -- Frequencies Log (First numeric frequencies, followed by percentage frequencies) OVERALL% SPAM% HAM% S/O RANK SCORE NAME 280511 109315 171196 0.390 0.00 0.00 (all messages) 1083 1048 35 0.979 1.00 6.00 SPAMMY_FOREIGN_LACNIC 8585 8521 64 0.995 1.00 10.00 SPAMMY_FOREIGN_RIPE 589 569 20 0.978 0.67 8.00 SPAMMY_FOREIGN_AOL 1042 949 93 0.941 0.00 8.00 SPAMMY_FOREIGN_APNIC OVERALL% SPAM% HAM% S/O RANK SCORE NAME 280511 109315 171196 0.390 0.00 0.00 (all messages) 100.000 38.9700 61.0300 0.390 0.00 0.00 (all messages as %) 0.386 0.9587 0.0204 0.979 1.00 6.00 SPAMMY_FOREIGN_LACNIC 3.060 7.7949 0.0374 0.995 1.00 10.00 SPAMMY_FOREIGN_RIPE 0.210 0.5205 0.0117 0.978 0.67 8.00 SPAMMY_FOREIGN_AOL 0.371 0.8681 0.0543 0.941 0.00 8.00 SPAMMY_FOREIGN_APNIC SPAMMY_FOREIGN_RIPE looks particularly useful here. {{{ header __RCVD_APNIC Received =~ /(\[|\()(58|59|60|61|202|203|210|211|218|219|220|221|222)\.\d+\.\d+\.\d+.+ by / header __RCVD_RIPE Received =~ /(\[|\()(62|80|81|82|83|84||85|86|87|88|193|194|195|212|213|217)\.\d+\.\d+\.\d+.+ by / header __RCVD_LACNIC Received =~ /(\[|\()20(0|1)\.\d+\.\d+\.\d+.+ by / meta __RCVD_FOREIGN (__RCVD_APNIC || __RCVD_RIPE || __RCVD_LACNIC) # Check for some domains which shouldn't send from foreign continents header __FROM_AOL From =~ /\@aol\.com/i meta SPAMMY_FOREIGN_AOL (__RCVD_FOREIGN && __FROM_AOL) score SPAMMY_FOREIGN_AOL 8 describe SPAMMY_FOREIGN_AOL Claims to be from AOL but is foreign header __FROM_RIPE From:addr =~ /\.(at|ba|ch|cz|de|dk|eg|es|fi|fr|gr|hr|hu|ie|il|is|it|lt|lu|lv|ma|nl|no|pl|pt|ro|ru|se|sk|ua|uk|za)>?$/i meta SPAMMY_FOREIGN_RIPE (__FROM_RIPE && (__RCVD_APNIC || __RCVD_LACNIC)) score SPAMMY_FOREIGN_RIPE 10 describe SPAMMY_FOREIGN_RIPE Claims to be from RIPE but is not header __FROM_APNIC From:addr =~ /\.(au|cn|in|jp|kr|nz|ph|pk|sg|th)>?$/i meta SPAMMY_FOREIGN_APNIC (__FROM_APNIC && (__RCVD_RIPE || __RCVD_LACNIC)) score SPAMMY_FOREIGN_APNIC 8 describe SPAMMY_FOREIGN_APNIC Claims to be from APNIC but is not header __FROM_LACNIC From:addr =~ /\.(ar|br|cl|co|mx|pe|ve)>?$/i meta SPAMMY_FOREIGN_LACNIC (__FROM_LACNIC && (__RCVD_RIPE || __RCVD_APNIC)) score SPAMMY_FOREIGN_LACNIC 6 describe SPAMMY_FOREIGN_LACNIC Claims to be from LACNIC but is not }}} NEEDSMC
# [automatically generated by automc: start] # DONEMC 8: completed request from comment 8 (could not find freqs for rule '__RCVD_APNIC_b4236_c8'/'__RCVD_APNIC') (could not find freqs for rule '__RCVD_RIPE_b4236_c8'/'__RCVD_RIPE') (could not find freqs for rule '__RCVD_LACNIC_b4236_c8'/'__RCVD_LACNIC') (could not find freqs for rule '__RCVD_FOREIGN_b4236_c8'/'__RCVD_FOREIGN') (could not find freqs for rule '__FROM_AOL_b4236_c8'/'__FROM_AOL') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_AOL_b4236_c8'/'SPAMMY_FOREIGN_AOL') (could not find freqs for rule '__FROM_RIPE_b4236_c8'/'__FROM_RIPE') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8'/'SPAMMY_FOREIGN_RIPE') (could not find freqs for rule '__FROM_APNIC_b4236_c8'/'__FROM_APNIC') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8'/'SPAMMY_FOREIGN_APNIC') (could not find freqs for rule '__FROM_LACNIC_b4236_c8'/'__FROM_LACNIC') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8'/'SPAMMY_FOREIGN_LACNIC') above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Thu May 12 01:50:05 2005: __RCVD_APNIC_b4236_c8 = __RCVD_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c8&date=20050512 __RCVD_RIPE_b4236_c8 = __RCVD_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c8&date=20050512 __RCVD_LACNIC_b4236_c8 = __RCVD_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c8&date=20050512 __RCVD_FOREIGN_b4236_c8 = __RCVD_FOREIGN from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c8&date=20050512 __FROM_AOL_b4236_c8 = __FROM_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 = SPAMMY_FOREIGN_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c8&date=20050512 __FROM_RIPE_b4236_c8 = __FROM_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8&date=20050512 __FROM_APNIC_b4236_c8 = __FROM_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8&date=20050512 __FROM_LACNIC_b4236_c8 = __FROM_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8&date=20050512 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-daf.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-daf.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 494196 370856 123340 0.750 0.00 0.00 (all messages) 100.000 75.0423 24.9577 0.750 0.00 0.00 (all messages as %) bug 4236 cmt 6: ignored, lint failed # [automatically generated by automc: end]
# [automatically generated by automc: start] # DONEMC 8: completed request from comment 8 (could not find freqs for rule '__RCVD_APNIC_b4236_c8'/'__RCVD_APNIC') (could not find freqs for rule '__RCVD_RIPE_b4236_c8'/'__RCVD_RIPE') (could not find freqs for rule '__RCVD_LACNIC_b4236_c8'/'__RCVD_LACNIC') (could not find freqs for rule '__RCVD_FOREIGN_b4236_c8'/'__RCVD_FOREIGN') (could not find freqs for rule '__FROM_AOL_b4236_c8'/'__FROM_AOL') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_AOL_b4236_c8'/'SPAMMY_FOREIGN_AOL') (could not find freqs for rule '__FROM_RIPE_b4236_c8'/'__FROM_RIPE') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8'/'SPAMMY_FOREIGN_RIPE') (could not find freqs for rule '__FROM_APNIC_b4236_c8'/'__FROM_APNIC') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8'/'SPAMMY_FOREIGN_APNIC') (could not find freqs for rule '__FROM_LACNIC_b4236_c8'/'__FROM_LACNIC') (could not find freqs for rule 'T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8'/'SPAMMY_FOREIGN_LACNIC') above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Thu May 12 01:50:09 2005: __RCVD_APNIC_b4236_c8 = __RCVD_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c8&date=20050512 __RCVD_RIPE_b4236_c8 = __RCVD_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c8&date=20050512 __RCVD_LACNIC_b4236_c8 = __RCVD_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c8&date=20050512 __RCVD_FOREIGN_b4236_c8 = __RCVD_FOREIGN from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c8&date=20050512 __FROM_AOL_b4236_c8 = __FROM_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 = SPAMMY_FOREIGN_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c8&date=20050512 __FROM_RIPE_b4236_c8 = __FROM_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8&date=20050512 __FROM_APNIC_b4236_c8 = __FROM_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8&date=20050512 __FROM_LACNIC_b4236_c8 = __FROM_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c8&date=20050512 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8&date=20050512 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-daf.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-daf.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 494196 370856 123340 0.750 0.00 0.00 (all messages) 100.000 75.0423 24.9577 0.750 0.00 0.00 (all messages as %) bug 4236 cmt 6: ignored, lint failed # [automatically generated by automc: end]
Subject: Re: Check continent of envelope sender against continent of IP -0.3 I think those might strongly penalize ex-patriots, Asian countries, etc. What about two of those in one message? meta FAKE_WORLD_TRAVELER (SPAMMY_FOREIGN_LACNIC + SPAMMY_FOREIGN_RIPE + SPAMMY_FOREIGN_AOL + SPAMMY_FOREIGN_APNIC > 1)
try that again -- NEEDSMC 8
# [automatically generated by automc: start] # DONEMC 12: completed request from comment 12 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_FOREIGN_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 10.00 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 6.00 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 (could not find freqs for rule 'T_MC_FAKE_WORLD_TRAVELER_b4236_c11'/'FAKE_WORLD_TRAVELER') above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Fri May 13 01:47:29 2005: __RCVD_APNIC_b4236_c8 = __RCVD_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c8&date=20050513 __RCVD_RIPE_b4236_c8 = __RCVD_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c8&date=20050513 __RCVD_LACNIC_b4236_c8 = __RCVD_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c8&date=20050513 __RCVD_FOREIGN_b4236_c8 = __RCVD_FOREIGN from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c8&date=20050513 __FROM_AOL_b4236_c8 = __FROM_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 = SPAMMY_FOREIGN_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c8&date=20050513 __FROM_RIPE_b4236_c8 = __FROM_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8&date=20050513 __FROM_APNIC_b4236_c8 = __FROM_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8&date=20050513 __FROM_LACNIC_b4236_c8 = __FROM_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8&date=20050513 T_MC_FAKE_WORLD_TRAVELER_b4236_c11 = FAKE_WORLD_TRAVELER from bug 4236 comment 11 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_FAKE_WORLD_TRAVELER_b4236_c11&date=20050513 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-daf.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-daf.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 493997 370591 123406 0.750 0.00 0.00 (all messages) 100.000 75.0189 24.9811 0.750 0.00 0.00 (all messages as %) bug 4236 cmt 9: ignored, lint failed bug 4236 cmt 10: ignored, lint failed # [automatically generated by automc: end]
# [automatically generated by automc: start] # DONEMC 12: completed request from comment 12 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_FOREIGN_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 10.00 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 6.00 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 (could not find freqs for rule 'T_MC_FAKE_WORLD_TRAVELER_b4236_c11'/'FAKE_WORLD_TRAVELER') above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Fri May 13 01:47:34 2005: __RCVD_APNIC_b4236_c8 = __RCVD_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c8&date=20050513 __RCVD_RIPE_b4236_c8 = __RCVD_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c8&date=20050513 __RCVD_LACNIC_b4236_c8 = __RCVD_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c8&date=20050513 __RCVD_FOREIGN_b4236_c8 = __RCVD_FOREIGN from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c8&date=20050513 __FROM_AOL_b4236_c8 = __FROM_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 = SPAMMY_FOREIGN_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c8&date=20050513 __FROM_RIPE_b4236_c8 = __FROM_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8&date=20050513 __FROM_APNIC_b4236_c8 = __FROM_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8&date=20050513 __FROM_LACNIC_b4236_c8 = __FROM_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8&date=20050513 T_MC_FAKE_WORLD_TRAVELER_b4236_c11 = FAKE_WORLD_TRAVELER from bug 4236 comment 11 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_FAKE_WORLD_TRAVELER_b4236_c11&date=20050513 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-daf.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-daf.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 493997 370591 123406 0.750 0.00 0.00 (all messages) 100.000 75.0189 24.9811 0.750 0.00 0.00 (all messages as %) bug 4236 cmt 9: ignored, lint failed bug 4236 cmt 10: ignored, lint failed # [automatically generated by automc: end]
# [automatically generated by automc: start] # DONEMC 12: completed request from comment 12 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __RCVD_FOREIGN_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 10.00 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 8.00 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 1.00 __FROM_LACNIC_b4236_c8 0.000 0.0000 0.0000 0.500 0.39 6.00 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 (could not find freqs for rule 'T_MC_FAKE_WORLD_TRAVELER_b4236_c11'/'FAKE_WORLD_TRAVELER') above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Fri May 13 01:47:41 2005: __RCVD_APNIC_b4236_c8 = __RCVD_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_APNIC_b4236_c8&date=20050513 __RCVD_RIPE_b4236_c8 = __RCVD_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_RIPE_b4236_c8&date=20050513 __RCVD_LACNIC_b4236_c8 = __RCVD_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_LACNIC_b4236_c8&date=20050513 __RCVD_FOREIGN_b4236_c8 = __RCVD_FOREIGN from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__RCVD_FOREIGN_b4236_c8&date=20050513 __FROM_AOL_b4236_c8 = __FROM_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_AOL_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_AOL_b4236_c8 = SPAMMY_FOREIGN_AOL from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_AOL_b4236_c8&date=20050513 __FROM_RIPE_b4236_c8 = __FROM_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_RIPE_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8 = SPAMMY_FOREIGN_RIPE from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_RIPE_b4236_c8&date=20050513 __FROM_APNIC_b4236_c8 = __FROM_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_APNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8 = SPAMMY_FOREIGN_APNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_APNIC_b4236_c8&date=20050513 __FROM_LACNIC_b4236_c8 = __FROM_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FROM_LACNIC_b4236_c8&date=20050513 T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8 = SPAMMY_FOREIGN_LACNIC from bug 4236 comment 8 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_SPAMMY_FOREIGN_LACNIC_b4236_c8&date=20050513 T_MC_FAKE_WORLD_TRAVELER_b4236_c11 = FAKE_WORLD_TRAVELER from bug 4236 comment 11 full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_FAKE_WORLD_TRAVELER_b4236_c11&date=20050513 # ham results used: ham-bzoetekouw.log ham-cthielen.log ham-daf.log ham-parkerm.log ham-quinlan.log ham-rODbegbie.log ham-theo.log # spam results used: spam-bzoetekouw.log spam-cthielen.log spam-daf.log spam-parkerm.log spam-quinlan.log spam-rODbegbie.log spam-theo.log 493997 370591 123406 0.750 0.00 0.00 (all messages) 100.000 75.0189 24.9811 0.750 0.00 0.00 (all messages as %) bug 4236 cmt 9: ignored, lint failed bug 4236 cmt 10: ignored, lint failed # [automatically generated by automc: end]
Since SA can't have a generic country/continent blacklist in the normal ruleset, I'm tempted to close this as wontfix. Anyone else want to comment?
Yeah, this would be best off as an optional ruleset (via a channel), and since we don't currently have optional rulesets I'd say to close it.
agreed. Plus we have the RelayCountry plugin, which allows simple rules to be written pretty easily for this... see also bug 4509.