Bug 7835 - Domain blacklists domain wildcarding
Summary: Domain blacklists domain wildcarding
Status: NEW
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Libraries (show other bugs)
Version: unspecified
Hardware: All All
: P2 enhancement
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-07 10:23 UTC by Raymond Dijkxhoorn
Modified: 2020-07-09 08:49 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Raymond Dijkxhoorn 2020-07-07 10:23:01 UTC
The current SA libraries dont take into account that both DBL and SURBL provide wildcarded lists. They strip down the domain to the base level. Where this iosnt needed. Due to this the community is missing many listings that are inside both of those lists.

Now we can submit requests to add domains to the util_rb_2tld files but that doesnt really scale and its too slow also. 

For example: 

page.link isnt listed inside SURBL but <abused-subdomain>.page.link is. 
And this is just an example to outline. 

We see that many of the bad actors are abusing free services. Cloudplatforms and such and adding domains to 2/3tld files could work but again way to slow. If you want to take full advantage of the capabilities that SURBL hands to the community it would be far better to not strip down the domains to the base level all the time. the same applies for the DBL list that is also wildcarded. URIBL isnt wildcarded as far as i know but Alex could comment on that. 

If you need more info dont hesitate to mail me. 

With kind regards, Raymond Dijkxhoorn - SURBL
Comment 1 Henrik Krohns 2020-07-09 08:19:06 UTC
Some related talk also found in Bug 7165.

Yes it should be feasable to use a flag for example "tflags SURBL_FOO notrim".

And this could be enabled for all multi.surbl.org queries?
Comment 2 Riccardo Alfieri 2020-07-09 08:32:23 UTC
FWIW, we at Spamhaus support Raymond's request.

Using untrimmed hostnames would provide for sure more spam catching from both SURBL and Spamhaus lists
Comment 3 Raymond Dijkxhoorn 2020-07-09 08:37:54 UTC
(In reply to Henrik Krohns from comment #1)
> Some related talk also found in Bug 7165.
> 
> Yes it should be feasable to use a flag for example "tflags SURBL_FOO
> notrim".
> 
> And this could be enabled for all multi.surbl.org queries?

Yes. All of the multi lookups are wildcarded. 

So it applies to SURBL ABUSE, PH, CR and MW lookups. 

I saw SpamHaus was also added as a watcher.

I am sure Riccardo can comment on the SpamHaus zones to be changed.

Thanks! Raymond
Comment 4 Riccardo Alfieri 2020-07-09 08:49:13 UTC
All lookups to DBL should have the "notrim" flag set. 

ZRD supports them too, but since it's only for DQS customers, I'll take care of adding the necessary changes in our plugin when/if (I really hope this will happen!) there is support in SA