Bug 356 - SA 2.21 needs to have rule dependencies (new sort breaks current RBL rules)
Summary: SA 2.21 needs to have rule dependencies (new sort breaks current RBL rules)
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: spamassassin (show other bugs)
Version: SVN Trunk (Latest Devel Version)
Hardware: Other other
: P2 normal
Target Milestone: ---
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-05-27 23:39 UTC by Marc MERLIN
Modified: 2002-06-04 16:03 UTC (History)
0 users



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description Marc MERLIN 2002-05-27 23:39:06 UTC
Ok, so SA 2.21  sorts the rules by score in an attempt  to stop scanning the    
message if the user specified a scan threshold                                  
Unfortunately, this completely breaks rule dependencies. Granted, those are     
mostly (only?) RBL rules for now, but rule dependency is needed somewhere.      
                                                                                
Let me quote 20_head_tests.cf:                                                  
                                                                                
# X prefix is so that these are run after RCVD_IN_*. tests are run in           
# alphanumerically-sorted order.  (These used to be Osirusoft.com-specific, but 
# now, other DNSBLs are using the same convention.)                             
header X_OSIRU_SPAM_SRC         eval:check_rbl_results_for('relay', '127.0.0.4')
describe X_OSIRU_SPAM_SRC       DNSBL: sender is Confirmed Spam Source          
                                                                                
                                                                                
I just so happens  that I spent the last week overhauling  DNS RBL checks so    
that they are  done properly wrt DUL checks and  checks across multiple RBLs.  
                                 
Of course, the new sort code not  only breaks the current behavior, but also    
breaks my patch completely                                                      
                                                                                
For now, I've added the very dirty sort in run_eval_tests to negate the per     
score sorting:                                                                  
                                                                                
sub run_eval_tests {                                                            
  my ($self, $evalhash, $prepend2desc, @extraevalargs) = @_;                    
  my ($rulename, $pat, @args);                                                  
  local ($_);                                                                   
                                                                                
  my @tests = keys %{$evalhash};                                                
  my @negative_tests;                                                           
  my @positive_tests;                                                           
  # add negative tests;                                                         
  foreach my $test (@tests) {                                                   
    if ($self->{conf}{scores}{$test} < 0) {                                     
      push @negative_tests, $test;                                              
    }                                                                           
    else {                                                                      
      push @positive_tests, $test;                                              
    }                                                                           
  }                                                                             
  @negative_tests = sort { $self->{conf}{scores}{$a} <=>                        
$self->{conf}{scores}{$b} } @negative_tests;                                    
  @positive_tests = sort { $self->{conf}{scores}{$b} <=>                        
$self->{conf}{scores}{$a} } @positive_tests;                                    
                                                                                
  foreach my $rulename (sort (@negative_tests, @positive_tests)) {              
                        ^^^^                                                    
                        here                                                    
                                                                                
We need to  work out some way to  add rule dependency if you  really want to    
sort by score                                                                   
                                                                                
For  that matter,  my  other upcoming  patch  (not yet  written  due to  the    
dependency problem)  is supposed  to start  all the DNS  RBL queries  in the    
background at the beginning and read the result after all the rules have run    
or after  10 seconds, whichever comes  last, and ignore the  results if they    
haven't come in time (stuck DNS query)                                          
Any direction  on how  this is  going to  be approached  would help  me with    
writing the patch.                                                              
                                                                                
Thanks,                                                                         
Marc
Comment 1 Matt Sergeant 2002-05-28 04:27:54 UTC
Subject: Re: [SAdev]  New: SA 2.21 needs to have rule dependencies
 (new sort breaks current RBL rules)

bugzilla-daemon@hughes-family.org wrote:
> http://www.hughes-family.org/bugzilla/show_bug.cgi?id=356
> 
>            Summary: SA 2.21 needs to have rule dependencies (new sort breaks
>                     current RBL rules)
>            Product: Spamassassin
>            Version: current-CVS
>           Platform: Other
>         OS/Version: other
>             Status: NEW
>           Severity: normal
>           Priority: P2
>          Component: spamassassin
>         AssignedTo: spamassassin-devel@lists.sourceforge.net
>         ReportedBy: marc_soft@merlins.org
> 
> 
> Ok, so SA 2.21  sorts the rules by score in an attempt  to stop scanning the    
> message if the user specified a scan threshold                                  
> Unfortunately, this completely breaks rule dependencies. Granted, those are     
> mostly (only?) RBL rules for now, but rule dependency is needed somewhere.      

See my post to the users list about this. Just use a simple memoizing 
system, and it makes this go away.

Here's the outline of check_for_yelling now:

sub check_for_yelling {
     my ($self, $body) = @_;

     if (exists $self->{num_yelling_lines}) {
         return $self->{num_yelling_lines} > 0;
     }

     ... # do checks

     $self->{num_yelling_lines} = $num_lines;

     return ($num_lines > 0);
}

and check_for_num_yelling_lines:

sub check_for_num_yelling_lines {
     my ($self, $body, $threshold) = @_;

     $self->check_for_yelling($body);

     return ($self->{num_yelling_lines} >= $threshold);
}

Hope that helps!

Matt.

Comment 2 Marc MERLIN 2002-06-05 00:03:09 UTC
I don't need this anymore, I worked around it by putting RBLs in a separate set
so that they can get run alphabetically