Bug 7649 - FEATURE PATCH: Added output for spamd logs to show relay ip addresses
Summary: FEATURE PATCH: Added output for spamd logs to show relay ip addresses
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: spamc/spamd (show other bugs)
Version: 3.4.1
Hardware: PC Linux
: P2 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-29 07:36 UTC by Evan Booyens
Modified: 2018-10-29 10:54 UTC (History)
3 users (show)


Attachment Type Modified Status Actions Submitter/CLA Status
A patch to add sender IP logging patch None Evan Booyens [NoCLA]
Haraka.pm text/plain None Steve Freegard [NoCLA]
Haraka.cf text/plain None Steve Freegard [NoCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Evan Booyens 2018-10-29 07:36:48 UTC 
--- spamd       2018-10-23 14:27:28.948367854 +0200
+++ ../spamd    2018-10-23 13:17:52.001146322 +0200
@@ -1985,7 +1985,29 @@
   }
 
   my $scantime = sprintf( "%.1f", time - $start_time );
-
+  ############################################################################################ 
+  ## 
+  ## Hack added by to add relay server addresses to base report (can be used by fail2ban, etc)
+  ##  
+  my @from_addrs = $mail->get_pristine_header("Received"); 
+  my $nums = @from_addrs; 
+  my $line; 
+  my @raddrs; 
+  my %seen;
+  my @raddrs_uniq;
+  foreach $line (@from_addrs){ 
+  if($line=~/(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/) { 
+  if($1 == 127 && $2 == 0 && $3 == 0 && $4 == 1) { 
+  }else{ 
+  push(@raddrs, "$1.$2.$3.$4"); 
+  } 
+  } # end if 
+  } # end foreach 
+  %seen = ();
+  @raddrs_uniq = grep { ! $seen{$_} ++ } @raddrs;
+  my $from_addrs2 = join(",",@raddrs_uniq); 
+  
+  ############################################################################################
   info("spamd: $was_it_spam ($msg_score/$msg_threshold) for $current_user:$> in"
        . " $scantime seconds, $actual_length bytes." );
 
@@ -1995,7 +2017,8 @@
             "user=".$current_user, "uid=".$>,
             "required_score=".$msg_threshold,
             "rhost=".$remote_hostname, "raddr=".$remote_hostaddr,
-            "rport=".$remote_port);
+            "rport=".$remote_port,
+            "num_relays=".$nums, "sender_ips=".$from_addrs2);
 
   {
     # no re "strict";  # since perl 5.21.8: Ranges of ASCII printables...
Comment 1 Evan Booyens 2018-10-29 07:39:36 UTC 
Created attachment 5616 [details]
A patch to add sender IP logging

I had a need to log sender ip's as extracting these from 4,000 MTA's was not a practical solution, but 40 spamd servers is far easier.
Comment 2 Henrik Krohns 2018-10-29 07:48:48 UTC 
Doesn't seem like a feature that general population needs.

Atleast there should be config options for it.

Also it doesn't make any sense to parse Received headers again, as SA has already did that and the info is available in internals. Your code parses "anything that looks like IP" from any Received header, which doesn't make much sense. Do you actually want to see everything, or just perhaps the first external relay or all trusted relays etc, otherwise it's just full of faked IPs..
Comment 3 Evan Booyens 2018-10-29 10:09:59 UTC 
(In reply to Henrik Krohns from comment #2)
> Doesn't seem like a feature that general population needs.
> 
> Atleast there should be config options for it.
> 
> Also it doesn't make any sense to parse Received headers again, as SA has
> already did that and the info is available in internals. Your code parses
> "anything that looks like IP" from any Received header, which doesn't make
> much sense. Do you actually want to see everything, or just perhaps the
> first external relay or all trusted relays etc, otherwise it's just full of
> faked IPs..

Hi - thanks for the feedback

I would prefer to extract this info from the internals as it saves on re-parsing the email. Some pointers would be helpful, else I can just troll through the code.
Comment 4 Henrik Krohns 2018-10-29 10:25:30 UTC 
I'm happy to implement it properly for you, as it's pretty simple to do.

The hard part is actually making it generally useful and configurable. So I'd still like to know what IPs you specifically want to look at. All possible IPs from all Received headers? That could be an 'all' option. Or perhaps all the relays are always shown and tagged properly?

relay_ips=internal:1.2.3.4,external:2.3.4.5,untrusted:5.6.7.8
or
relays_internal=1.2.3.4 relays_external=2.3.4.5 relays_untrusted=5.6.7.8

(well it gets complicated, internal are also trusted, but there can be other trusted... untrusted are external.. needs a little bit of thought)

num_relays seems a bit redundant, since it's trivial to calculate? Just bloats the log.

As the report line is purely spamd feature, we also need to think if it's a spamd command line option (I think so) or some local.cf option.
Comment 5 Steve Freegard 2018-10-29 10:35:47 UTC 
Created attachment 5617 [details]
Haraka.pm
Comment 6 Steve Freegard 2018-10-29 10:36:15 UTC 
Created attachment 5618 [details]
Haraka.cf
Comment 7 Steve Freegard 2018-10-29 10:37:09 UTC 
There's already an easy way to add this type of stuff into the spamd log output without any modifications to the spamd code just by using $pms->set_spamd_result_item.

I've attached Haraka.pm and Haraka.cf that we supply as part of the Haraka contrib to add the Haraka UUID (Message ID) and the last external IP address to the spamd logs.

It's pretty straightforward to modify this as per your needs.
Comment 8 Henrik Krohns 2018-10-29 10:47:30 UTC 
Right thanks Steve, I don't much use spamd so it's a bit unfamiliar yet, set_spamd_result_item looks just like what is wanted here.

Simply use these as required to get wanted IPs

$pms->{relays_internal}
$pms->{relays_external}
$pms->{relays_trusted}
$pms->{relays_untrusted}
Comment 9 Evan Booyens 2018-10-29 10:54:07 UTC 
(In reply to Henrik Krohns from comment #2)
> Doesn't seem like a feature that general population needs.
> 
> Atleast there should be config options for it.
> 
> Also it doesn't make any sense to parse Received headers again, as SA has
> already did that and the info is available in internals. Your code parses
> "anything that looks like IP" from any Received header, which doesn't make
> much sense. Do you actually want to see everything, or just perhaps the
> first external relay or all trusted relays etc, otherwise it's just full of
> faked IPs..

Hi - thanks for the feedback

I would prefer to extract this info from the internals as it saves on re-parsing the email. Some pointers would be helpful, else I can just troll through the code.

(In reply to Henrik Krohns from comment #8)
> Right thanks Steve, I don't much use spamd so it's a bit unfamiliar yet,
> set_spamd_result_item looks just like what is wanted here.
> 
> Simply use these as required to get wanted IPs
> 
> $pms->{relays_internal}
> $pms->{relays_external}
> $pms->{relays_trusted}
> $pms->{relays_untrusted}

Thanks for the fast response guys - I have something I can use now without having to patch spamd, so we can close this as a patch request.