Bug 2185 - full-body pattern fails to test against ALL attachments as documented
Summary: full-body pattern fails to test against ALL attachments as documented
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 2.55
Hardware: Other Linux
: P5 major
Target Milestone: 2.70
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-07-03 05:03 UTC by Anthony Howe
Modified: 2004-01-24 12:51 UTC (History)
0 users



Attachment Type Modified Status Actions Submitter/CLA Status
Maintain MIME attachement headers, but still ignore content. patch None Anthony Howe [NoCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Anthony Howe 2003-07-03 05:03:22 UTC
The documentation claims that a "full" rule will test the whole body including
all attachments:

----
full SYMBOLIC_TEST_NAME /pattern/modifiers
    Define a full-body pattern test. pattern is a Perl regular expression.

The 'full body' of a message is the un-decoded text, including all parts
(including images or other attachments). SpamAssassin no longer tests full tests
against decoded text; use rawbody for that.
----

I've been trying to do some basic anti-virus filtering by using "full" rules.

For example:

full __VIRUS_SOBIG_E1	
/Content-Type:\s*[^\;]+\;\s*name(\*[0-9]+)?=(3D)?\"(your_details|application|document|screensaver|movie)\.zip\"/i
full __VIRUS_SOBIG_E2	
/Content-Type:\s*[^\;]+\;\s*name(\*[0-9]+)?=(3D)?(your_details|application|document|screensaver|movie)\.zip/i
meta VIRUS_SOBIG_E		(__VIRUS_SOBIG_E1 || __VIRUS_SOBIG_E2)
describe VIRUS_SOBIG_E		Virus see
http://securityresponse.symantec.com/avcenter/venc/data/w32.sobig.e@mm.html
score VIRUS_SOBIG_E		20.0

However, while debugging to see WHY my tests were NOT matching known examples, I
found in 

   Mail::SpamAssassin::PerMsgStatus::check

commentary to the contrary and evidence that attachments are not being checked
as expected:

161:        timelog("Starting full message tests", "fullmsgtest", 1);
162         # and do full tests: first with entire, full, undecoded message
163         # still skip application/image attachments though
164         {
165:          my $fulltext = join ('', $self->{msg}->get_all_headers(), "\n",
  DB<3> l
166==>                                  @{$self->get_raw_body_text_array()});
167:          $self->do_full_tests(\$fulltext);
168:          $self->do_full_eval_tests(\$fulltext);
169:          undef $fulltext;
170         }
171:        timelog("Finished full message tests", "fullmsgtest", 2);

...

167:          $self->do_full_tests(\$fulltext);
  DB<3> p $fulltext
Return-Path: <abcwallpaper@caramail.com>
Received: from mx1.atsat.com ([193.41.75.85]) by pop1.atsat.com
          (Netscape Messaging Server 3.5)  with ESMTP id 479
          for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:38:11 +0200
Received: from pop1.atsat.com (pop1.atsat.com [193.41.75.75])
        by mx1.atsat.com (8.12.9/8.12.9) with ESMTP id h638Q5ND029296
        for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:26:05 +0200
Received: from mx1.atsat.com ([193.41.75.85]) by pop1.atsat.com
          (Netscape Messaging Server 3.5)  with ESMTP id 606
          for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:38:09 +0200
Received: from JB (Mix-Avignon-206-4-176.w193-249.abo.wanadoo.fr [193.249.255.176])
        by mx1.atsat.com (8.12.9/8.12.9) with SMTP id h638PBND029161
        for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:25:14 +0200
Message-Id: <200307030825.h638PBND029161@mx1.atsat.com>
From: <abcwallpaper@caramail.com>
To: <phane@azur-routage.com>
Subject: [SPAM] Re: Application
Date: Thu, 3 Jul 2003 10:38:51 +0200
Importance: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: multipart/mixed;
        boundary="CSmtpMsgPart123X456_000_00829DC4"

This is a multipart message in MIME format

--CSmtpMsgPart123X456_000_00829DC4
Content-Type: text/plain;
        charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Please see the attached zip file for details.
--CSmtpMsgPart123X456_000_00829DC4
[skipped application/x-zip-compressed attachment]
Comment 1 Anthony Howe 2003-07-03 05:27:49 UTC
Created attachment 1127 [details]
Maintain MIME attachement headers, but still ignore content.

This proposed patch would allow for more interesting rules useful in some basic
virus filtering on MIME part headers, without actually including the entire
attachment body. Including the MIME part headers will not in case memory use
that much vs including the whole attachment body.

For example:

  DB<4> p $fulltext
Return-Path: <abcwallpaper@caramail.com>
Received: from mx1.atsat.com ([193.41.75.85]) by pop1.atsat.com
	  (Netscape Messaging Server 3.5)  with ESMTP id 479
	  for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:38:11 +0200
Received: from pop1.atsat.com (pop1.atsat.com [193.41.75.75])
	by mx1.atsat.com (8.12.9/8.12.9) with ESMTP id h638Q5ND029296
	for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:26:05 +0200
Received: from mx1.atsat.com ([193.41.75.85]) by pop1.atsat.com
	  (Netscape Messaging Server 3.5)  with ESMTP id 606
	  for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:38:09 +0200
Received: from JB (Mix-Avignon-206-4-176.w193-249.abo.wanadoo.fr
[193.249.255.176])
	by mx1.atsat.com (8.12.9/8.12.9) with SMTP id h638PBND029161
	for <phane@azur-routage.com>; Thu, 3 Jul 2003 10:25:14 +0200
Message-Id: <200307030825.h638PBND029161@mx1.atsat.com>
From: <abcwallpaper@caramail.com>
To: <phane@azur-routage.com>
Subject: [SPAM] Re: Application
Date: Thu, 3 Jul 2003 10:38:51 +0200
Importance: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="CSmtpMsgPart123X456_000_00829DC4"

This is a multipart message in MIME format

--CSmtpMsgPart123X456_000_00829DC4
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Please see the attached zip file for details.
--CSmtpMsgPart123X456_000_00829DC4
Content-Type: application/x-zip-compressed;
	name="your_details.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="your_details.zi"

[skipped application/x-zip-compressed content]
Comment 2 Anthony Howe 2003-07-03 22:27:03 UTC
Forgot to note the version is 2.55
Comment 3 Theo Van Dinter 2004-01-24 21:51:43 UTC
in 2.70 right now, full will use the pristine body.