5171 – [review] sa-update lint checks fail due to differing plugins

Bug 5171 - [review] sa-update lint checks fail due to differing plugins

Summary: [review] sa-update lint checks fail due to differing plugins

Status:	RESOLVED FIXED

Alias:	None

Product:	Spamassassin
Classification:	Unclassified
Component:	sa-update (show other bugs)
Version:	SVN Trunk (Latest Devel Version)
Hardware:	Other other

Importance:	P1 normal
Target Milestone:	3.1.8
Assignee:	SpamAssassin Developer Mailing List

URL:
Whiteboard:	ready to commit
Keywords:

Depends on:	5454
Blocks:
	Show dependency tree

Reported:	2006-11-08 08:35 UTC by Justin Mason
Modified:	2007-05-10 03:51 UTC (History)
CC List:	0 users

Attachment	Type	Modified	Status	Actions	Submitter/CLA Status
3.1 patch to use site pre files in sa-update lint checks	patch			None	Daryl C. W. O'Shea
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Justin Mason 2006-11-08 08:35:00 UTC

Henry noted this yesterday; I just ran into it myself.   Here's the log:

 12...;   sudo sa-update --debug && sudo /etc/init.d/spamassassin reload
Password:
[30726] dbg: logger: adding facilities: all
[30726] dbg: logger: logging level is DBG
[30726] dbg: generic: SpamAssassin version 3.2.0-r441887
[30726] dbg: config: score set 0 chosen.
[30726] dbg: dns: no ipv6
[30726] dbg: dns: is Net::DNS::Resolver available? yes
[30726] dbg: dns: Net::DNS version: 0.48
[30726] dbg: generic: sa-update version svn454082
[30726] dbg: generic: using update directory: /var/lib/spamassassin/3.002000
[30726] dbg: diag: perl platform: 5.008004 linux
[30726] dbg: diag: module installed: Digest::SHA1, version 2.10
[30726] dbg: diag: module installed: HTML::Parser, version 3.45
[30726] dbg: diag: module installed: MIME::Base64, version 3.04
[30726] dbg: diag: module installed: DB_File, version 1.808
[30726] dbg: diag: module installed: Net::DNS, version 0.48
[30726] dbg: diag: module installed: Net::SMTP, version 2.29
[30726] dbg: diag: module installed: Mail::SPF::Query, version 1.997
[30726] dbg: diag: module not installed: IP::Country::Fast ('require' failed)
[30726] dbg: diag: module not installed: Razor2::Client::Agent ('require' failed)
[30726] dbg: diag: module installed: Net::Ident, version 1.20
[30726] dbg: diag: module not installed: IO::Socket::INET6 ('require' failed)
[30726] dbg: diag: module installed: IO::Socket::SSL, version 0.96
[30726] dbg: diag: module installed: Time::HiRes, version 1.59
[30726] dbg: diag: module installed: DBI, version 1.46
[30726] dbg: diag: module installed: Getopt::Long, version 2.34
[30726] dbg: diag: module installed: LWP::UserAgent, version 2.033
[30726] dbg: diag: module installed: HTTP::Date, version 1.46
[30726] dbg: diag: module installed: Archive::Tar, version 1.23
[30726] dbg: diag: module installed: IO::Zlib, version 1.04
[30726] dbg: diag: module not installed: Encode::Detect ('require' failed)
[30726] dbg: gpg: Searching for 'gpg'
[30726] dbg: util: current PATH is:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin
[30726] dbg: util: executable for gpg was found at /usr/bin/gpg
[30726] dbg: gpg: found /usr/bin/gpg
[30726] dbg: gpg: release trusted key id list:
5E541DC959CB8BAC7C78DFDC4056A61A5244EC45
26C900A46DD40CD5AD24F6D7DEE01987265FA05B 0C2B1D7175B852C64B3CDC716C55397824F434CE
[30726] dbg: channel: attempting channel updates.spamassassin.org
[30726] dbg: channel: update directory
/var/lib/spamassassin/3.002000/updates_spamassassin_org
[30726] dbg: channel: channel cf file
/var/lib/spamassassin/3.002000/updates_spamassassin_org.cf
[30726] dbg: channel: channel pre file
/var/lib/spamassassin/3.002000/updates_spamassassin_org.pre
[30726] dbg: dns: 0.2.3.updates.spamassassin.org => 472035, parsed as 472035
[30726] dbg: channel: no MIRRORED.BY file available
[30726] dbg: http: GET request, http://spamassassin.apache.org/updates/MIRRORED.BY
[30726] dbg: channel: MIRRORED.BY file retrieved
[30726] dbg: channel: reading MIRRORED.BY file
[30726] dbg: channel: found mirror http://spamassassin.kluge.net/updates/
[30726] dbg: channel: selected mirror http://spamassassin.kluge.net/updates
[30726] dbg: http: GET request, http://spamassassin.kluge.net/updates/472035.tar.gz
[30726] dbg: http: GET request,
http://spamassassin.kluge.net/updates/472035.tar.gz.sha1
[30726] dbg: http: GET request,
http://spamassassin.kluge.net/updates/472035.tar.gz.asc
[30726] dbg: sha1: verification wanted: 9367ad3a12df7be1091fbe5fc33ca7c2d41566b8
[30726] dbg: sha1: verification result: 9367ad3a12df7be1091fbe5fc33ca7c2d41566b8
[30726] dbg: channel: populating temp content file
[30726] dbg: gpg: populating temp signature file
[30726] dbg: gpg: calling gpg
[30726] dbg: gpg: gpg: Signature made Wed Nov 8 08:50:17 2006 GMT using RSA key
ID 24F434CE
[30726] dbg: gpg: [GNUPG:] SIG_ID miA0tspwNRPvuv1aabQVCNMCk5k 2006-11-08 1162975817
[30726] dbg: gpg: [GNUPG:] GOODSIG 6C55397824F434CE updates.spamassassin.org
Signing Key <release@spamassassin.org>
[30726] dbg: gpg: gpg: Good signature from "updates.spamassassin.org Signing Key
<release@spamassassin.org>"
[30726] dbg: gpg: [GNUPG:] VALIDSIG 0C2B1D7175B852C64B3CDC716C55397824F434CE
2006-11-08 1162975817 0 3 0 1 2 00 5E541DC959CB8BAC7C78DFDC4056A61A5244EC45
[30726] dbg: gpg: [GNUPG:] TRUST_UNDEFINED
[30726] dbg: gpg: gpg: WARNING: This key is not certified with a trusted signature!
[30726] dbg: gpg: gpg: There is no indication that the signature belongs to the
owner.
[30726] dbg: gpg: Primary key fingerprint: 5E54 1DC9 59CB 8BAC 7C78 DFDC 4056
A61A 5244 EC45
[30726] dbg: gpg: Subkey fingerprint: 0C2B 1D71 75B8 52C6 4B3C DC71 6C55 3978
24F4 34CE
[30726] dbg: gpg: found signature made by key
0C2B1D7175B852C64B3CDC716C55397824F434CE
[30726] dbg: gpg: key id 0C2B1D7175B852C64B3CDC716C55397824F434CE is release trusted
[30726] dbg: channel: file verification passed, testing update
[30726] dbg: channel: preparing temp directory for new channel
[30726] dbg: generic: update tmp directory /tmp/.spamassassin30726sun7b2tmp
[30726] dbg: channel: extracting archive
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/10_default_prefs.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_advance_fee.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_body_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_compensate.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_dnsbl_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_drugs.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_fake_helo_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_head_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_html_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_meta_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_net_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_phrases.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_porn.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_ratware.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/20_uri_tests.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/23_bayes.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_accessdb.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_antivirus.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_body_tests_pl.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_dcc.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_dkim.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_domainkeys.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_hashcash.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_pyzor.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_razor2.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_replace.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_spf.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_textcat.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/25_uribl.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_de.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_fr.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_it.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_nl.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_pl.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/30_text_pt_br.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/50_scores.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_awl.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_shortcircuit.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_whitelist.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_whitelist_dk.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_whitelist_dkim.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_whitelist_spf.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/60_whitelist_subject.cf
[30726] dbg: extracting: /tmp/.spamassassin30726sun7b2tmp/72_active.cf
[30726] dbg: generic: SpamAssassin version 3.2.0-r441887
[30726] dbg: config: score set 0 chosen.
[30726] dbg: dns: no ipv6
[30726] dbg: dns: is Net::DNS::Resolver available? yes
[30726] dbg: dns: Net::DNS version: 0.48
[30726] dbg: ignore: using a test message to lint rules
[30726] dbg: config: using "/tmp/.spamassassin30726sun7b2tmp/doesnotexist" for
site rules pre files
[30726] dbg: config: using "/tmp/.spamassassin30726sun7b2tmp" for sys rules pre
files
[30726] dbg: config: using "/tmp/.spamassassin30726sun7b2tmp" for default rules dir
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/10_default_prefs.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_advance_fee.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_body_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_compensate.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_dnsbl_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_drugs.cf
[30726] dbg: config: read file
/tmp/.spamassassin30726sun7b2tmp/20_fake_helo_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_head_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_html_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_meta_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_net_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_phrases.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_porn.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_ratware.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/20_uri_tests.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/23_bayes.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_accessdb.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_antivirus.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_body_tests_pl.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_dcc.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_dkim.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_domainkeys.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_hashcash.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_pyzor.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_razor2.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_replace.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_spf.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_textcat.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/25_uribl.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_de.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_fr.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_it.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_nl.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_pl.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/30_text_pt_br.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/50_scores.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_awl.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_shortcircuit.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_whitelist.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_whitelist_dk.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_whitelist_dkim.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/60_whitelist_spf.cf
[30726] dbg: config: read file
/tmp/.spamassassin30726sun7b2tmp/60_whitelist_subject.cf
[30726] dbg: config: read file /tmp/.spamassassin30726sun7b2tmp/72_active.cf
[30726] dbg: config: using "/tmp/.spamassassin30726sun7b2tmp/doesnotexist" for
site rules dir
[30726] dbg: config: using "/tmp/.spamassassin30726sun7b2tmp/doesnotexist" for
user prefs file
config: warning: score set for non-existent rule UNRESOLVED_TEMPLATE

config: warning: score set for non-existent rule MIME_BASE64_TEXT

config: warning: score set for non-existent rule SUBJ_ILLEGAL_CHARS

config: warning: score set for non-existent rule RATWARE_NAME_ID

config: warning: score set for non-existent rule DATE_IN_PAST_03_06

config: warning: score set for non-existent rule MIME_QP_LONG_LINE

config: warning: score set for non-existent rule DATE_IN_FUTURE_96_XX

config: warning: score set for non-existent rule FORGED_HOTMAIL_RCVD2

config: warning: score set for non-existent rule MIME_BASE64_BLANKS

config: warning: score set for non-existent rule DATE_IN_FUTURE_48_96

config: warning: score set for non-existent rule MSGID_OUTLOOK_INVALID

config: warning: score set for non-existent rule FAKE_HELO_YAHOO_CA

config: warning: score set for non-existent rule MIME_HTML_MOSTLY

config: warning: score set for non-existent rule SUBJ_ALL_CAPS

config: warning: score set for non-existent rule RCVD_HELO_IP_MISMATCH

config: warning: score set for non-existent rule DATE_IN_FUTURE_06_12

config: warning: score set for non-existent rule LOCALPART_IN_SUBJECT

config: warning: score set for non-existent rule FORGED_GW05_RCVD

config: warning: score set for non-existent rule RCVD_NUMERIC_HELO

config: warning: score set for non-existent rule ALL_TRUSTED

config: warning: score set for non-existent rule MIME_HTML_ONLY

config: warning: score set for non-existent rule CHARSET_FARAWAY

config: warning: score set for non-existent rule DATE_IN_FUTURE_24_48

config: warning: score set for non-existent rule SORTED_RECIPS

config: warning: score set for non-existent rule MISSING_HEADERS

config: warning: score set for non-existent rule FAKE_HELO_MAIL_COM

config: warning: score set for non-existent rule MIME_BAD_ISO_CHARSET

config: warning: score set for non-existent rule FAKE_HELO_MSN

config: warning: score set for non-existent rule DATE_IN_PAST_96_XX

config: warning: score set for non-existent rule FROM_ILLEGAL_CHARS

config: warning: score set for non-existent rule DATE_IN_PAST_06_12

config: warning: score set for non-existent rule NO_RDNS_DOTCOM_HELO

config: warning: score set for non-existent rule DATE_IN_PAST_12_24

config: warning: score set for non-existent rule FAKE_HELO_EXCITE

config: warning: score set for non-existent rule ROUND_THE_WORLD_LOCAL

config: warning: score set for non-existent rule NO_RELAYS

config: warning: score set for non-existent rule DATE_IN_FUTURE_12_24

config: warning: score set for non-existent rule MPART_ALT_DIFF_COUNT

config: warning: score set for non-existent rule HTTPS_IP_MISMATCH

config: warning: score set for non-existent rule HEADER_COUNT_CTYPE

config: warning: score set for non-existent rule BLANK_LINES_80_90

config: warning: score set for non-existent rule DATE_IN_FUTURE_03_06

config: warning: score set for non-existent rule SUSPICIOUS_RECIPS

config: warning: score set for non-existent rule FORGED_YAHOO_RCVD

config: warning: score set for non-existent rule MISSING_HB_SEP

config: warning: score set for non-existent rule FAKE_HELO_EMAIL_COM

config: warning: score set for non-existent rule MPART_ALT_DIFF

config: warning: score set for non-existent rule DATE_IN_PAST_24_48

config: warning: score set for non-existent rule RATWARE_EFROM

config: warning: score set for non-existent rule CHARSET_FARAWAY_HEADER

config: warning: score set for non-existent rule FAKE_HELO_LYCOS

config: warning: score set for non-existent rule MISSING_MIME_HB_SEP

config: warning: score set for non-existent rule RCVD_ILLEGAL_IP

config: warning: score set for non-existent rule HEAD_ILLEGAL_CHARS

config: warning: score set for non-existent rule UNPARSEABLE_RELAY

[30726] dbg: conf: finish parsing
[30726] dbg: config: score set 0 chosen.
[30726] dbg: message: main message type: text/plain
[30726] dbg: message: ---- MIME PARSER START ----
[30726] dbg: message: parsing normal part
[30726] dbg: message: ---- MIME PARSER END ----
[30726] dbg: metadata: X-Spam-Relays-Trusted:
[30726] dbg: metadata: X-Spam-Relays-Untrusted:
[30726] dbg: metadata: X-Spam-Relays-Internal:
[30726] dbg: metadata: X-Spam-Relays-External:
[30726] dbg: message: no encoding detected
[30726] dbg: rules: local tests only, ignoring RBL eval
[30726] dbg: check: running tests for priority: 0
[30726] dbg: rules: running header regexp tests; score so far=0
[30726] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit: "@lint_rules>"
[30726] dbg: rules: ran header rule __MSGID_OK_DIGITS ======> got hit: "1163003081"
[30726] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
[30726] dbg: rules: ran header rule __SANE_MSGID ======> got hit:
"<1163003081@lint_rules>
[30726] dbg: rules: "
[30726] dbg: rules: ran header rule MISSING_DATE ======> got hit: "UNSET"
rules: failed to run HEAD_LONG test, skipping:
        (Can't locate object method "check_msg_parse_flags" via package
"Mail::SpamAssassin::PerMsgStatus" at (eval 336) line 16.
)
[30726] dbg: rules: running body-text per-line regexp tests; score so far=0.001
[30726] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "I"
[30726] dbg: uri: running uri tests; score so far=0.001
[30726] dbg: rules: running raw-body-text per-line regexp tests; score so far=0.001
[30726] dbg: rules: running full-text regexp tests; score so far=0.001
[30726] dbg: rules: running meta tests; score so far=0.001
[30726] dbg: check: running tests for priority: 500
[30726] dbg: rules: running header regexp tests; score so far=0.001
[30726] dbg: rules: running body-text per-line regexp tests; score so far=0.001
[30726] dbg: uri: running uri tests; score so far=0.001
[30726] dbg: rules: running raw-body-text per-line regexp tests; score so far=0.001
[30726] dbg: rules: running full-text regexp tests; score so far=0.001
[30726] dbg: rules: running meta tests; score so far=0.001
[30726] info: rules: meta test HTML_SHORT_CENTER has undefined dependency
'__HTML_LENGTH_384'
[30726] info: rules: meta test HTML_SHORT_CENTER has undefined dependency
'__TAG_EXISTS_CENTER'
[30726] info: rules: meta test FORGED_AOL_TAGS has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test FORGED_AOL_TAGS has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test DIGEST_MULTIPLE has undefined dependency
'RAZOR2_CHECK'
[30726] info: rules: meta test DIGEST_MULTIPLE has undefined dependency 'DCC_CHECK'
[30726] info: rules: meta test DIGEST_MULTIPLE has undefined dependency
'PYZOR_CHECK'
[30726] info: rules: meta test HTML_MIME_NO_HTML_TAG has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test TVD_FW_GRAPHIC_ID3 has undefined dependency
'__TVD_OUTLOOK_IMG'
[30726] info: rules: meta test TVD_FW_GRAPHIC_ID3 has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test EMPTY_MESSAGE has undefined dependency
'__MIME_ATTACHMENT'
[30726] info: rules: meta test UPPERCASE_75_100 has undefined dependency
'__UPPERCASE_75_100'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_HEAD'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_META'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_BODY'
[30726] info: rules: meta test FORGED_QUALCOMM_TAGS has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test FORGED_QUALCOMM_TAGS has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test MULTI_FORGED has undefined dependency
'__FORGED_AOL_RCVD'
[30726] info: rules: meta test MULTI_FORGED has undefined dependency
'__FORGED_HOTMAIL_RCVD'
[30726] info: rules: meta test MULTI_FORGED has undefined dependency
'__FORGED_EUDORAMAIL_RCVD'
[30726] info: rules: meta test MULTI_FORGED has undefined dependency
'__FORGED_JUNO_RCVD'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_3 has undefined dependency
'__HTML_LENGTH_1536_2048'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_3 has undefined dependency
'__HTML_LINK_IMAGE'
[30726] info: rules: meta test CONFIRMED_FORGED has undefined dependency
'__FORGED_RCVD_TRAIL'
[30726] info: rules: meta test CONFIRMED_FORGED has undefined dependency
'__FORGED_AOL_RCVD'
[30726] info: rules: meta test CONFIRMED_FORGED has undefined dependency
'__FORGED_HOTMAIL_RCVD'
[30726] info: rules: meta test CONFIRMED_FORGED has undefined dependency
'__FORGED_EUDORAMAIL_RCVD'
[30726] info: rules: meta test CONFIRMED_FORGED has undefined dependency
'__FORGED_JUNO_RCVD'
[30726] info: rules: meta test MIME_CHARSET_FARAWAY has undefined dependency
'__MIME_CHARSET_FARAWAY'
[30726] info: rules: meta test OBFUSCATING_COMMENT has undefined dependency
'HTML_MESSAGE'
[30726] info: rules: meta test HTML_MISSING_CTYPE has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test HTML_MISSING_CTYPE has undefined dependency
'HTML_MESSAGE'
[30726] info: rules: meta test UPPERCASE_50_75 has undefined dependency
'__UPPERCASE_50_75'
[30726] info: rules: meta test TVD_PH_FR5 has undefined dependency
'__ENV_AND_HDR_FROM_MATCH'
[30726] info: rules: meta test FORGED_MUA_IMS has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test FORGED_MUA_MOZILLA has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_1 has undefined dependency
'__HTML_LENGTH_0000_1024'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_1 has undefined dependency
'__HTML_LINK_IMAGE'
[30726] info: rules: meta test SHORT_HELO_AND_INLINE_IMAGE has undefined
dependency '__PART_STOCK_IMG'
[30726] info: rules: meta test DYN_RDNS_AND_INLINE_IMAGE has undefined
dependency '__PART_STOCK_IMG'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_2 has undefined dependency
'__HTML_LENGTH_1024_1536'
[30726] info: rules: meta test HTML_SHORT_LINK_IMG_2 has undefined dependency
'__HTML_LINK_IMAGE'
[30726] info: rules: meta test DYN_RDNS_SHORT_HELO_FROM has undefined dependency
'__ENV_AND_HDR_FROM_MATCH'
[30726] info: rules: meta test __FORGED_OE has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test HTML_CHARSET_FARAWAY has undefined dependency
'__HTML_CHARSET_FARAWAY'
[30726] info: rules: meta test TVD_FW_GRAPHIC_ID3_2 has undefined dependency
'__TVD_OUTLOOK_IMG'
[30726] info: rules: meta test TVD_FW_GRAPHIC_ID3_2 has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test DYN_RDNS_SHORT_HELO_HTML has undefined dependency
'HTML_MESSAGE'
[30726] info: rules: meta test __FORGED_OUTLOOK_DOLLARS has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test FORGED_IMS_TAGS has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test FORGED_IMS_TAGS has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test FORGED_IMS_TAGS has undefined dependency
'__TAG_EXISTS_HEAD'
[30726] info: rules: meta test FORGED_IMS_TAGS has undefined dependency
'__TAG_EXISTS_META'
[30726] info: rules: meta test FORGED_IMS_TAGS has undefined dependency
'__TAG_EXISTS_BODY'
[30726] info: rules: meta test HTML_TITLE_SUBJ_DIFF has undefined dependency
'__HTML_TITLE_SUBJ_DIFF'
[30726] info: rules: meta test HTML_TITLE_SUBJ_DIFF has undefined dependency
'__MIME_ATTACHMENT'
[30726] info: rules: meta test FORGED_MUA_OIMO has undefined dependency
'__UNUSABLE_MSGID'
[30726] info: rules: meta test FORGED_MUA_EUDORA has undefined dependency
'__UNUSABLE_MSGID'
[30726] dbg: check: is spam? score=1.729 required=5
[30726] dbg: check: tests=MISSING_DATE,MISSING_SUBJECT,NO_RECEIVED
[30726] dbg: check:
subtests=__HAS_MSGID,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID
channel: lint check of update failed, channel failed
[30726] dbg: generic: cleaning up temporary directory/files
[30726] dbg: generic: unlinking 10_default_prefs.cf
[30726] dbg: generic: unlinking 20_advance_fee.cf
[30726] dbg: generic: unlinking 20_body_tests.cf
[30726] dbg: generic: unlinking 20_compensate.cf
[30726] dbg: generic: unlinking 20_dnsbl_tests.cf
[30726] dbg: generic: unlinking 20_drugs.cf
[30726] dbg: generic: unlinking 20_fake_helo_tests.cf
[30726] dbg: generic: unlinking 20_head_tests.cf
[30726] dbg: generic: unlinking 20_html_tests.cf
[30726] dbg: generic: unlinking 20_meta_tests.cf
[30726] dbg: generic: unlinking 20_net_tests.cf
[30726] dbg: generic: unlinking 20_phrases.cf
[30726] dbg: generic: unlinking 20_porn.cf
[30726] dbg: generic: unlinking 20_ratware.cf
[30726] dbg: generic: unlinking 20_uri_tests.cf
[30726] dbg: generic: unlinking 23_bayes.cf
[30726] dbg: generic: unlinking 25_accessdb.cf
[30726] dbg: generic: unlinking 25_antivirus.cf
[30726] dbg: generic: unlinking 25_body_tests_pl.cf
[30726] dbg: generic: unlinking 25_dcc.cf
[30726] dbg: generic: unlinking 25_dkim.cf
[30726] dbg: generic: unlinking 25_domainkeys.cf
[30726] dbg: generic: unlinking 25_hashcash.cf
[30726] dbg: generic: unlinking 25_pyzor.cf
[30726] dbg: generic: unlinking 25_razor2.cf
[30726] dbg: generic: unlinking 25_replace.cf
[30726] dbg: generic: unlinking 25_spf.cf
[30726] dbg: generic: unlinking 25_textcat.cf
[30726] dbg: generic: unlinking 25_uribl.cf
[30726] dbg: generic: unlinking 30_text_de.cf
[30726] dbg: generic: unlinking 30_text_fr.cf
[30726] dbg: generic: unlinking 30_text_it.cf
[30726] dbg: generic: unlinking 30_text_nl.cf
[30726] dbg: generic: unlinking 30_text_pl.cf
[30726] dbg: generic: unlinking 30_text_pt_br.cf
[30726] dbg: generic: unlinking 50_scores.cf
[30726] dbg: generic: unlinking 60_awl.cf
[30726] dbg: generic: unlinking 60_shortcircuit.cf
[30726] dbg: generic: unlinking 60_whitelist.cf
[30726] dbg: generic: unlinking 60_whitelist_dk.cf
[30726] dbg: generic: unlinking 60_whitelist_dkim.cf
[30726] dbg: generic: unlinking 60_whitelist_spf.cf
[30726] dbg: generic: unlinking 60_whitelist_subject.cf
[30726] dbg: generic: unlinking 72_active.cf
[30726] dbg: diag: updates complete, exiting with code 4



The cause: quoting from [update]/20_body_tests.cf:

ifplugin Mail::SpamAssassin::Plugin::MIMEEval
...
rawbody MIME_BASE64_BLANKS      eval:check_for_mime('mime_base64_blanks')
describe MIME_BASE64_BLANKS     Extra blank lines in base64 encoding
...
endif

: jm 289...; grep MIMEEval update472035/*
update472035/20_body_tests.cf:ifplugin Mail::SpamAssassin::Plugin::MIMEEval
update472035/20_head_tests.cf:ifplugin Mail::SpamAssassin::Plugin::MIMEEval
update472035/20_html_tests.cf:ifplugin Mail::SpamAssassin::Plugin::MIMEEval
update472035/20_meta_tests.cf:ifplugin Mail::SpamAssassin::Plugin::MIMEEval
update472035/20_ratware.cf:ifplugin Mail::SpamAssassin::Plugin::MIMEEval


It looks like the update *does* need to allow loadplugin lines, after all,
otherwise (a) half of the scoreset that has been moved into plugins stops
working, causing (b) lint failures.   At least, *some* loadplugin lines --
perhaps the contents of the v320.pre file, or using a whitelist in the
update-generation backend.

Comment 1 Theo Van Dinter 2006-11-08 08:44:49 UTC

This isn't a code problem, it's a rules/update problem.  "score" lines for
plugin tests need to be in ifplugin conditional sections.

Comment 2 Justin Mason 2006-11-08 08:56:17 UTC

ok, think I have this fixed.  

svn commit -m "bug 5171: allow rules/*.pre files to be added to rules updates;
otherwise none of the core eval-rule ruleset are defined, causing lots of lint
failures"  build/mkupdates/run_part2
Sending        build/mkupdates/run_part2
Transmitting file data .
Committed revision 472539.

let's see how the updates turn out.  

did sa-update *used* to allow .pre files?

Comment 3 Justin Mason 2006-11-08 08:59:14 UTC

> This isn't a code problem, it's a rules/update problem.  "score" lines for
> plugin tests need to be in ifplugin conditional sections.

those rules are coming from these plugins:

# Plugins which used to be EvalTests.pm
# broken out into separate plugins
loadplugin Mail::SpamAssassin::Plugin::Bayes
loadplugin Mail::SpamAssassin::Plugin::BodyEval
loadplugin Mail::SpamAssassin::Plugin::DNSEval
loadplugin Mail::SpamAssassin::Plugin::HTMLEval
loadplugin Mail::SpamAssassin::Plugin::HeaderEval
loadplugin Mail::SpamAssassin::Plugin::MIMEEval
loadplugin Mail::SpamAssassin::Plugin::RelayEval
loadplugin Mail::SpamAssassin::Plugin::URIEval
loadplugin Mail::SpamAssassin::Plugin::WLBLEval

in other words, the entire set of eval rules built into the core ruleset.
also, it's not just score lines -- it's descriptions and meta rules too.

It would be a serious pain, both to implement now and to maintain later, to
require "ifplugin" for all score lines, descriptions, and meta rules that refer
to the core eval-rules set.  I'd be -1 on such a requirement.

Comment 4 Theo Van Dinter 2006-11-08 09:38:28 UTC

(In reply to comment #3)
> > This isn't a code problem, it's a rules/update problem.  "score" lines for
> > plugin tests need to be in ifplugin conditional sections.
> 
> those rules are coming from these plugins:
> in other words, the entire set of eval rules built into the core ruleset.
> also, it's not just score lines -- it's descriptions and meta rules too.

meta rules won't throw an error, but score and description lines will.

> It would be a serious pain, both to implement now and to maintain later, to
> require "ifplugin" for all score lines, descriptions, and meta rules that refer
> to the core eval-rules set.  I'd be -1 on such a requirement.

Then essentially what you want is all the rule plugins to be standardly loaded
modules again which is what we've been trying to get away from...  The general
rule for using plugins is that all related configs *have* to be in an ifplugin
conditional.  Even if you load all the pre files, the user can comment out
plugins, so you'll be back to where you started.

Comment 5 Justin Mason 2006-11-09 10:21:47 UTC

> meta rules won't throw an error, but score and description lines will.

well, sure, it's not a *fatal* error.  but still, this:

[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__MIME_HTML'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_HTML'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_HEAD'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_META'
[30726] info: rules: meta test FORGED_OUTLOOK_TAGS has undefined dependency
'__TAG_EXISTS_BODY'

is pretty ugly, and I'd still consider it some class of an error ;)

The only workaround I could see would be to "ifplugin
Mail::SpamAssassin::Plugin::HTMLEval" every single meta rule that used one of
those rules; then ifplugin their score lines and descriptions; then ifplugin
meta rules using them in turn etc. etc.

> Then essentially what you want is all the rule plugins to be standardly loaded
> modules again which is what we've been trying to get away from...  The general
> rule for using plugins is that all related configs *have* to be in an ifplugin
> conditional.  Even if you load all the pre files, the user can comment out
> plugins, so you'll be back to where you started.

Sure.  But there's a limit to how deep it can go.  I think it's fine to
"ifplugin" stuff like DKIM, DCC, Hashcash etc -- there are small, independent
rulesets there.  In fact, in my opinion Bayes would be an arguably good
addition, too.  But the rules moved from EvalTests.pm -- they're still core
in a way.

I know I argued that they *should* be made into plugins -- I agree it was a
good idea.  However, my use-case scenario I argued this from, was for somebody
who's using SpamAssassin with their own, entirely built-from-scratch ruleset;
ie. someone who didn't want to use any of SpamAssassin's builtin rules. (I know
of quite a few groups doing this for various reasons.) In other words I don't
think it is a good idea to go from saying "they should be plugins" to making
rules like "and since any plugin can be safely disabled without error, any
lines referring to them require an 'ifplugin' scope".  Some plugins have to be
considered "core" requirements of the default ruleset.

(by the way it's not just those rule plugins that this applies to --
the new Check plugin is similarly a "core" requirement.)

I'd be very happy to add caveats and warnings to the comments of the .pre
files noting where this is the case, btw.

Comment 6 Daryl C. W. O'Shea 2006-11-16 19:10:17 UTC

It'd be much better to load the site's pre files (for the channel lint check),
than including the pre files in the update and ending up with the "core plugins"
all being loaded twice after the update (once from the site pre dir and once
from the update dir).

This will work for both of the current arguments:

 - core plugin rules don't need ifplugin lines because (supposedly) people using
   these updates will have all the core plugins enabled in their site pre files
   so the channel lint will succeed

 - core plugin rules do need ifplugin lines... these rules will now be linted
   if the user has the plugin enabled in their site pre dir


So, I think everyone would be OK with this.  Implemented in r476035.


Myself, I'd prefer that the core rules all had ifplugin lines.  Past the
immediate clean up work (which I volunteer to do) I'm not clear on what the
ongoing maintenance issue is.


As for the meta dependency warnings, I think we all agree that the info is
annoying.  I propose that we change these (and the other similar info warnings)
from info to a "-Drules".

Comment 7 Justin Mason 2006-11-17 02:36:41 UTC

thanks Daryl!  +1 on your patch.

'Myself, I'd prefer that the core rules all had ifplugin lines.  Past the
immediate clean up work (which I volunteer to do) I'm not clear on what the
ongoing maintenance issue is.'

well, if you do that cleanup work, please post the result as a patch here -- I'd
like to see how messy the result is.  Personally, I suspect it's going to be an
unmaintainable nest of intertwined "ifplugin" lines :(

'As for the meta dependency warnings, I think we all agree that the info is
annoying.  I propose that we change these (and the other similar info warnings)
from info to a "-Drules".'

I quite like this. +1

Comment 8 Daryl C. W. O'Shea 2006-11-17 22:30:53 UTC

(In reply to comment #7)
> well, if you do that cleanup work, please post the result as a patch here -- I'd
> like to see how messy the result is.  Personally, I suspect it's going to be an
> unmaintainable nest of intertwined "ifplugin" lines :(

No nesting at all (meta rules aren't affected), so committed as r476455.

Comment 9 Daryl C. W. O'Shea 2006-11-17 22:33:51 UTC

oops... hit commit before the page fully loaded, stupid dial up

Comment 10 Daryl C. W. O'Shea 2006-11-18 01:34:00 UTC

missed moving HEAD_LONG... r476478

Comment 11 Daryl C. W. O'Shea 2006-11-18 16:13:22 UTC

r476665 reverts the temporary fix to build/mkupdates/run_part2 now that
sa-update and the rules have been 'fixed'

Comment 12 Justin Mason 2006-11-23 11:34:12 UTC

as far as I can tell, this can be closed...

Comment 13 Daryl C. W. O'Shea 2006-11-23 11:50:33 UTC

I think I'd like to backport r476035 to 3.1 (which I believe will apply as is)
if we're going to do another 3.1 release.

Still waiting on someone to point out how I broke sa-update with it though. ;)

Comment 14 Daryl C. W. O'Shea 2006-12-04 13:23:27 UTC

Created attachment 3761 [details]
3.1 patch to use site pre files in sa-update lint checks

Comment 15 Theo Van Dinter 2007-01-31 19:56:54 UTC

I think this can wait, pushing to 3.1.9

Comment 16 Daryl C. W. O'Shea 2007-01-31 21:21:06 UTC

I'd like to see this in 3.1.8.  It's been running on all of my 3.1 systems since
November with no issues.  It's been in trunk since then too.

Comment 17 Justin Mason 2007-02-02 02:49:39 UTC

+1

Comment 18 Theo Van Dinter 2007-02-05 15:41:55 UTC

So the reason I pushed this to 3.1.9 originally is that there doesn't seem to be
a consensus on whether or not this is a good idea that even solves the problem,
if in fact there is a problem with the code.

For instance, the same problem will exist if people comment out plugins and then
get an update that doesn't properly have ifplugin conditionals around
plugin-dependent rules and associated configs.

This issue also, currently, doesn't impact 3.1 since the core eval rules are
still in EvalTests.pm, and all of the plugin-dependent rules are enclosed in
conditionals.

So in short, I'm around a -0.8 on this patch -- I'd rather solve the problem
correctly via conditionals than put in a kluge which doesn't really change the
behavior in the long run.

Comment 19 Daryl C. W. O'Shea 2007-02-05 17:02:07 UTC

(In reply to comment #18)
> So the reason I pushed this to 3.1.9 originally is that there doesn't seem to be
> a consensus on whether or not this is a good idea that even solves the problem,
> if in fact there is a problem with the code.

I consider sa-update being able to install an update that could encounter
parsing/whatever errors when spamd is restarted to be a problem.  IMO if the
goal is to be able to automatically restart spamd after sa-update installs an
update we should be sure that the update will actually lint with the particular
system's config.  Maybe that's not our goal.  Although, spamd restarting may not
be avoidable (the system may restart), so installing an update that won't lint
on the particular system seems like a bad idea to me either way.

I thought I'd convinced you that we shouldn't be doing the current partial lint
in bug 5044.  Although your bug 5044 comment #10 could have been you just hoping
I'd shut up about it. :)


> For instance, the same problem will exist if people comment out plugins and then
> get an update that doesn't properly have ifplugin conditionals around
> plugin-dependent rules and associated configs.

I don't think that's the same problem, rather it's a problem that's already
handled.  The update would fail a lint and wouldn't be installed.


> This issue also, currently, doesn't impact 3.1 since the core eval rules are
> still in EvalTests.pm, and all of the plugin-dependent rules are enclosed in
> conditionals.

Sure it does.  If our channel, or anyone else's (like SARE's SPF rules which
have had typo problems before), has invalid/typo'd config lines inside ifplugin
lines the sa-update lint isn't going to catch it, but it will show up when spamd
is restarted (if the user has that plugin enabled).

I just don't see the point in linting a part of the update.  Why only lint
everything outside of ifplugin lines?  Why not lint everything inside ifplugin
lines (for plugins that are enabled) and ignore everything else?  Neither way
makes any sense to me.

Either the update should be checked to see if it'll work with the system's
config or it shouldn't be checked at all.

> So in short, I'm around a -0.8 on this patch -- I'd rather solve the problem
> correctly via conditionals than put in a kluge which doesn't really change the
> behavior in the long run.

Conditionals don't solve the problem since they're required for the problem to
even exist.

The problem I'm looking to solve isn't something about updates with missing
conditional lines (we both agree those updates are broken and shouldn't be
used).  The problem is dealing with updates that have broken lines within
conditionals.

Kluge?  I don't know.  The lint simply uses the .pre files found in the defined
site config dir (the same .pre files spamd/spamassassin is actually going to use
itself).  The end behaviour is certainly different.  The current way will load a
broken update if the brokenness is contained by conditionals.  The proposed way
won't.

Comment 20 Justin Mason 2007-02-08 09:56:43 UTC

ok, maybe we should re-push this to 3.1.9, since there's still a little
disagreement...

Comment 21 Justin Mason 2007-02-13 04:42:58 UTC

pushing to 3.1.9

Comment 22 Theo Van Dinter 2007-02-13 09:02:39 UTC

Ok, after thinking about this some more, I can see your point Daryl.  We're
discussing two different, though somewhat related, issues here, and the patch is
really about one of them and not the other.

So ok, I'll +1 this one for 3.1.8.  We can address the conditional issue in the
updates.