Bug 6176 - [review] spamc truncates lines read from spamc.conf
Summary: [review] spamc truncates lines read from spamc.conf
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: spamc/spamd (show other bugs)
Version: 3.3.0
Hardware: Other All
: P2 enhancement
Target Milestone: 3.3.0
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard: needs 1 more vote
Keywords:
Depends on:
Blocks:
 
Reported: 2009-08-10 03:44 UTC by Justin Mason
Modified: 2010-01-07 17:29 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status
truncate without message fix application/octet-stream None Nico Prenzel [HasCLA]
truncate without message fix V2 application/octet-stream None Nico Prenzel [HasCLA]
proposed patch patch None Mark Martinec [HasCLA]
proposed patch, take 2 patch None Mark Martinec [HasCLA]
Enhanced error message patch None Mark Martinec [HasCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Justin Mason 2009-08-10 03:44:34 UTC
investigate for 3.3.0.  from the users list... Giles Malet to users Aug 7 (3 days ago):
	
'I'm heading off on vacation in moments, but didn't want this to get lost, as it cost me hours of work to find!

This on:
Red Hat Enterprise Linux AS release 4 (Nahant Update 7)
Red Hat Enterprise Linux Server release 5.3 (Tikanga)
spamassassin-3.2.5-1.el4.rf
spamassassin-3.2.5-1.el5.rf

If one puts the following two lines into spamc.conf (the second may wrap):

-H
-d localhost,mailchk-w05-internal,mailchk-w04-internal,mailchk-w01-internal,mailchk-w02-internal,mailchk-m01-internal,mailchk-m02-internal,mailchk-m03-internal,mailchk-m04-internal,mailchk-m05-internal

Then one finds this in the syslog:

spamc[5471]: getaddrinfo(ma) failed: Name or service not known

.. and user preferences were not being processed (this all started with a report that whitelisting wasn't working). Well, I'm not sure what it was doing, but it wasn't the correct thing.

It appears that spamc reads exactly 95 bytes from that line, and silently truncates the input. Turns out that the last thing in the buffer is `..,ma' -- thus the failed DNS lookups on `ma', and subsequent lossage. It would be better if spamc actually took note of a full buffer, and at least backed up to the prior separator, and at least logged this, or did something more sane than causing user preferences to stop working!

Thanks, and apologies for perhaps not doing this properly. I can't see anything in bugzilla, but may have missed it in my rush.

gdm'
Comment 1 Justin Mason 2009-12-07 12:59:00 UTC
this is quite serious; it'd be nice to fix before 3.3.0
Comment 2 Nico Prenzel 2009-12-17 04:58:50 UTC
Created attachment 4607 [details]
truncate without message fix

Hi,
this is a patch for the bug posted here. It also increases the buffer size to 150. So, the maximum character size for one line is now 148!
If configuration line exeeds the maximum then spamc quits with an error message.

Greeting NicoP.
Comment 3 Mark Martinec 2009-12-17 05:14:53 UTC
Looks good to me (except for <CR>s in the patch).
I'd push the max line size to 255+2 though.
Comment 4 Justin Mason 2009-12-17 05:38:42 UTC
why not something much higher? 8192 for instance?
Comment 5 Nico Prenzel 2009-12-17 05:41:27 UTC
Please,

hold on. It's not that well done. I'll do another round.
Comment 6 Nico Prenzel 2009-12-17 06:10:46 UTC
Created attachment 4608 [details]
truncate without message fix V2

Ok. Here is the second version. Does apply to trunk (without my first patch). And again with the not wanted <CRs>. Sorry, don't know how to easily get rid of that.

First version had a problem that it exited to early (in case of -V).
Also, applied larger line maximum (257). But, that's your decision.

NicoP.
Comment 7 Mark Martinec 2009-12-17 06:29:14 UTC
(In reply to comment #6)
> Created an attachment (id=4608) [details]
> truncate without message fix V2

+ if( (option[strlen(option)-1] != '\n') && (!feof(config)) ) {

Hmm, fgets could conceivably return an empty string (null at index 0),
so strlen(option)-1 could yield a -1. Better to be safe and test
the length first.

> Ok. Here is the second version. Does apply to trunk (without my first patch).
> And again with the not wanted <CRs>. Sorry, don't know how to easily get rid of
> that.

No problem, we just need to be careful when applying a patch.

> First version had a problem that it exited to early (in case of -V).

Thanks.

> Also, applied larger line maximum (257). But, that's your decision.

Good. I don't mind any reasonably large value. Justin's 8k is good too.
Comment 8 Nico Prenzel 2009-12-17 08:18:35 UTC
replace the corresponding while loop with the following:
while(!(feof(config)) && (fgets(option, CONFIG_MAX_LINE_SIZE, config) != NULL)) {

then I don't see any more case where fgets could return a null pointer.
Comment 9 Mark Martinec 2009-12-17 10:31:43 UTC
Created attachment 4611 [details]
proposed patch

Please review and vote for the attached patch.
  (a slightly cleaned version of V2 (tabs, cr, option_l, 8kB)).
Comment 10 Kevin A. McGrail 2009-12-17 10:58:53 UTC
(In reply to comment #9)
> Created an attachment (id=4611) [details]
> proposed patch
> 
> Please review and vote for the attached patch.
>   (a slightly cleaned version of V2 (tabs, cr, option_l, 8kB)).

KAM +1
Comment 11 Justin Mason 2009-12-17 13:26:56 UTC
+1
Comment 12 Mark Martinec 2009-12-17 13:52:06 UTC
Bug 6176: spamc truncates lines read from spamc.conf
Sending        spamc/spamc.c
Sending        spamc/spamc.h.in
Committed revision 891935.
Comment 13 Mark Martinec 2009-12-17 13:53:06 UTC
Closing.
Comment 14 Warren Togami 2009-12-19 21:56:52 UTC
[warren@computer trunk]$ cat sample-nonspam.txt |spamc
[warren@computer trunk]$ echo $?
0

spamd -D sees no connection attempt from spamc.  Same result if spamd is running or not.

http://svn.apache.org/viewvc?view=revision&revision=891935
I backed out this patch and spamc works again.
Comment 15 Warren Togami 2009-12-19 22:47:08 UTC
NOTE: Output of nothing with zero return value means blackholing of mail.  Very dangerous.  This should be a very simple test-case possible with "make test".  A working spamd is not necessary to test this.
Comment 16 Warren Togami 2009-12-20 14:59:11 UTC
I backed this out just in case folks are dogfooding the trunk.

I am traveling most of Dec 21st.  I suppose rc1.proposed2 can be cut after you folks decide what to do about this bug.  I will be able to cut it on Tuesday, unless jm or somebody wants to do the cut on Monday.
Comment 17 Mark Martinec 2009-12-21 18:27:21 UTC
Created attachment 4613 [details]
proposed patch, take 2

Apparently nobody is brave enough to touch the spamc.conf, so I'm
reluctantly doing it. Also fixed a buffer overflow when there
are more than 24 arguments present (command line + from a config file),
and fixed one spelling.
Comment 18 Justin Mason 2009-12-22 03:02:29 UTC
(In reply to comment #17)
> Created an attachment (id=4613) [details]
> proposed patch, take 2
> 
> Apparently nobody is brave enough to touch the spamc.conf, so I'm
> reluctantly doing it. Also fixed a buffer overflow when there
> are more than 24 arguments present (command line + from a config file),
> and fixed one spelling.

I hate to be annoying, but IMO it's critical that we come up with a .t script to exercise the failure case found in rc1.proposed1.
Comment 19 Justin Mason 2009-12-22 03:59:48 UTC
(In reply to comment #18)
> I hate to be annoying, but IMO it's critical that we come up with a .t script
> to exercise the failure case found in rc1.proposed1.

I've just done this:

: 106...; svn commit -m "bug 6176: add test script for error that broke rc1.proposed1"
Sending        MANIFEST
Adding         t/spamc_bug6176.t
Transmitting file data ..
Committed revision 893149.
Comment 20 Justin Mason 2009-12-22 04:01:38 UTC
(In reply to comment #17)
> Created an attachment (id=4613) [details]
> proposed patch, take 2
> 
> Apparently nobody is brave enough to touch the spamc.conf, so I'm
> reluctantly doing it. Also fixed a buffer overflow when there
> are more than 24 arguments present (command line + from a config file),
> and fixed one spelling.

+1
Comment 21 Warren Togami 2009-12-22 10:44:53 UTC
Tested in production and reviewed patch manually.

+1

I'm cutting rc1.proposed2 after you commit this, unless somebody suggests we delay for any particular other patch.
Comment 22 Mark Martinec 2009-12-22 10:55:15 UTC
Bug 6176 spamc.c fix: spamc truncates lines read from spamc.conf (take two)
Sending        spamc/spamc.c
Sending        spamc/spamc.h.in
Transmitting file data ..
Committed revision 893270.
Comment 23 Mark Martinec 2009-12-22 10:56:04 UTC
re-closing
Comment 24 Mark Martinec 2009-12-27 17:06:52 UTC
Created attachment 4616 [details]
Enhanced error message

Attached is a little additional patch to spamc.c to distinguish a
line-too-long condition from a non-NL-terminated line, and issue an
appropriate diagnostic for each case, instead of always claiming
that line is too long.
Comment 25 Mark Martinec 2009-12-27 17:08:22 UTC
Reopening for a small enhancement.
Comment 26 Justin Mason 2010-01-07 14:11:04 UTC
+1
Comment 27 Kevin A. McGrail 2010-01-07 17:12:19 UTC
(In reply to comment #26)
> +1

+1 KAM
Comment 28 Mark Martinec 2010-01-07 17:29:52 UTC
Bug 6176: distinguish too long lines from non-NL-terminated (last) line
in the issued diagnostic
Sending spamc/spamc.c
Committed revision 897074.