Bug 57229 - mod_sed dies on long lines
Summary: mod_sed dies on long lines
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_sed (show other bugs)
Version: 2.4.10
Hardware: PC Linux
: P2 major (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
Depends on:
Reported: 2014-11-17 11:45 UTC by Oliver Kaufmann
Modified: 2020-02-16 11:12 UTC (History)
1 user (show)


Note You need to log in before you can comment on or make changes to this bug.
Description Oliver Kaufmann 2014-11-17 11:45:39 UTC
When mod_sed is used to replace strings in a document that has long lines (i.e. some Kilobytes), it dies on the first long line that matches and terminates transmission of the rest of the document.

This is reproducable.
How: Make a skript that produces output in a single line and matches. Increase length of line and see when mod_said begins to fail.

Bug is present in apache http 2.4.10 and apache http trunk. 

Here is the rundown:

mod1.c has
#define INIT_BUF_SIZE 1024

If a filtered document has a longer line, buffer gets grown.
But as soon as buffer is grown, mod_sed fails afterwards.

Partial Workaround (compiletime only):
#define INIT_BUF_SIZE 1024
to a larger value that is safe for all lines in all documents you will process.
#define INIT_BUF_SIZE 16384

mod_sed breaks reliable document delivery.

Denial of Service

Please urgently check and fix this mod_sed bug.

My consideration:
It seems to be a buffer problem.
mod_sed very likely works on user submitted content.
From the code I see mod_sed uses a lot of pointers involved with the buffers.
There might be a high risk for buffer overrun/underrun or remote code executions.

Comment 1 jkaluza 2014-11-25 12:21:16 UTC
I'm not able to reproduce it in quick test. Could you attach some minimal configuration which reproduces this issue?
Comment 2 Oliver Kaufmann 2014-12-01 13:01:22 UTC
Bug has been reproduced and confirmed by jkaluza @ redhat . com.
Comment 3 Christophe JAILLET 2016-06-22 12:28:07 UTC

could you provide a way to easily reproduce the issue?

One fix has been applied recently to 2.4.x branch (r1748706) and another one will be proposed for backport in the coming hours. (r1749401)
It is really unlikely that one of these fixes is related to your problem.

But, while at it, if we could investigate on the issue you have reported, it would be fine.
Comment 4 Christophe JAILLET 2016-07-08 05:04:35 UTC
Way to reproduce, sent by Kaufmann, Oliver:

sample php code to reliably trigger the problem:

print "first line with newline (all good)<br>\n";
print "now printing one single matching line with $mynum*$mylength=$bytes bytes expanding to $bytes2 bytes:<br>\n";
for ($i=0; $i < $mynum; $i++)
                print "$myline";
print "\nThis is a test line. If you see it, all is good. Else mod_sed crashed.<br>\n"; 

Apache config snipplet:

LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule sed_module modules/mod_sed.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule info_module modules/mod_info.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule php5_module modules/libphp5.so
AddHandler php5-script php
In vhost:
RequestHeader unset Accept-Encoding
AddOutputFilterByType Sed text/html text/css text/javascript application/javascript
Header unset Content-Length
OutputSed "s/https:\/\/www.asl.com/https:\/\/ge-dev.itools.de.cse.comfin.ge.com/g"

Software: Apache 2.4.10 + mod_sed compiled from source.
mod_sed  source taken from trunk at the moment, but error also happened on stock apache 2.4.10 release version, this is why we switched to trunk.
For our tests we have also compiled sed1.c with different #define INIT_BUF_SIZE values (stock value is 1024). Error scales with this buffer size.
In our current test we have set INIT_BUF_SIZE to 8092 (not 8192).

Remember: We have INIT_BUF_SIZE 8092 in this test.
Ouput: nothing if $mynum>=176. Browser times out and says Connection Problem (Message depends on Browser vendor of course).
Output: correct if $mynum<=175;
first line with newline (all good)
now printing one single matching line with 175*23=4025 bytes expanding to 7350 bytes:
Last line is also shown

Beware: Sometimes (maybe on first request, but seldom) you may get output with values near 176 as well. If you get output this way, you can see that substitution stopped before all lines were processed. Output then shows correctly substituted lines at the beginning but at the end the original lines are shown and rest of document may be completely truncated. Usually if you see this, you can try to reload the page and might then end up with an empty page and the connection problem. With $mynum=178 you will always trigger the error.

Just run your own tests with different buffer sizes.

Linux system: Redhat 6.5 x64
HW: HP Proliant G6 with 192GB of RAM
No other jobs or load on this test system.

Comment 5 Giovanni Bechis 2020-02-16 11:12:31 UTC
I am not able to reproduce the issue on 2.4.41 on CentOS7.