Bug 8482 - nph- CGIs and/or server-pushed methodes not working properly
Summary: nph- CGIs and/or server-pushed methodes not working properly
Status: CLOSED FIXED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_cgi (show other bugs)
Version: 2.0.43
Hardware: All All
: P1 major with 14 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-04-24 18:11 UTC by zapp
Modified: 2004-11-16 19:05 UTC (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description zapp 2002-04-24 18:11:25 UTC
i'm using a nph- CGI, which isn't working properly with apache 2.0.35, although 
the changelog says, that nph- is working again!

description:
after starting the nph- CGI, any browser receives some data (no prob so far)
but after a certain (almost everytime the same) amount, the browers stops 
getting data.
the more strange thing is .. the cgi (cgi-irc) still works!
and after quiting that CGI, the browers receives the rest of the data-packets.

so i assume, that the server-pushed method gets a kind of overflow ..
Comment 1 zapp 2002-04-24 18:12:45 UTC
forget to say: that script worked on apache 1.3.19
Comment 2 Andy Green 2002-05-08 11:09:24 UTC
I am also experiencing this with a nph- cgi.  It seems to be after 64K-128K of 
data the remainder is corrupted when sent through to the requesting web 
browser.  The data is clean FOR SURE when it is issued by the cgi.
Comment 3 Jeff Trawick 2002-05-08 14:11:24 UTC
Do you have a sample which exhibits the problem?  How is the data generated?  
Slowly over time?  In one burst?  This nph- script works for me (all data 
received by browser):
#!/bin/sh

echo "HTTP/1.1 200 OK"
echo "Date: Thu Nov 29 16:22:33 CET 2001"
echo "Server: Apache"
echo "Connection: close"
echo "Content-Type: text/html"
echo
cat /home/trawick/apacheinst/manual/mod/mod_ssl.html
cat /home/trawick/apacheinst/manual/mod/mod_ssl.html
Comment 4 zapp 2002-05-10 10:17:36 UTC
well, i'm not using such a simple one. i'm using cgi-irc 
(http://cgiirc.sourceforge.net). it's a chat skript, so the data is generated 
slowly over time. anonther point maybe could be the multipart content-type:

#!/usr/bin/perl
   print "HTTP/1.0 200 OK\n";
   print "Cache-Control: no-cache\nExpires: -1\nPragma: no-cache\n";
   print "Content-type: multipart/x-mixed-replace;boundary=BOUNDARY\n\n";
Comment 5 zapp 2002-05-11 16:00:10 UTC
still the same problem in version 2.0.36
Comment 6 zapp 2002-06-19 14:18:36 UTC
still not working in 2.0.39

the problem, that the browsers gets only a certain amount of data, seems to 
fixed, but now it looks like the connection isn't a keep-alive one.

as mentioned, the problem apears using the cgi-irc (cgiirc.sourceforge.net).
while the browser still can send data to the server (which this cgi is handling 
properly), the data which should be sent from the cgi, never reaches the 
browser.

it's working on apache 1.3.19 .. but not on apache 2.0.39
if you wish to take a look:

http://www.galgenberg.net/cgi-bin/cgiirc/irc.cgi <-- apache 2.0.39

http://132.187.222.249/cgi-bin/cgiirc/irc.cgi <-- apache 1.3.19
Comment 7 David Leadbeater 2002-06-20 13:09:41 UTC
Hi, i'm the author of CGI:IRC - thanks to zapp for pointing me at this bug. 
I've now written a test-case that shows this working on apache 1.3 but not 
apache 2.

It doesn't seem to be related to nph scripts, just anything sending content 
that shouldn't be buffered.

The test makes use of javascript in the stream and for me works fine on apache 
1.3 (two installs tested).

Apache 2: http://cgiirc.blitzed.org/teststream.cgi
Apache 1.3: http://dgl.cx/test/teststream.cgi
(Also nph- scripts at same locations as above to show it affects both nph and 
non-nph scripts).
Source code: http://cgiirc.sourceforge.net/files/teststream.cgi

Hope this helps.
Comment 8 David Leadbeater 2002-06-20 13:11:41 UTC
(Note that if the test fails but these values only differ by a second then it's 
likely to be a laggy connection). On apache 2 they nearly all appear at the 
same time. Also I know CGI:IRC has problems with opera and konqueror so don't 
test on those browsers).
Comment 9 David Leadbeater 2002-06-20 13:12:43 UTC
And sourceforge appear broken for serving scripts so use 
http://cgiirc.sourceforge.net/files/teststream.pl.txt and rename it..
Comment 10 Jeff Trawick 2002-06-20 13:17:54 UTC
Thanks for the recent updates.  Just to avoid wasting anybody's time:

this is an acknowledged problem that multiple developers have been able
to reproduce

it is considered high-priority (tracked in STATUS file)

hopefully this will be resolved before long
Comment 11 David Leadbeater 2002-06-30 21:25:34 UTC
Originally I said non-nph scripts had problems too, this doesn't seem to be the
case and i've now got cgiirc.blitzed.org working on apache 2.0 by not using an
nph script.

I think this problem might be related to bug number 8388? As The only thing
remaining is the script doesn't always seem to be correctly killed by apache
when the connection terminates, on apache 1.3 I'm quite sure it used to send a
SIGINT (or similar) to the cgi script when the connection was closed, but apache
2 doesn't appear to be doing that?
Comment 12 Nic Doye 2002-07-31 11:06:59 UTC
We have a CGI script which works fine in 1.3.20 (and on the command line: 'su -
nobody -c "/opt/bin/perl /blah/cgi-bin/dpradm
page=18.html\&arg1=val1\&arg2=val2"') but never returns all the data to the
browser (or telnet port80 session).

This is on Solaris 2.6.

As a result, it looks like our only Apache 2 box is going to roll back to 1.3.26.
Comment 13 Nic Doye 2002-08-12 10:49:21 UTC
I believe that Apache 2.0.40 has fixed this bug for me. (only build difference
being: --disable-threads).
Comment 14 zapp 2002-08-20 09:14:18 UTC
Bug still exists in 2.0.40
Comment 15 Nic Doye 2002-08-20 09:17:38 UTC
Agreed - bug still exists (but occurs less). I dont think this is limited to NPH
either.
Comment 16 Joshua Slive 2002-10-17 02:34:22 UTC
[This is a mass bug update.]
This bug reports a problem in an older version of Apache 2.
Could you please update to the most recent version and see
if you can reproduce this problem.  If the bug still exists,
please update the bug with the latest version number.  If 
the bug no longer exists, please close the bug report.

Sorry for this impersonal response, but we get many more bug
reports than our volunteers can keep up with.
Thanks for using Apache!
Comment 17 zapp 2002-10-17 12:59:05 UTC
sry guys, but it's still not working in 2.0.43
Comment 18 David Leadbeater 2002-10-23 22:39:59 UTC
I haven't had chance to try apache 2 recently, but people are still reporting
this and asking about it.
Any chance of some sort of update about when it might be fixed?
Comment 19 Jeff Trawick 2002-10-23 22:55:37 UTC
Zapp, were you using mod_cgi or mod_cgid when you tried with 2.0.43?

2.0.43 has some fixes to help streaming in general (nph or not).

A problem specific to mod_cgid which broke streaming with wasn't fixed 
until right after 2.0.43 was released.

Comment 20 zapp 2002-10-24 10:23:48 UTC
hi trawick

LoadModule cgi_module libexec/apache2/mod_cgi.so

we're going to update to the latest CVS version this (maybe tomorrow) 
afternoon. keep u informed!

any ideas, when v2.0.43++ will be released?
Comment 21 Jeff Trawick 2002-10-24 12:01:36 UTC
If you were using mod_cgi with 2.0.43, you have all the fixes that are
available.

Here is teststream.pl on cvs HEAD with mod_cgid:

---------
Testing streaming of content (with scripting)
If most of the tests below succeed then it is likely streaming of content works
fine. If they fail then either the webserver, a proxy between you and the
webserver or your browser is having problems with streams (and sometimes scripts
embedded inside streams). If the difference in any failed tests is only a second
then it is more likely that your internet connection to the server is slow.

Test 0: OK
Test 1: Failed: 1035460289 != 1035460288
Test 2: Failed: 1035460294 != 1035460293
Test 3: Failed: 1035460299 != 1035460298
Test 4: Failed: 1035460304 != 1035460303
Test 5: Failed: 1035460309 != 1035460308
Less than 4 tests were successful. You probably have a problem with streaming of
content (or a slow internet connection)
End of test.
------------

Note that the failures were all only 1 second too late.

Here is teststream.pl on cvs HEAD with mod_cgi:

------
Testing streaming of content (with scripting)
If most of the tests below succeed then it is likely streaming of content works
fine. If they fail then either the webserver, a proxy between you and the
webserver or your browser is having problems with streams (and sometimes scripts
embedded inside streams). If the difference in any failed tests is only a second
then it is more likely that your internet connection to the server is slow.

Test 0: OK
Test 1: OK
Test 2: OK
Test 3: OK
Test 4: OK
Test 5: OK
6 out of 6 tests were OK, looks like streaming is working
End of test.
-----

Should I assume that there is a functional problem with mod_cgid based
on this test, or perhaps it is just a bit slower than mod_cgi?
Comment 22 zapp 2002-10-24 12:58:06 UTC
weird..

testing with CGI:IRC v0.5

2.0.43:
client receives an exact number of bytes, then nothing for about 10 secs. after 
that, CGI:IRC connects to the IRC-Server and the client completely receives the 
rest. the connection is now closed. (data can be sent to (irc-)server, but 
cannot be received)

2.0.44-dev: (httpd-2.0_20021024101256.tar.gz)
client receives exact number of bytes, too. no waiting. CGI:IRC starts 
immediately. BUT the client doesn't receive the all the data. the amount of 
bytes, received by the client after CGI:IRC has connected to IRC-Server, is 
changing from time to time; after a dozen tests, the data hasn't been fully 
received one single time.
the connection stays alive till termination (by user =)
(same as above: data sending, not receiving...)

maybe david can provide some more details, what's going on that could terminate 
the data-flow.
Comment 23 David Leadbeater 2002-10-24 14:38:34 UTC
I've just installed apache 2.0.43 to test with - using mod_cgi and prefork
(default ./configure options). 

teststream works fine -but- if it is called nph-teststream.cgi all but one of
the tests fail (and it's out by much more than a second).

So, the first problem is fixed by renaming it something not beinging with nph-
(I don't understand that - I thought the idea was to do less processing on nph-
scripts). For CGI:IRC renaming nph-irc.cgi to something like irc-main.cgi and
updating config (and removing the 'HTTP/1.0 200 OK' line on non-CVS versions)
fixes it. 

The other problem briefly mentioned above is still present, no signal is sent to
the process to terminate it when the user stops the request, Apache 1.3 does,
although I can't find this in any CGI specification, but it's a very useful feature.
Comment 24 Jeff Trawick 2002-10-24 15:43:17 UTC
Thanks for the reminder about calling the script nph-...  Nothing written
to the network in that case, so I'll start debugging that.
Comment 25 Jeff Trawick 2002-10-24 18:10:26 UTC
The nph flavor of teststream.pl works fine for me with this patch.

Wanna try the patch with the more complicated test (irc script)?

Index: server/core.c
===================================================================
RCS file: /home/cvs/httpd-2.0/server/core.c,v
retrieving revision 1.213
diff -u -r1.213 core.c
--- server/core.c       14 Oct 2002 20:08:15 -0000      1.213
+++ server/core.c       24 Oct 2002 17:26:10 -0000
@@ -3653,6 +3653,7 @@
     conn_rec *c = f->c;
     core_net_rec *net = f->ctx;
     core_output_filter_ctx_t *ctx = net->out_ctx;
+    apr_read_type_e eblock = APR_NONBLOCK_READ;

     if (ctx == NULL) {
         ctx = apr_pcalloc(c->pool, sizeof(*ctx));
@@ -3728,7 +3729,16 @@
                 const char *str;
                 apr_size_t n;

-                rv = apr_bucket_read(e, &str, &n, APR_BLOCK_READ);
+                rv = apr_bucket_read(e, &str, &n, eblock);
+                if (APR_STATUS_IS_EAGAIN(rv)) {
+                    /* send what we have so far since we shouldn't expect more
+                     * output for a while...  next time we read, block
+                     */
+                    more = apr_brigade_split(b, e);
+                    eblock = APR_BLOCK_READ;
+                    break;
+                }
+                eblock = APR_NONBLOCK_READ;
                 if (n) {
                     if (!fd) {
                         if (nvec == MAX_IOVEC_TO_WRITE) {
Comment 26 Jeff Trawick 2002-10-25 15:38:13 UTC
The last patch posted to this PR has been tested successfully and will be
in the next release of Apache.

There is still a concern about the issue described in PR 8388, but the
issue of server-pushed output for nph- and non-nph- seems to be resolved.
Comment 27 ValK 2003-06-19 01:55:32 UTC
Apache/2.0.46 (Unix) mod_perl/1.99_09 Perl/v5.6.1 mod_ssl/2.0.46 OpenSSL/0.9.6c
Server at valk.ath.cx Port 443

Using https connection it not work... with http it work some helps?
Comment 28 Cliff Woolley 2003-06-19 02:02:32 UTC
Valk: your question would appear to be unrelated to this bug report.  And it doesn't 
appear to be a bug report of its own, but rather a user-support question.  Please 
email your question to modssl-users@modssl.org.  They will, no doubt, want more 
details on your configuration in order to help diagnose your problem.