When accessing Tomcat through mod_proxy_ajp: <IfModule mod_proxy.c> ProxyPass /webapp ajp://localhost:8009/webapp ProxyPassReverse /webapp ajp://localhost:8009/webapp </IfModule> and the initial request is received by Apache over HTTPS, large POST data bodies are often not received by Tomcat. The POST request in question had about 56kB of data. The problem is intermittent, in that the POST data does, on occasion, arrive properly. Even if the POST data does not arrive (Tomcat reports null values for POST data parameters) the request's Content-Length header is still received properly, but not the data itself. This is easily shown by accessing a JSP page that displays both the header and the POST request parameters. The problem was experienced running both Apache 2.2.0 and Apache 2.2.4 against Tomcat 5.5.20, both on SuSE 10.1 and on Mac OS X 10.4.8. Please note: - It is Apache that is accessed over HTTPS, not Tomcat. - Problem does not occur when accessing Apache through non-secure HTTP. - Problem does not occur when accessing Tomcat directly over HTTP (port 8080), bypassing Apache. - Problem DOES occur when using mod_jk instead of mod_proxy_ajp, and again, only over HTTPS. - Problem does not occur when proxying into the Tomcat HTTP connector: <IfModule mod_proxy.c> ProxyPass /webapp http://localhost:8080/webapp ProxyPassReverse /webapp http://localhost:8080/webapp </IfModule> The notes above lead me to believe the issue is with mod_proxy_ajp, but then, I'm not an expert. Some digging around in bugzilla found an eerily similar bug in mod_proxy_http, bug number 37415 (http:// issues.apache.org/bugzilla/show_bug.cgi?id=37145), that was fixed in httpd-2.0.56. Which might explain why the HTTP proxy connection to Tomcat port 8080 is working, but the AJP proxy is not.
Do you use SSLVerifyClient somewhere in the httpd SSL case?
(In reply to comment #1) > Do you use SSLVerifyClient somewhere in the httpd SSL case? No, SSLVerifyClient is not used at all. In fact, here is the complete (comments stripped) vhost-ssl.conf include file: <IfDefine SSL> <IfDefine !NOSSL> <VirtualHost _default_:443> DocumentRoot "/srv/www/htdocs" ErrorLog /var/log/apache2/error_log TransferLog /var/log/apache2/access_log SSLEngine on SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL SSLCertificateFile (valid *.domain crt) SSLCertificateKeyFile (valid key) SSLCACertificateFile (valid DigiCert CA crt) <Files ~ "\.(cgi|shtml|phtml|php3?)$"> SSLOptions +StdEnvVars </Files> <Directory "/srv/www/cgi-bin"> SSLOptions +StdEnvVars </Directory> SetEnvIf User-Agent ".*MSIE.*" \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 CustomLog /var/log/apache2/ssl_request_log ssl_combined RedirectMatch ^/$ https://cms.macewan.ca/webapp </VirtualHost> </IfDefine> </IfDefine> Hope this helps
Thanks for the update. A couple of other questions: 1. Do you get either all POST data in Tomcat or no data at all or do you have situations where you get only parts of the data? 2. If you get data in Tomcat (either partial or complete) is this data always correct or do you receive sometimes corrupted data (e.g. wrong order of data, some data missing within, ...) Please set your loglevel to debug and try to post / attach (depending on the size) the output of your error log for a successfull POST request and a POST request where data is lost. For my convenience: Can you please attach the test jsp that displays both the header and the POST request parameters as mentioned in your first comment?
(In reply to comment #3) > 1. Do you get either all POST data in Tomcat or no data at all or do you have > situations where you get only parts of the data? AFAIK, it's all or nothing. > 2. If you get data in Tomcat (either partial or complete) is this data always > correct or do you receive sometimes corrupted data (e.g. wrong order of data, > some data missing within, ...) Again AFAIK, it seems to be fine when it works. I was trying to reproduce the issue on my local machine today (no access to problem app), fresh httpd-2.2.4, fresh tomcat 5.5.20, fresh little webapp, and I cannot reproduce it. Bum. I'll get more when I'm back at the other site. Thanks so far.
Created attachment 20252 [details] access_log excerpt (incorrect, ignore)
Created attachment 20253 [details] error_log excerpt
I attached excerpts from the access_log and error_log files. LogLevel is set tp debug. The interesting rquests are the POST requests at 10:15:39 and 10:15:45. (The one inbetween is not really involved in the issue.) The two requests mentioned are the exact same request (although the request content-length seems to be different...) The first one succeeds, the second one fails to reach Tomcat properly.
Created attachment 20254 [details] access_log excerpt
Okay, not a lot of testing done, but it seems like I can make the problem go away by upgrading the following two components: - tomcat-native from 1.1.6 to 1.1.9 - libapr (only for tomcat-native) from 1.2.7 to 1.2.8 This way the requests seem to go through okay using httpd 2.2.0 or 2.2.4 (with included apr versions). FYI
(In reply to comment #9) > following two components: > > - tomcat-native from 1.1.6 to 1.1.9 > - libapr (only for tomcat-native) from 1.2.7 to 1.2.8 > > This way the requests seem to go through okay using httpd 2.2.0 or 2.2.4 (with included apr versions). This is possible. I noticed the following difference between the working and the non working request: [Wed May 23 10:15:39 2007] [debug] mod_proxy_ajp.c(193): proxy: data to read (max 8186 at 4) [Wed May 23 10:15:39 2007] [debug] mod_proxy_ajp.c(208): proxy: got 2048 bytes of data [Wed May 23 10:15:39 2007] [debug] ajp_header.c(643): ajp_read_header: ajp_ilink_received 06 [Wed May 23 10:15:39 2007] [debug] ajp_header.c(653): ajp_parse_type: got 06 This means mod_proxy_ajp sends the first 2048 bytes of the 3488 bytes of the POST request body immediately and it gets back a request from Tomcat's AJP connector to sent more (type 06) (in this case the missing 1440 bytes). And as the SSL BIO dump afterwards indicates mod_proxy_ajp sends POST request body data to the Tomcat afterwards. Next we have [Wed May 23 10:15:39 2007] [debug] ajp_header.c(643): ajp_read_header: ajp_ilink_received 04 [Wed May 23 10:15:39 2007] [debug] ajp_header.c(653): ajp_parse_type: got 04 which indicates that Tomcat starts sending the response by sending the response headers across. In the second case things look a little bit different: [Wed May 23 10:15:45 2007] [debug] mod_proxy_ajp.c(193): proxy: data to read (max 8186 at 4) [Wed May 23 10:15:45 2007] [debug] mod_proxy_ajp.c(208): proxy: got 2048 bytes of data [Wed May 23 10:15:45 2007] [debug] ajp_header.c(643): ajp_read_header: ajp_ilink_received 04 [Wed May 23 10:15:45 2007] [debug] ajp_header.c(653): ajp_parse_type: got 04 mod_proxy_ajp again only sents the first 2048 bytes of the POST request body immediately, but instead of asking for more (as in the first case), Tomcat immediately starts with the response by sending the headers. So this could be a Tomcat bug. For further verification I think it could be helpful to have a network sniff of the communication between httpd and Tomcat to see if there are errors in the messages that could trigger this behaviour on the Tomcat side. Why does this not happen in the HTTP case? Well I do not know exactly, but it may be that in the HTTP case the whole POST request body is sent immediately. Of course this should only make a difference for POST request bodies < 8k as if they are larger they need to split into different ajp packets anyway. So it might be nice to have a debug error log of the same POST request over HTTP on the frontend. What about the 56k you talk about in your initial comment? Is this the size of the request body or is this the size of the response?
did anyone find any further info on this? I have a very similar issue, on CentOS 6. Running tomcat 6.0.24, openjdk 1.7 and apache 2.2.15. When I send a request (using curl for testing purposes) with 1 POST paramter of ~18k in size - it can be read just fine by request.getparameter in tomcat webapp (when I do curl directly to tomcat port 8080). When I send the exact same request to apache - setup to do proxypass using ajp (and running in https vhost), tomcat dies with an iooutofboundsexception.. meaning it dies when trying to read the actual parameter contents.
If I post through port http instead of https - it works as it's supposed to. Both vhosts have EXACT same setup - except for using certs on SSL. proxyrequests off proxypass /mywebapp/ ajp://localhost:8009/mywebapp/ retry=0 ProxyTimeout 600 ProxyIOBufferSize 19000 <Proxy *> Order deny,allow Allow from all </Proxy>
updated to also be relevant for v2.2.15
I seem to have the same problem but with: apache 2.4.6 tomcat 7.0.69 tomcat-native 1.1.34 using prepackaged Centos and EPEL binaries. Was there ever a resolution for this? It doesn't seem to relate to the POST data size - even tiny content doesn't make it to Tomcat.
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.