Bug 57087 - mod_proxy_fcgi doesn't send cgi CONTENT_LENGTH variable when the client request used Transfer-Encoding:chunked
Summary: mod_proxy_fcgi doesn't send cgi CONTENT_LENGTH variable when the client reque...
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy_fcgi (show other bugs)
Version: 2.4.7
Hardware: PC All
: P2 normal with 27 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-10-14 08:52 UTC by Philip Cass
Modified: 2018-09-03 08:39 UTC (History)
4 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Cass 2014-10-14 08:52:43 UTC
As per RFC-3875:

"The server MUST set this meta-variable if and only if the request is accompanied by a message-body entity.  The CONTENT_LENGTH value must reflect the length of the message-body after the server has removed any transfer-codings or content-codings."

When a request is sent using Transfer-Encoding:chunked, while mod_proxy_fcgi is removing the transfer coding (by turning each chunk into a fastcgi record) it does not add a CONTENT_LENGTH meta-variable if the request does not contain a Content-Length header (which of course it will not)

This causes problems with, for one, php-fpm which will not parse a POST request body as it assumes the body is of length zero.

This is analogous to bug 53332 and bug 50274 in mod_fcgid
Comment 1 Jim Jagielski 2016-03-07 21:22:50 UTC
Sounds like a bug for sure... looking into it and what, if anything, can be done.
Comment 2 alok 2018-06-27 16:54:22 UTC
Has their been any activity on this? We have just stumbled across the same issue.

Thanks
Comment 3 Luca Toscano 2018-09-01 07:51:35 UTC
As reference, there is a similar bug opened years ago for mod_fcgid (https://bz.apache.org/bugzilla/show_bug.cgi?id=53332). It would be great to fix this bug, it might be a good chance to move people to mod_proxy_fcgi :)
Comment 4 Steffen Moser 2018-09-01 08:01:14 UTC
We've just encountered this problem even with Nextcloud 13.0.5 on Apache 2.4.33 and PHP-FPM 7.1.17, OS: Oracle Solaris 11.3 SRU 34.

Uploads to Nextcloud (which internally uses SabreDAV) initiated by macOS Finder's builtin WebDAV client produce empty files in Nextcloud 13.0.5 storage. When changing from "mod_proxy_fcgi" and PHP-FPM back to the "libphp" module, the problem is gone, but I can only run Apache in prefork MPM then due to multi-threading-related issues in PHP. 

The main problem is that the user on the WebDAV client site doesn't necessarily get noticed about a problem at all. The uploaded files are just empty which is a major failure in my opinion, because data integrity is not guaranteed. Only when uploading larger files (haven't found the size limit yet), the user receivers an unspecified "WebDAV error -36". 

After analyzing this further, I am very sure that it is caused by the chunked encoding WebDAV PUT requests which are used by Apple's Finder. 

Some other WebDAV clients don't show the problem with the same server setup because they use other methods. According to the discussion of the PHP developers [1], this is a behavior against the specification. 


[1] https://bugs.php.net/bug.php?id=60826
Comment 5 Rainer Jung 2018-09-01 09:16:38 UTC
Could you try to set the Apache request environment variable proxy-sendcl?

This should force buffering/spooling request bodies locally (Apache) until they are completely read and then forwariding them with the Content-Length header set.
Comment 6 Luca Toscano 2018-09-01 17:11:30 UTC
Hi Rainer,

I can repro with this simple php script:

<?php
$request = new http\Env\Request();
$request->getBody();
echo($request);
echo("\n");
?>

1) curl -k https://localhost/receiveChunk.php --header "Pragma: no-cache" --header -X POST -d "blablablabla"

POST /receiveChunk.php HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 12
Pragma: no-cache
Accept: */*
User-Agent: curl/7.52.1
Host: localhost

blablablabla

2) curl -k https://localhost/receiveChunk.php --header "Pragma: no-cache" --header "Transfer-Encoding: chunked" -X POST -d "blablablabla"

POST /receiveChunk.php HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Transfer-Encoding: chunked
Pragma: no-cache
Accept: */*
User-Agent: curl/7.52.1
Host: localhost


I also added "SetEnv proxy-sendctl" to my vhost config but didn't observe any difference.

With a bit more detail, I also added LogLevel proxy_fcgi:trace8 and spotted the following log only in the first case:

[Sat Sep 01 17:04:29.816361 2018] [proxy_fcgi:trace8] [pid 3968:tid 140084712040192] mod_proxy_fcgi.c(385): [client ::1:34478] AH01062: sending env var 'CONTENT_LENGTH' value '12'
Comment 7 Luca Toscano 2018-09-01 17:12:50 UTC
> I also added "SetEnv proxy-sendctl"

Sorry it was (of course) "SetEnv proxy-sendctl 1"
Comment 8 Rainer Jung 2018-09-01 17:25:27 UTC
Hi Luca,

it is proxy-sendcl (cl for content length) not proxy-sendctl (not t in ctl).

It is a general mod_proxy feature used in modules/proxy/mod_proxy_http.c. I'm not sure it will help here, but worth a try.

Regards,

Rainer
Comment 9 Luca Toscano 2018-09-01 17:50:14 UTC
(In reply to Rainer Jung from comment #8)
> it is proxy-sendcl (cl for content length) not proxy-sendctl (not t in ctl).

Today is typo-day, I used sendcl in my tests but typed in here 'ctl', sorry :(
I retried to be sure, same result.

> 
> It is a general mod_proxy feature used in modules/proxy/mod_proxy_http.c.
> I'm not sure it will help here, but worth a try.

I checked and indeed it is implemented only in _http, not _fcgi, but it is a good pointer!

After reading a bit the code, I'd add a condition in mod_proxy_fcgi's dispatch() to add the missing CL if TE:chunked is present, but I am not super clear if it is possible to do such a thing from the data in request_req. It seems to me that it shouldn't be super difficult but I need to read more code before adding a meaningful code change. I'll try during the next days if anybody doesn't beat me :)
Comment 10 Luca Toscano 2018-09-01 17:57:15 UTC
(In reply to Luca Toscano from comment #9)
> After reading a bit the code, I'd add a condition in mod_proxy_fcgi's
> dispatch() to add the missing CL if TE:chunked is present, but I am not

send_environment() not dispatch :)
Comment 11 Yann Ylavic 2018-09-02 01:18:31 UTC
(In reply to Luca Toscano from comment #10)
> send_environment() not dispatch :)

Possibly, mod_proxy_http's spool_reqbody_cl() should be copied (and adapted) to mod_proxy_fgci, such that it's called before "Step Two" in proxy_fcgi_handler(), and its spooled brigade is somehow passed further to send_environment(), which can then use the brigade for both C-L and forward.

Hope this helps...
Comment 12 Luca Toscano 2018-09-03 08:39:15 UTC
(In reply to Yann Ylavic from comment #11)
> (In reply to Luca Toscano from comment #10)
> > send_environment() not dispatch :)
> 
> Possibly, mod_proxy_http's spool_reqbody_cl() should be copied (and adapted)
> to mod_proxy_fgci, such that it's called before "Step Two" in
> proxy_fcgi_handler(), and its spooled brigade is somehow passed further to
> send_environment(), which can then use the brigade for both C-L and forward.
> 
> Hope this helps...

Thanks a lot for the pointer Yann! So, unveiling all my ignorance about the CL header, IIUC when TE:chunked is not used (in a POST request), the client sends the CL header and then mod_proxy_fcgi is able to populate the CGI's CONTENT_LENGTH header easily, but when TE:chunked is used, there is no quick way to get the number of bytes in the body/payload unless something explicitly counts them. So mod_proxy_http's trick is to pull the body/payload from the input chain, and then use it later on to easily count the CL.