Bug 60330 - reverse proxy of 100-continue aware backend, sends 100 continue even when backend does not
Summary: reverse proxy of 100-continue aware backend, sends 100 continue even when bac...
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy_http (show other bugs)
Version: 2.4.23
Hardware: PC All
: P2 normal with 6 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
: 55433 57853 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-11-02 16:07 UTC by Jay R. Wren
Modified: 2018-07-06 19:19 UTC (History)
4 users (show)



Attachments
in the reverse proxy case, if request had Expects: 100-continue, delay writing 100 continue response until backend has sent 100-continue response (4.87 KB, patch)
2016-11-11 22:28 UTC, Jay R. Wren
Details | Diff
Forward 100-continue (and minimize race when reusing backend connections) (47.09 KB, patch)
2016-11-16 00:07 UTC, Yann Ylavic
Details | Diff
curl(1) to Tomcat directly (3.93 KB, text/plain)
2018-07-06 10:57 UTC, Michael Osipov
Details
curl(1) to Tomcat via HTTPd (2.95 KB, text/plain)
2018-07-06 10:57 UTC, Michael Osipov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jay R. Wren 2016-11-02 16:07:57 UTC
Reverse proxy of 100-continue aware backend, sends 100 continue even when backend does not. This causes a client to think it should write a request body, while the backend may still respond with a 400 and not read the request body. mod_proxy_http then responds with 502 as a result of AH01097: pass request body failed

The backend is doing the right thing: it did not send a 100 continue so it should not be required to read a request body, regardless of transfer encoding or content-length.

Expected:

mod_proxy_http reverse proxy should not send 100-continue to a client unless the backend does.
Comment 1 Jay R. Wren 2016-11-11 22:28:03 UTC
Created attachment 34438 [details]
in the reverse proxy case, if request had Expects: 100-continue, delay writing 100 continue response until backend has sent 100-continue response
Comment 2 Yann Ylavic 2016-11-16 00:07:40 UTC
Created attachment 34451 [details]
Forward 100-continue (and minimize race when reusing backend connections)

I proposed this patch a while ago on the dev@ list ([1]), this is an update for latest trunk, with more (though incomplete) testing.

Could you please give it a try?

[1]. https://lists.apache.org/thread.html/4e541e032b8a77ebec8248534637b47cdcd4f38af79baa5259845db0@1430360070@%3Cdev.httpd.apache.org%3E
Comment 3 Jim Jagielski 2016-11-21 20:25:22 UTC
*** Bug 55433 has been marked as a duplicate of this bug. ***
Comment 4 Jay R. Wren 2016-12-09 03:08:26 UTC
Yann,

I tried that patch, but I still get 503 error when expecting a 100 Continue response.
Comment 5 Michael Osipov 2018-07-04 16:04:20 UTC
Any chance that this will be fixed? Have the very same problem from a backend Tomcat. I guess I need to drop mod_proxy and try mod_ajp or drop Apache HTTPd altogether for this.
Comment 6 Yann Ylavic 2018-07-04 21:05:01 UTC
*** Bug 57853 has been marked as a duplicate of this bug. ***
Comment 7 Yann Ylavic 2018-07-04 21:18:54 UTC
Sorry it's been a long time, I think we need more informations here as to the exact issue.

What exactly isn't working with the proposed patch?
Where are 100-continue or request bodies lacking or sent inappropriately, on which side (client/backend)?
IOW, can we please have a description/scenario of what is supposed to work and how, possibly with the expected request/response on both sides?
What is the configuration being tested?
Comment 8 Michael Osipov 2018-07-06 08:05:17 UTC
(In reply to Yann Ylavic from comment #7)
> Sorry it's been a long time, I think we need more informations here as to
> the exact issue.
> 
> What exactly isn't working with the proposed patch?
> Where are 100-continue or request bodies lacking or sent inappropriately, on
> which side (client/backend)?
> IOW, can we please have a description/scenario of what is supposed to work
> and how, possibly with the expected request/response on both sides?
> What is the configuration being tested?

Hi Yann,

I can provide a full verbose log of curl(1) for Tomcat behind HTTPd with faulty behavior and direct Tomcat communication. Moreover, I can expore the httpd.conf for that offending behavior.
Comment 9 Yann Ylavic 2018-07-06 08:28:23 UTC
Yes please do, along with the httpd error_log with LogLevel trace7.
Thanks!
Comment 10 Michael Osipov 2018-07-06 10:57:04 UTC
Created attachment 36015 [details]
curl(1) to Tomcat directly
Comment 11 Michael Osipov 2018-07-06 10:57:28 UTC
Created attachment 36016 [details]
curl(1) to Tomcat via HTTPd
Comment 12 Michael Osipov 2018-07-06 11:01:12 UTC
The error log has been sent privately due to sensitive data. Looking forward to an analysis.
Comment 13 Michael Osipov 2018-07-06 11:43:39 UTC
FYI, I have tried mod_proxy_{http,ajp} and mod_jk.
Comment 14 Yann Ylavic 2018-07-06 11:56:55 UTC
Thanks Michael, at first glance the error_log is with mod_proxy_ajp, while attachment 34451 [details] is about mod_proxy_http (and I'd like to keep the scope there for now).
I agree that unpatched mod_proxy_http sends "100 continue" too soon (actually independently on the client and backend side).
The patch is precisely to avoid that (hop by hop 100-continue handling), did you give it a try? If yes, could I have the error_log with mod_proxy_http?
Comment 15 Michael Osipov 2018-07-06 19:19:47 UTC
(In reply to Yann Ylavic from comment #14)
> Thanks Michael, at first glance the error_log is with mod_proxy_ajp, while
> attachment 34451 [details] is about mod_proxy_http (and I'd like to keep the
> scope there for now).
> I agree that unpatched mod_proxy_http sends "100 continue" too soon
> (actually independently on the client and backend side).
> The patch is precisely to avoid that (hop by hop 100-continue handling), did
> you give it a try? If yes, could I have the error_log with mod_proxy_http?

I agree, I have tried all possible modules with the same negative result. I will redo for you. Moreover, I will compile from trunk along with your patch and try to reproduce. Does it still apply cleanly to trunk?

Please be patient, I won't be able to test anything before 2018-07-23.

Thank you.