Bug 60717

There seems to be two main aspects to this problem:
1. How mod_proxy_http handles a backend sending a TCP RST.
2. How mod_proxy_http handles a 401 response code from the backend, especially in relation to a "Expect: 100-continue" in the request.


The test case is an Apache Httpd server (2.4.25) used as a front-end to a Jetty (9.3.16) server, using mod_proxy_http as a reverse proxy.

The mod_proxy configuration is as follows:

    SetEnv HTTPS 1
    <Location /test/>
        ProxyPass http://localhost:8080/test/ retry=10
        ProxyPassReverse http://localhost:8080/test/
        RequestHeader set X-Forwarded-Proto "https" env=HTTPS
        RequestHeader set X-Forwarded-Port 443 env=HTTPS
    </Location>

	
The Jetty service is configured to use HTTP Basic authentication.

When using Curl to send an external POST request using a wrong username/password, I get a 502 status code from Apache Httpd.


Here is what is seen from the external client:

> POST /test/ HTTP/1.1
> Authorization: Basic ...
> User-Agent: curl/...
> Host: test.example.com
> Accept: application/xml
> Referer: https://test.example.com/test/
> Content-Type: application/xml
> Content-Length: 40220
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 502 Bad Gateway
< Date: Thu, 09 Feb 2017 11:27:03 GMT
< Server: Apache/2.4.25
< Content-Length: 232
< Content-Type: text/html; charset=iso-8859-1
* HTTP error before end of send, stop sending



Here is what was sent between Apache Httpd and the Jetty server locally:


A:    POST /test/ HTTP/1.1
A:    Host: test.example.com
A:    Authorization: Basic ....
A:    User-Agent: curl/...
A:    Accept: application/xml
A:    Referer: https://test.example.com/test/
A:    Content-Type: application/xml
A:    Expect: 100-continue
A:    X-Forwarded-Proto: https
A:    X-Forwarded-Port: 443
A:    X-Forwarded-For: ....
A:    X-Forwarded-Host: ....
A:    X-Forwarded-Server: ...
A:    Connection: Keep-Alive
A:    Content-Length: 40220
A:    
J:    HTTP/1.1 401 Bad credentials
J:    X-Content-Type-Options: nosniff
J:    X-XSS-Protection: 1; mode=block
J:    Pragma: no-cache
J:    Strict-Transport-Security: max-age=31536000 ; includeSubDomains
J:    X-Frame-Options: SAMEORIGIN
J:    WWW-Authenticate: Basic realm="Realm"
J:    Cache-Control: must-revalidate,no-cache,no-store
J:    Content-Length: 0
J:    Connection: close
J:    
A:    <?xml version="1.0" encoding="UTF-8"?>
A:    ...





Here is what the Wireshark packet summary looks like:

No.     Time           Destination Port Protocol Length Info
      1 0.000000       8080             TCP      94     43646 ? 8080 [SYN] Seq=0 Win=65476 Len=0 MSS=65476 SACK_PERM=1 TSval=577106868 TSecr=0 WS=128
      2 0.000084       43646            TCP      94     8080 ? 43646 [SYN, ACK] Seq=0 Ack=1 Win=65464 Len=0 MSS=65476 SACK_PERM=1 TSval=577106868 TSecr=577106868 WS=128
      3 0.000168       8080             TCP      86     43646 ? 8080 [ACK] Seq=1 Ack=1 Win=65536 Len=0 TSval=577106868 TSecr=577106868
      4 0.036775       8080             HTTP     642    POST /test/ HTTP/1.1 
      5 0.036836       43646            TCP      86     8080 ? 43646 [ACK] Seq=1 Ack=557 Win=66688 Len=0 TSval=577106905 TSecr=577106905
      6 0.039358       43646            HTTP     423    HTTP/1.1 401 Bad credentials 
      7 0.039562       43646            TCP      86     8080 ? 43646 [FIN, ACK] Seq=338 Ack=557 Win=66688 Len=0 TSval=577106908 TSecr=577106905
      8 0.043988       8080             TCP      86     43646 ? 8080 [ACK] Seq=557 Ack=338 Win=66560 Len=0 TSval=577106912 TSecr=577106907
      9 0.046297       8080             TCP      24662  [TCP segment of a reassembled PDU]
     10 0.046390       43646            TCP      74     8080 ? 43646 [RST] Seq=339 Win=0 Len=0
     11 0.047071       8080             TCP      86     43646 ? 8080 [RST, ACK] Seq=25133 Ack=339 Win=66560 Len=0 TSval=577106915 TSecr=577106908




Part of the problem here is that Jetty almost immediately sends a TCP RST, after sending its 401 response (with a "Connection: close"), partly to prevent DOS attacks due to unauthenticated requests: https://github.com/eclipse/jetty.project/issues/651


Where I think there may be a problem on Apache Httpd's side. The original client sent an "Expect: 100-continue" header, and that was forwarded by mod_proxy. Yet, Jetty replied with a 401 response before getting any of the request's entity, and certainly before sending a 100 response.
I believe in this case that mod_proxy should:
- not send "HTTP/1.1 100 Continue" back to the client
- not carry on with sending the request's entity to Jetty

On top of this, because Jetty sends a TCP RST, this causes mod_proxy to send a 502 back to the client, instead of the 401 (with headers) it already received from the backend.

A consequence is that some clients that don't use pre-emptive HTTP Basic authentication (i.e. those that will only send the Authorization header when challenged with a 401 response) will just take this 502 response as a failure, instead of trying again with credentials.


There already are a couple of issues related to this:
- bug 51867 concluded (rightly, I think) that the backend "needs to consume the body of requests even if it does not need them". However, in this case we're using "Expect: 100-continue" and the 401 response is sent before the request entity is sent.
- bug 49405: although sending a TCP RST is indeed a bit abrupt, this is done after sending a full valid 401 response (with all headers, "Content-Length: 0", and "Connection: close), before any request entity was sent to the backend. It should at least return that instead of 502 (especially w.r.t. 100-continue issue).