Bug 57439

Summary: Allow to change body timeout per location
Product: Apache httpd-2 Reporter: Armin Abfalterer <a.abfalterer>
Component: mod_reqtimeoutAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: NEW ---    
Severity: enhancement CC: micha
Priority: P2 Keywords: PatchAvailable
Version: 2.5-HEAD   
Target Milestone: ---   
Hardware: All   
OS: All   
Attachments: Allow to change body timeout per location
Allow to change body timeout per location

Description Armin Abfalterer 2015-01-13 14:27:11 UTC 
Created attachment 32367 [details]
Allow to change body timeout per location

Initial situation
*****************

I am using Apache httpd as a reverse proxy to several web applications. In addition, I use mod_reqtimeout to mitigate DoS attacks by setting timeouts and minimum data rates.

As the RequestReadTimeout setting can only be performed per virtual host all locations of a virtual host are bound to the same timing limits. The only way to change the timeout for a location is to move it to a separate virtual host.

However, this might not always be feasible and practicable so that the adjustment or de-activation of the timeout per location is desirable.

Use case
********

A client uses a WebSocket to frequently send data to an application server. The WebSocket is initiated by a JS script that is provided on another location. Besides the WS application, other HTTP-based services may be served on the same virtual host. The applications and services may depend on each other so they can not be separated.

<VirtualHost>
  RequestReadTimeout header=2 body=10

  <Location /client.js>
    ProxyPass http://wshost:11111/
    ProxyPassReverse http://wshost:11111/
  </Location>

  <Location /websocket>
    ProxyPass ws://wshost:22222/
    ProxyPassReverse  ws://wshost:22222/
  </Location>

  <Location /http-service-1>
    ...
  </Location>

  <Location /http-service-2>
    ...
  </Location>

</VirtualHost>

Following problems arise with the current implementation of RequestReadTimeout.

* The timeout/data rate requirements of the services may differ, so the least restricting timings have to be used for all locations
* mod_proxy_wstunnel in its current implementation completely removes mod_reqtimeout from the list of input filters in order to avoid interceptions due to timeouts or unaccomplished data rates. Hence, unused connections (e.g. crashed client) are never terminated. 
* There is no possiblity to make data rate requirements to WS clients, e.g. well-known application with predictable data rate

Assuming that mod_proxy_wstunnel does not remove the mod_reqtimeout input filter following configuration would be desirable for this use case.
 
<VirtualHost>
  RequestReadTimeout header=10-30 body=10

  <Location /client.js>
    ProxyPass http://wshost:11111/
    ProxyPassReverse http://wshost:11111/
  </Location>

  <Location /websocket>
    RequestReadTimeout body=3600

    ProxyPass ws://wshost:22222/
    ProxyPassReverse  ws://wshost:22222/
  </Location>

  <Location /http-service-1>
    # de-activation
    RequestReadTimeout body=0
    ...
  </Location>

  <Location /http-service-2>
    RequestReadTimeout body=10-7200,DataRate=50
    ...
  </Location>

</VirtualHost>

Note that for a location only the "body settings" can be adjusted.

Approach
********

For the stated reasons, I would like to provide a patch that implements the adjustment of the body timeout and body data rate per location.

The patch uses the post_perdir_config hook (instead of post_read_request) in order to get access to the location configuration. Based on this configuration the timeout and data rate requirement are adjusted for the receiving of the body data.
Comment 1 Eric Covener 2015-01-13 14:31:26 UTC 
> Hence, unused connections (e.g. crashed client) are never terminated.

wstunnel shouldn't depend on a timeout to see that a client has closed their end of the socket.  It should be woken up when the application crash triggers the socket to be closed by the clients host OS.
Comment 2 Micha Lenk 2015-01-13 15:34:50 UTC 
Eric, that's not the point. mod_reqtimeout isn't used to enforce timeouts
on cooperative clients but on (intentionally or unintentionally) uncooperative
clients.

I agree with Armin that HTTP clients might need different timeout settings than
WebSocket clients. With the current mod_reqtimeout implementation this is
impossible if HTTP clients and WebSocket clients share the same virtual server.
So there is a need to address that. Do you agree?
Comment 3 Eric Covener 2015-01-13 15:56:40 UTC 
(In reply to Micha Lenk from comment #2)
> Eric, that's not the point. mod_reqtimeout isn't used to enforce timeouts
> on cooperative clients but on (intentionally or unintentionally)
> uncooperative clients.

I'm referring specifically to the quoted text about a crashed client.  

> I agree with Armin that HTTP clients might need different timeout settings
> than WebSocket clients. With the current mod_reqtimeout implementation this is
> impossible if HTTP clients and WebSocket clients share the same virtual
> server So there is a need to address that. Do you agree?

I agree that different body timeouts per-directory are useful for HTTP and it doesn't seem problematic, but I haven't thought enough about what it means for websockets. In trunk, there is already an idle timeout available, I am not sure if mod_reqtimeout could work in that case (it gets in the way when there is I/O, not when the connection is idle because there are no reads occurring)
Comment 4 Armin Abfalterer 2015-01-19 14:47:28 UTC 
(In reply to Eric Covener from comment #3)
> I'm referring specifically to the quoted text about a crashed client.  

With "crashed client" I actually meant half-open connections due to a client OS crash.

> I am not sure if mod_reqtimeout could work in that case (it gets in the way
> when there is I/O, not when the connection is idle because there are no
> reads occurring)

mod_reqtimeout assigns the wanted timeout to the socket, thus, there must not necessarily be any I/O
Comment 5 Eric Covener 2015-01-19 15:09:31 UTC 
(In reply to Armin Abfalterer from comment #4)
> (In reply to Eric Covener from comment #3)
> > I'm referring specifically to the quoted text about a crashed client.  
> 
> With "crashed client" I actually meant half-open connections due to a client
> OS crash.

Okay, I see.

> 
> > I am not sure if mod_reqtimeout could work in that case (it gets in the way
> > when there is I/O, not when the connection is idle because there are no
> > reads occurring)
> 
> mod_reqtimeout assigns the wanted timeout to the socket, thus, there must
> not necessarily be any I/O

I don't think that tiemout affects what you want when the caller is mostly stuck in poll/select/epoll with their own timeout passed in.  You'd have to get lucky and have a read/write pending but that should be rare in wstunnel.
Comment 6 Eric Covener 2015-01-19 15:12:11 UTC 
I should elaborate, because wstunnel is non-HTTP it cannot block in either direction (where the APR socket timeout is used), it needs to poll on both for read/write.
Comment 7 Micha Lenk 2015-09-18 11:11:07 UTC 
Created attachment 33116 [details]
Allow to change body timeout per location

I stripped some renamed variables from the patch that caused some extra noise.

Additionally the patch is now based on SVN branch 2.4.x, rev. 1703807.

It was successfully tested with Apache 2.4.16.