Setup: Apache2.2.3 setup as a reverse proxy, in a test environment ready to replace a production box. Problem: Excessive 'If-Modified-Since' traffic to upstream server. Cause: Every night htcacheclean is run. A returning Client requests an object/file using 'If-Modifed-Since' in the header, as the client has locally cached copy. Symptom: Reverse proxy makes 'If-Modifed-Since' request to upstream server as object is not available in cache. Upstream server will reply with '304 Not-Modifed'. Note, Proxy still doesn't have a copy. The expected action would be for the proxy to get the object, and report '304 Not-Modifed' to the client, so that the next request is served out of cache, and no further traffic occurs between proxy and upstream server. Observation: If the clients cache is cleared, the next request comes from the upstream server, then subsequent requests come from the proxy's cache. Hack: In mod-proxy-http.c we have removed the test 'if r->main' check (snippet below), which appears to remove the "If-*" headers to the upstream server, but this may have other side effects, but gives us the desired operation. /* for sub-requests, ignore freshness/expiry headers */ // CJW - Ignore Freshness on requests //if (r->main) { if ( !strcasecmp(headers_in[counter].key, "If-Match") || !strcasecmp(headers_in[counter].key, "If-Modified-Since") || !strcasecmp(headers_in[counter].key, "If-Range") || !strcasecmp(headers_in[counter].key, "If-Unmodified-Since") || !strcasecmp(headers_in[counter].key, "If-None-Match")) { continue; } //} How should this senario really be fixed?
The right fix is a configuration directive that tells us the proxy is supposed to remove conditionals on a GET when the response is not already in the cache. Not all sites are concerned about reducing the number of 304 responses from the backend. For example, your solution will mean that anything that mod_cache has decided not to cache for other reasons, such as being a low hit potential or simply too big, will result in repeated full transfers from the backend instead of 304s.
Referring to http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html section 10.3.5 304 Not Modified "If a 304 response indicates an entity not currently cached, then the cache MUST disregard the response and repeat the request without the conditional" Reading the RFC, suggests that no matter what, if the image isn't in cache it should be requested. Our deployment is an offsite image-cache, with 1Gb connection to internet. With our hack, I tend to agree with the concerns of large files (wheter modified or not) being repeatedly requested from the backend should the cache server decide not to cache it due to CacheMaxFileSize or some other reason. The sites that are reverse proxied don't have large image files and represent ~95% of our traffic, the other remaining sites that may have large images are not proxied - the traffic goes over our 10Mb link, versus 1Gb at ISP. In general, the hack works for us, but don't want to be maintaining the code on an on going basis. Is there any work being done on this? Thanks Alec
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.