Bug 43838 - If-Modified-Since request and htcacheclean conflict
Summary: If-Modified-Since request and htcacheclean conflict
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy (show other bugs)
Version: 2.2.3
Hardware: Other Linux
: P2 enhancement (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
Keywords: MassUpdate
Depends on:
Reported: 2007-11-11 15:39 UTC by Alec Davis
Modified: 2018-11-07 21:08 UTC (History)
0 users


Note You need to log in before you can comment on or make changes to this bug.
Description Alec Davis 2007-11-11 15:39:11 UTC
  Apache2.2.3 setup as a reverse proxy, in a test environment ready to replace 
a production box.

  Excessive 'If-Modified-Since' traffic to upstream server.

   Every night htcacheclean is run.
   A returning Client requests an object/file using 'If-Modifed-Since' in the 
header, as the client has locally cached copy.

   Reverse proxy makes 'If-Modifed-Since' request to upstream server as object 
is not available in cache. Upstream server will reply with '304 Not-Modifed'. 
Note, Proxy still doesn't have a copy. 

   The expected action would be for the proxy to get the object, and 
report '304 Not-Modifed' to the client, so that the next request is served out 
of cache, and no further traffic occurs between proxy and upstream server.

  If the clients cache is cleared, the next request comes from the upstream 
server, then subsequent requests come from the proxy's cache.

In mod-proxy-http.c we have removed the test 'if r->main' check (snippet 
below), which appears to remove the "If-*" headers to the upstream server, but 
this may have other side effects, but gives us the desired operation.

        /* for sub-requests, ignore freshness/expiry headers */
        // CJW - Ignore Freshness on requests
        //if (r->main) {
            if (    !strcasecmp(headers_in[counter].key, "If-Match")
                 || !strcasecmp(headers_in[counter].key, "If-Modified-Since")
                 || !strcasecmp(headers_in[counter].key, "If-Range")
                 || !strcasecmp(headers_in[counter].key, "If-Unmodified-Since")
                 || !strcasecmp(headers_in[counter].key, "If-None-Match")) {

How should this senario really be fixed?
Comment 1 Roy T. Fielding 2007-12-28 18:22:26 UTC
The right fix is a configuration directive that tells us the
proxy is supposed to remove conditionals on a GET when
the response is not already in the cache.  Not all sites are
concerned about reducing the number of 304 responses from the
backend. For example, your solution will mean that anything
that mod_cache has decided not to cache for other reasons, such
as being a low hit potential or simply too big, will result
in repeated full transfers from the backend instead of 304s.

Comment 2 Alec Davis 2008-01-28 17:24:55 UTC
Referring to  http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
section 10.3.5 304 Not Modified

"If a 304 response indicates an entity not currently cached, then the cache 
MUST disregard the response and repeat the request without the conditional"

Reading the RFC, suggests that no matter what, if the image isn't in cache it 
should be requested.

Our deployment is an offsite image-cache, with 1Gb connection to internet.

With our hack, I tend to agree with the concerns of large files (wheter 
modified or not) being repeatedly requested from the backend should the cache 
server decide not to cache it due to CacheMaxFileSize or some other reason.

The sites that are reverse proxied don't have large image files and represent 
~95% of our traffic, the other remaining sites that may have large images are 
not proxied - the traffic goes over our 10Mb link, versus 1Gb at ISP.

In general, the hack works for us, but don't want to be maintaining the code 
on an on going basis.
Is there any work being done on this?

Thanks Alec
Comment 3 William A. Rowe Jr. 2018-11-07 21:08:57 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.