Bug 21260 - CacheMaxExpire directive not enforced !
Summary: CacheMaxExpire directive not enforced !
Status: REOPENED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_cache (show other bugs)
Version: 2.0-HEAD
Hardware: All All
: P3 normal (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords: MassUpdate, PatchAvailable
Depends on:
Blocks:
 
Reported: 2003-07-02 03:43 UTC by Jean-Hugues ROYER
Modified: 2019-01-08 16:49 UTC (History)
3 users (show)

See Also:


Attachments
patch for 2.2.x (490 bytes, patch)
2007-01-03 07:01 UTC, Mike Williams
Details | Diff
Change CacheMaxExpire/CacheMinExpire doc to be more accurate (1.69 KB, patch)
2014-03-27 13:59 UTC, Edward Lu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jean-Hugues ROYER 2003-07-02 03:43:48 UTC
Hi,

The CacheMaxExpire directive is not enforced as stated in the documentation 
(quote: "This restriction is enforced even if an expiry date was supplied with 
the document").

After looking at the source of mod_cache.c you can see that the maxexpire value 
is only used IF the expiration date provided is bad AND the last modified date 
provided is good.

Basically if an expiration date is provided in the document fetched, the 
CacheMaxExpire value is ignored.

I assume it is a bug in the source rather than a bug in the documentation since 
I think that CacheMaxExpire directive must really be enforced...
Comment 1 Jean-Hugues ROYER 2003-07-02 23:11:43 UTC
I quickly fixed it using this code in mod_proxy.c:

Replacing:
    if (expc == BAD_DATE) {
...
    }
By:
   {
        double x = conf->cache.maxexpire;
        int y = 0;
        if (expc == BAD_DATE) {
             if (lmod != BAD_DATE)
                 x = (double)(date - lmod) * conf->cache.lmfactor;
             else
                 x = conf->cache.defaultexpire;
             y = 1;
        }
        else
             x = expc - now;
        if(x > conf->cache.maxexpire) {
             x = conf->cache.maxexpire;
             y = 1;
        }
        expc = now + (int)x;
        if(y)
             ap_log_error(APLOG_MARK, APLOG_DEBUG | APLOG_NOERRNO, r-
>server, "Expiry date calculated %ld", (long)expc);  
   }

Comment 2 Jeff Trawick 2003-11-21 18:17:47 UTC
Can you submit a real patch?  Please see 
http://httpd.apache.org/dev/patches.html
Comment 3 ApacheWorker 2005-07-02 01:17:04 UTC
Has this bug been fixed in 2.0.x?   
 
Thanks. 
Comment 4 Dick Snippe 2006-04-26 14:07:31 UTC
(In reply to comment #3)
> Has this bug been fixed in 2.0.x?   
>  
> Thanks. 

I'm using 2.2.0 and it appears that the bug hasn't been fixed.
Currently I have a setup with CacheMaxExpire 600, but I get answers from my
cache with "Age: 2294", so that can't be right.

It would appear that cache_util doesn't honour CacheMaxExpire at all. When
looking at modules/cache/cache_util.c, around line 298, I see:
    /* handle expiration */
    if (((smaxage != -1) && (age < (smaxage - minfresh))) ||
        ((maxage != -1) && (age < (maxage + maxstale - minfresh))) ||
        ((smaxage == -1) && (maxage == -1) &&
         (info->expire != APR_DATE_BAD) &&
         (age < (apr_time_sec(info->expire - info->date) + maxstale - minfresh)))) {
        const char *warn_head;

        warn_head = apr_table_get(h->resp_hdrs, "Warning");

        /* it's fresh darlings... */

i.e. nothing relating to CacheMaxExpire

The only references to CacheMaxExpire are in mod_cache.c, line 692 where Expires
headers are created based on last modified headers.
Comment 5 Mike Williams 2007-01-03 07:01:58 UTC
Created attachment 19344 [details]
patch for 2.2.x

If conf->maxex is set, and age is greater in seconds, expire the content.
"if (conf->maxex != -1)" doesn't seem to be needed, as without CacheMaxExpire
defined conf->maxex is 86400 (1 day).
Comment 6 Nick Kew 2007-09-01 12:15:33 UTC
looks like mod_cache
Comment 7 Graham Leggett 2009-10-03 09:01:38 UTC
CacheMaxExpire as used in this patch seems to limit the max cache time to a default of 86400 seconds (1 day), which is broken - caches should definitely be allowed to cache for more than a day.

Looking at CacheMinExpire, the directive is only valid when an Expires header is not present (as per the docs), and currently, CacheMaxExpire is also only valid when the expires header is missing.

Either the docs need to be changed, to reflect this, or the default value of CacheMaxExpire needs to default to no limit.
Comment 8 Niklas Edmundsson 2011-09-13 09:39:54 UTC
(In reply to comment #7)
> CacheMaxExpire as used in this patch seems to limit the max cache time to a
> default of 86400 seconds (1 day), which is broken - caches should definitely be
> allowed to cache for more than a day.

Both behaviors are probably wanted/needed by different users.

For us that have content that _might_ change we can live with a freshness-check once a day in order to have the cache deliver stale content for a day maximum (or whatever you set CacheMaxExpire to).

Others might want to have the current code behavior, I personally prefer the documented one.

If I remember correctly passing the cache expire triggers a freshness check, and if it's not-modified we're content with the cached body and write a new header.
Doing this once a day by default is sane IMHO, no matter what Expires-header is passed by the upper layer.
Comment 9 abiacco 2011-09-13 17:55:37 UTC
(In reply to comment #8)

> 
> If I remember correctly passing the cache expire triggers a freshness check,
> and if it's not-modified we're content with the cached body and write a new
> header.
> Doing this once a day by default is sane IMHO, no matter what Expires-header is
> passed by the upper layer.

But i don't think that's true.

Because let's say i have a file with a max-age value of 4 hours and a cachemaxexpire setting with a value of 5 minutes. I've had old file content being served from the cache (i.e. having an Age header > CacheMaxExpire and < max-age) when new file content existed on the FS.
Comment 10 Edward Lu 2014-03-27 13:59:18 UTC
Created attachment 31447 [details]
Change CacheMaxExpire/CacheMinExpire doc to be more accurate

This behavior is still around today, in trunk. To summarize the discussion:

CacheMaxExpire/CacheMinExpire only take effect if no valid expiry date is specified and a valid last modified date is specified. Additionally, they are unconditionally overriden by Cache-Control header directives. One can see this behavior by following the code paths in mod_cache.c (cache_save_filter(), do a search on "maxex") and cache_util.c (cache_check_freshness(), called from cache_select() in cache_storage.c).

The documentation, on the other hand, says that CacheMaxExpire overrides the expiry date supplied, and says pretty much nothing as to whether it affects the Cache-Control directives or not. CacheMinExpire says "This is only used if no valid expire time was supplied with the document", which is actually true, but ignores the fact that a valid Last-Modified value must be supplied as well.

There's also this confusing case in cache_save_filter(): if there's a max-age and a last-modified date but no valid expiry date, CacheMaxExpire/CacheMinExpire try to do something by setting the expiry date field (what would normally get set by the Expires header). However, that field just gets overriden by max-age anyway in cache_check_freshness(), so nothing is accomplished.

This behavior's apparently been around for more than ten years, so I've taken a shot at changing the documentation only. However, I do also feel that these directives are not very useful as they are currently implemented.
Comment 11 William A. Rowe Jr. 2018-11-07 21:08:08 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.
Comment 12 jfclere 2019-01-08 16:49:36 UTC
The doc says "This maximum value is enforced even if an expiry date was supplied with the document." that IS NOT the case, the Expires header for example overwrites the CacheMaxExpire directives (Expires is used).