| Summary: | mod_cache returns entire entity from origin server when partial entity requested, in violation of RFC2616 SHOULD clause | ||
|---|---|---|---|
| Product: | Apache httpd-2 | Reporter: | Robin Battey <apache> |
| Component: | mod_cache | Assignee: | Apache HTTPD Bugs Mailing List <bugs> |
| Status: | RESOLVED LATER | ||
| Severity: | normal | CC: | antoine.prevosto |
| Priority: | P2 | Keywords: | MassUpdate |
| Version: | 2.2.14 | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Linux | ||
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd. As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd. If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question. If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with. Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated. |
RFC 2616, section 14.35.3 Range Retrieval Requests states: If a proxy that supports ranges receives a Range request, forwards the request to an inbound server, and receives an entire entity in reply, it SHOULD only return the requested range to its client. It SHOULD store the entire received response in its cache if that is consistent with its cache allocation policies. This does not occur. Instead, if the entity is expired or uncached, and the origin server is contacted with the range request and returns the entire entity, the entire entity is forwarded to the client. I have tested this with mod_proxy_http/mod_disk_cache, md_proxy_http/mod_mem_cache, mod_proxy_ajp/mod_disk_cache, and mod_proxy_ajp/mod_mem_cache, all with similar results. All combinations were tried both with an empty cache, an unexpired entity in the cache, and with an expired entity in the cache. In all cases, the empty and expired entity attempts returned the entire response instead of the requested partial entity, and the unexpired entity returned only the requested range. I am running ubuntu 10.04.3 LTS, running the Apache 2.2.14 package in the Ubuntu repositories: zanfur@rbatteylaptop:~$ lsb_release -d Description: Ubuntu 10.04.3 LTS zanfur@rbatteylaptop:~$ apache2 -V Server version: Apache/2.2.14 (Ubuntu) Server built: Sep 1 2011 09:52:34 Server's Module Magic Number: 20051115:23 Server loaded: APR 1.3.8, APR-Util 1.3.9 Compiled using: APR 1.3.8, APR-Util 1.3.9 Architecture: 64-bit Server MPM: Worker threaded: yes (fixed thread count) forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/worker" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="" -D SUEXEC_BIN="/usr/lib/apache2/suexec" -D DEFAULT_PIDLOG="/var/run/apache2.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types" -D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf" Although this is not the latest version, I have gone through the diff of the modules/cache directory between versions 2.2.14 and 2.2.21 (current), and there were no changes remotely relating to this behavior. This bug makes the proxy useless for, for instance, mobile phones attempting to resume a download failed due to a network blip over a slow network. Here are the debug cache logs for an attempt to get a range request for a cached entity that has expired with mod_mem_cache: [Wed Sep 21 17:23:51 2011] [debug] cache_storage.c(272): Cached response for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk isn't fresh. Adding/replacing conditional request headers. [Wed Sep 21 17:23:51 2011] [debug] mod_cache.c(131): Adding CACHE_SAVE filter for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk [Wed Sep 21 17:23:51 2011] [debug] mod_cache.c(138): Adding CACHE_REMOVE_URL filter for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk [Wed Sep 21 17:23:51 2011] [debug] mod_cache.c(663): cache: Caching url: /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk?msisdn=918791555016&cc=69650117&icid=22&deviceTypeName=Samsung%20GT-I9000_ext&imei=Samsung%20GT-I9000_ext_FAKE_918791555016&seg1=3&format=html [Wed Sep 21 17:23:51 2011] [debug] mod_cache.c(669): cache: Removing CACHE_REMOVE_URL filter. [Wed Sep 21 17:23:58 2011] [info] mem_cache: Cached url: http://tatadasfprod.tatadocomo.com:80/http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk? Similarly for mod_disk_cache: [Wed Sep 21 17:29:00 2011] [debug] mod_disk_cache.c(758): disk_cache: Recalled headers for URL http://tatadasfprod.tatadocomo.com:80/http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk? [Wed Sep 21 17:29:00 2011] [debug] cache_storage.c(272): Cached response for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk isn't fresh. Adding/replacing conditional request headers. [Wed Sep 21 17:29:00 2011] [debug] mod_cache.c(131): Adding CACHE_SAVE filter for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk [Wed Sep 21 17:29:00 2011] [debug] mod_cache.c(138): Adding CACHE_REMOVE_URL filter for /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk [Wed Sep 21 17:29:01 2011] [debug] mod_cache.c(663): cache: Caching url: /http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk?msisdn=918791555016&cc=69650117&icid=22&deviceTypeName=Samsung%20GT-I9000_ext&imei=Samsung%20GT-I9000_ext_FAKE_918791555016&seg1=3&format=html [Wed Sep 21 17:29:01 2011] [debug] mod_cache.c(669): cache: Removing CACHE_REMOVE_URL filter. [Wed Sep 21 17:29:01 2011] [debug] mod_disk_cache.c(977): disk_cache: Stored headers for URL http://tatadasfprod.tatadocomo.com:80/http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk? [Wed Sep 21 17:29:09 2011] [debug] mod_disk_cache.c(1066): disk_cache: Body for URL http://tatadasfprod.tatadocomo.com:80/http/skserver/downloadClient.jsp/SK_918791555016_69650117.apk? cached. Steps to reproduce are below, hopefully with enough information. If more information is required, and and ye shall receive. HOW TO REPRODUCE Configure Apache to load the following modules: * mod_proxy * mod_proxy_http or mod_proxy_ajp * mod_cache * mod_cache_disk or mod_cache_mem Find a link that ignores Range headers in requests. Configure Apache so that it does the following: * Reverse proxies the above link using a ProxyPass or Rewrite [P] rule * Caches successful downloads of the link (sizes in range, etc) Request the entity in the above link with a valid Range request (wget -c is useful for this, though I just typed it in a "telnet localhost 80" window) Watch it get the entire entity for you instead. (no entity in cache) Request it again, with the exact same request. Watch it only give you the requested range. (unexpired entity in cache) Wait for the entity to expire (I manually set the expiry time to 60 seconds, so I didn't have to wait long...) Request it again, exactly the same as the previous two times. Watch as the entire entity is downloaded again. (expired entity in cache)