|Summary:||DEFLATE preventing 304 NOT MODIFIED response|
|Product:||Apache httpd-2||Reporter:||Noah <noah>|
|Component:||mod_deflate||Assignee:||Apache HTTPD Bugs Mailing List <bugs>|
|Severity:||regression||CC:||airmax, herr.ernst, jcea, peaceable_whale, sid.kshatriya, siracusa, szg0000, taco.vandenbroek, takashi.asfbugzilla|
|Bug Depends on:||39727|
A POC slightly better than the above conf lines,
use ap_hook_post_read_request instead of input filter
mod_deflate as found in apache2-src package on ubuntu version 2.2.8-1ubuntu0.3
Description Noah 2008-05-16 23:53:11 UTC
Comment 1 Takashi Sato 2008-05-17 00:22:47 UTC
If we remove "-gzip" from Etag for If-None-Match, a server returns 304. mod_deflate adds "-gzip" and sends it to us, but condition check of If-None-Match runs before mod_deflate ...???
Comment 2 Nick Kew 2008-05-17 06:47:55 UTC
This is actually a fix of Bug 39727. Yes, we know the core code testing for conditional responses still needs work.
Comment 3 John Siracusa 2008-06-24 07:32:39 UTC
Comment 4 rahul 2008-08-19 05:36:44 UTC
There is a workaround, If you are serving from a location say /js You can use a configuration like below to switch ETag: $1-gzip to $1 <Location /js> RequestHeader edit "If-None-Match" "^(.*)-gzip$" "$1" Header edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip" </Location> (Perhaps this should be done from mod_deflate as input filter with the condition that if Client requests with Accept-Encoding: gzip only then modify If*match headers to their un-gzip values.) (Also the current value of ETag (+gzip) is not RFC compliant, since RFC mandates a quoted string as the value of ETag. As the noted by john, -gzip should be inside quotes.)
Comment 5 rahul 2008-08-19 06:41:51 UTC
Created attachment 22453 [details] A POC slightly better than the above conf lines, Adds an input filter ETAG that can be used like below, along with DEFLATE to let apache recognize ($1)-gzip as $1 AddInputFilter ETAG .txt AddOutputFilterByType DEFLATE text/plain
Comment 6 John Siracusa 2008-08-19 07:47:26 UTC
Both the workaround in comment #4 and the patch in comment #5 appear to work. Neither are real "fixes" for the bug, however. (But I'm glad I have something to work with until a real fix arrives, so thanks rahul!)
Comment 7 rahul 2008-08-21 08:13:07 UTC
Created attachment 22468 [details] use ap_hook_post_read_request instead of input filter Incorporate suggested changes by Nick
Comment 8 Sidharth Kshatriya 2008-09-18 08:23:02 UTC
Hello! I'm also facing the same problem -- I had been scratching my head for sometime before I found this bug report. I have to reduce my page size urgently and not having gzip on is not really an option for me. I'm not really comfortable having a customized compiled version of apache2 on my system and the regular expression "hack" will also slow things down (will it?). Any alternative ideas? When might this bug be fixed? Thanks, Sidharth
Comment 9 Sidharth Kshatriya 2008-09-18 08:27:57 UTC
I had a look at ap_hook_post_read_request attachment...seems like I can get away with just compiling mod_deflate.c I'm using Apache 2.2.8. Can anyone help me on where I can get the correct source for the corresponding mod_deflate and how I can compile mod_deflate? Thanks for your help! Sidharth
Comment 10 Sidharth Kshatriya 2008-09-18 09:19:28 UTC
Hi, Sorry for flooding this mailing list. In this thread there is Rahul's mod_deflate patch and there are some patches in this bug thread: https://issues.apache.org/bugzilla/show_bug.cgi?id=39727 (mostly by Nick Kew) Which one is the best solution? Thanks, Sidharth
Comment 11 rahul 2008-09-18 10:27:15 UTC
You will need both. This patch expects the other to have been applied (this is already applied on trunk). Also please use the firstname.lastname@example.org for questions.
Comment 12 Sidharth Kshatriya 2008-09-18 11:03:49 UTC
Thanks for your response Rahul. Unfortunately I am new to all of this. I applied Nick's patch to my mod_deflate.c but I got patching errors...his patch is assuming a slightly different mod_deflate.c from mine. Would really appreciate if you were able to direct me to the place (or even better just gave me the patched mod_deflate.c version) that I could just compile. [ASIDE: I am using apxs2 -c mod_deflate.c with LDFLAGS="-lz" set in /usr/bin/aprconfig I notice that my original mod_deflate.so depends on libpthread. When I apply your patch to my file and compile, I don't get dependency with libpthread.] Please tell me if I would need to do any additional transformations to the file you would provide me (assuming you can :-) ). Thanks a Ton! Sidharth [Attaching the mod_deflate that I have. This I obtained by issuing sudo apt-get install apache2-src on ubuntu. This is version 2.2.8 Apache]
Comment 13 Sidharth Kshatriya 2008-09-18 11:10:32 UTC
Created attachment 22605 [details] mod_deflate as found in apache2-src package on ubuntu version 2.2.8-1ubuntu0.3
Comment 14 Roy T. Fielding 2009-04-03 18:41:47 UTC
This bug was added in 2.2.8 in a failed attempt to address Bug 39727. Since a real fix will require extensive changes, I have reverted the change in http://svn.apache.org/viewvc?view=rev&revision=761835 Note: if you are patching a released version of the source code, then extract the patch from the original change http://svn.apache.org/viewvc?view=rev&revision=608849 and use patch -R to apply it in reverse. Meanwhile, the original bug will still be tracked as issue 39727.
Comment 15 hackeron 2009-04-13 11:25:53 UTC
I'm still experiencing this with 2.2.11-2ubuntu2 -- but adding these 2 lines are suggested by rahul works: RequestHeader edit "If-None-Match" "^(.*)-gzip$" "$1" Header edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip"
Comment 16 Eric Covener 2013-06-23 17:41:44 UTC
Was this ever resolved in trunk/2.4? Alias /deflate /home/covener/SRC/httpd-trunk/built/htdocs/index.html <Location /deflate> SetOutputFilter DEFLATE </location> $ wget --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1 |grep ETag ETag: "58c5-4b26c6f28ce80-gzip" $ wget --header='If-None-Match: "58c5-4b26c6f28ce80-gzip"' --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1 --2013-06-23 13:39:47-- http://localhost/deflate Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Sun, 23 Jun 2013 17:39:47 GMT Server: Apache/2.5.0-dev (Unix) OpenSSL/1.0.1c Last-Modified: Wed, 23 Nov 2011 20:04:58 GMT ETag: "58c5-4b26c6f28ce80-gzip" Accept-Ranges: bytes Vary: Accept-Encoding Content-Encoding: gzip Content-Length: 206 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/html covener@cov-w520:~/SRC/httpd-trunk$ wget --header='If-None-Match: "58c5-4b26c6f28ce80"' --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1|grep HTTP/1.1 HTTP/1.1 304 Not Modified
Comment 17 Eric Covener 2014-04-11 02:43:20 UTC
It's a copout, but I've made this configurable in r1586542 with a default for now of maintaining the 2.4 behavior.
Comment 18 Yann Ylavic 2014-04-16 11:27:53 UTC
*** Bug 56354 has been marked as a duplicate of this bug. ***
Comment 19 Jesús Cea 2014-04-23 21:27:46 UTC
Eric, I am graceful for the new directive, but why not implementing a better ETAG verification instead of altering the ETAG generation?. If you get an etag with a "-gzip" suffix, you just verify the etag without that suffix. You could link this to the "accept-encoding" header, maybe.
Comment 20 Eric Covener 2014-04-23 21:38:26 UTC
(In reply to Jesús Cea from comment #19) > Eric, I am graceful for the new directive, but why not implementing a better > ETAG verification instead of altering the ETAG generation?. My immediate concern is the unnecessary difference between 2.2 and 2.4, allowing the 2.2 behavior in 2.4 is a lot simpler to tackle then adding some third behavior (I admitted before it was a copout)
Comment 21 Martin Heide 2015-02-13 08:10:40 UTC
(In reply to Eric Covener from comment #17) > It's a copout, but I've made this configurable in r1586542 with a default > for now of maintaining the 2.4 behavior. Hi Eric, will your workaround of r1586542 make it into some Apache 2.4 release? I saw that it is integrated in 2.5, but it is not mentioned in the 2.4 docs. I tried the DeflateAlterETag directive, but it does not seem to available in Ubuntu 14.04 LTS (Apache 2.4.7). Since you only added a directive, this cannot break any existing server configurations, so it should be available in 2.4, too!
Comment 22 joost.dekeijzer 2015-11-08 22:00:51 UTC
Am I correct to read in http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.26 that the If-None-Match header can contain mulitple ETAG values? If so, maybe an alternative version of rahuls workaround could be: RequestHeader edit "If-None-Match" '^"((.*)-gzip)"$' '"$1", "$2"' With above line, I think there is no need to modify the outgoing Header and the modefied RequestHeader works in with or without deflation
Comment 23 Rainer Jung 2018-02-25 19:56:08 UTC
Undo spam change
Comment 24 Rainer Canavan 2018-09-11 16:11:16 UTC
Undo spam change.
Comment 25 Arjen de Korte 2018-11-05 10:58:38 UTC
(In reply to Martin Heide from comment #21) > (In reply to Eric Covener from comment #17) > > It's a copout, but I've made this configurable in r1586542 with a default > > for now of maintaining the 2.4 behavior. > > Hi Eric, > will your workaround of r1586542 make it into some Apache 2.4 release? I saw > that it > is integrated in 2.5, but it is not mentioned in the 2.4 docs. > I tried the DeflateAlterETag directive, but it does not seem to available in > Ubuntu 14.04 LTS (Apache 2.4.7). > > Since you only added a directive, this cannot break any existing server > configurations, so it should be available in 2.4, too! *tap*tap*tap*, is this thing on? Why do we have BrotliAlterETag in 2.4, but DeflateAlterETag not and is r1586542 still waiting to be backported?
Comment 26 Jacques Distler 2019-02-15 07:53:41 UTC
With a minor modification, the workaround in Comment 22 works for both mod_deflate and mod_brotli : RequestHeader edit "If-None-Match" '^"((.*)-(gzip|br))"$' '"$1", "$2"' (using the default AddSuffix setting for both modules' ETag handling). Seems to me that this should just be incorporated into the "If-None-Match"-handling code in the server, rather than relying on users to monkey with their httpd.config file. But what do I know ?
Comment 27 Olivier Mehani 2019-08-02 00:16:02 UTC
I think I'm seeing the same issue with * Apache 2.4.39 * HTTP/2 * GZipped transfer When I first retrieve the URL, I get an ETag with '-gzip' appended. $ curl -I 'https://example.net/file.png' --compressed HTTP/2 200 date: Fri, 02 Aug 2019 00:12:51 GMT server: Apache/2.4.39 (Unix) LibreSSL/2.9.1 mod_chroot/0.5 last-modified: Fri, 02 Aug 2019 00:08:49 GMT etag: "11b104-4fa0-58f1729a20024-gzip" accept-ranges: bytes cache-control: max-age=31536000 expires: Sat, 01 Aug 2020 00:12:51 GMT vary: Accept-Encoding content-encoding: gzip strict-transport-security: max-age=15768000; content-length: 18360 content-type: image/png Any subsequent requests for that full ETag result in a 200 with the data being re-transfered. $ curl -I 'https://example.net/file.png' --compressed -H 'If-None-Match: "11b104-4fa0-58f1729a20024-gzip"' HTTP/2 200 date: Fri, 02 Aug 2019 00:12:55 GMT server: Apache/2.4.39 (Unix) LibreSSL/2.9.1 mod_chroot/0.5 last-modified: Fri, 02 Aug 2019 00:08:49 GMT etag: "11b104-4fa0-58f1729a20024-gzip" accept-ranges: bytes cache-control: max-age=31536000 expires: Sat, 01 Aug 2020 00:12:55 GMT vary: Accept-Encoding content-encoding: gzip strict-transport-security: max-age=15768000; content-length: 18360 content-type: image/png If I remove the '-gzip' suffix, I get the expected 304 response. $ curl -I 'https://example.net/file.png' --compressed -H 'If-None-Match: "11b104-4fa0-58f1729a20024"' HTTP/2 304 date: Fri, 02 Aug 2019 00:12:58 GMT server: Apache/2.4.39 (Unix) LibreSSL/2.9.1 mod_chroot/0.5 etag: "11b104-4fa0-58f1729a20024" expires: Sat, 01 Aug 2020 00:12:58 GMT cache-control: max-age=31536000
Comment 28 Olivier Mehani 2019-08-02 01:11:44 UTC
The updated solution from Comment 26 fixed the issue (didn't try Comment 22).
Comment 29 Rainer Canavan 2020-10-14 14:46:36 UTC
Undo spam change