Bug 45023 - DEFLATE preventing 304 NOT MODIFIED response
Summary: DEFLATE preventing 304 NOT MODIFIED response
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_deflate (show other bugs)
Version: 2.2.8
Hardware: All All
: P2 regression with 20 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
Keywords: RFC
: 56354 (view as bug list)
Depends on: 39727
  Show dependency tree
Reported: 2008-05-16 23:53 UTC by Noah
Modified: 2018-09-12 01:03 UTC (History)
9 users (show)

A POC slightly better than the above conf lines, (2.68 KB, patch)
2008-08-19 06:41 UTC, rahul
Details | Diff
use ap_hook_post_read_request instead of input filter (2.12 KB, patch)
2008-08-21 08:13 UTC, rahul
Details | Diff
mod_deflate as found in apache2-src package on ubuntu version 2.2.8-1ubuntu0.3 (47.88 KB, text/plain)
2008-09-18 11:10 UTC, Sidharth Kshatriya

Note You need to log in before you can comment on or make changes to this bug.
Description Noah 2008-05-16 23:53:11 UTC
On my server, I recently added the DEFLATE filter for all application/x-javascript files, and noticed that apache has stopped sending 304 NOT MODIFIED on ALL my js files *always*.

Observe this header trace:

GET /main.js HTTP/1.1
Host: xxxxxxxxxxxxx
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/20080404 Firefox/
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
If-Modified-Since: Mon, 12 May 2008 16:09:00 GMT
If-None-Match: "44d0ac3fd1f00"-gzip
Cache-Control: max-age=0

HTTP/1.x 200 OK
Date: Sat, 17 May 2008 06:09:25 GMT
Server: Apache
Last-Modified: Mon, 12 May 2008 16:09:00 GMT
Etag: "44d0ac3fd1f00"-gzip
Accept-Ranges: bytes
Cache-Control: max-age=86400, must-revalidate, private
Expires: Sun, 18 May 2008 06:09:25 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 9797
Keep-Alive: timeout=2, max=299
Connection: Keep-Alive
Content-Type: application/x-javascript

It seems to me, that a 304 NOT MODIFIED would have been the appropriate response here.

Before I added DEFLATE to javascript files, the above response was always 304 NOT MODIFIED (except the first one).
Now with deflate, it is *always* 200 OK.

Is this by design, or is this a bug?

One thing to mention is - I have both mod_deflate and mod_expires statically compiled into httpd.
Comment 1 Takashi Sato 2008-05-17 00:22:47 UTC
If we remove "-gzip" from Etag for If-None-Match, a server returns 304.
mod_deflate adds "-gzip" and sends it to us, but condition check of If-None-Match runs before mod_deflate ...???
Comment 2 Nick Kew 2008-05-17 06:47:55 UTC
This is actually a fix of Bug 39727.  Yes, we know the core code testing for conditional responses still needs work.
Comment 3 John Siracusa 2008-06-24 07:32:39 UTC
This bug nearly defeats the purpose of mod_deflate.  I turned mod_deflate on with the intent of saving bandwidth when sending JavaScript and CSS files.  Instead, I'm using *more* bandwidth for every request but the first.  (Previously, subsequent requests would result in small, no-content 304 responses.  Now the compressed JS and CSS files are sent every time.)

BTW, the change that appends "-gzip" to the etag (revision 607219, I think) is a bit wacky in its own right in that it appends outside the double quotes.  IOW, if the original header is:

    Etag: "5954c6-10f4-449d11713aac0"

the modified header ends up as:

    Etag: "5954c6-10f4-449d11713aac0"-gzip

when it probably should be:

    Etag: "5954c6-10f4-449d11713aac0-gzip"

with the "-gzip" inside the quotes.  I altered the code to do this, but it had no affect on this 304 bug.  I just wanted to note it here so the situation is avoided when this bug is fixed for real.
Comment 4 rahul 2008-08-19 05:36:44 UTC
There is a workaround, 
If you are serving from a location say /js
You can use a configuration like below to switch ETag: $1-gzip to $1 

<Location /js>
RequestHeader  edit "If-None-Match" "^(.*)-gzip$" "$1"
Header  edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip"

(Perhaps this should be done from mod_deflate as input filter
with the condition that if Client requests with Accept-Encoding: gzip
only then modify If*match headers to their un-gzip values.)

(Also the current value of ETag (+gzip) is not RFC compliant, since RFC mandates
a quoted string as the value of ETag. As the noted by john, -gzip should be inside quotes.)
Comment 5 rahul 2008-08-19 06:41:51 UTC
Created attachment 22453 [details]
A POC slightly better than the above conf lines,

Adds an input filter ETAG that can be used like below, along with DEFLATE
to let apache recognize ($1)-gzip as $1

AddInputFilter ETAG .txt
AddOutputFilterByType DEFLATE text/plain
Comment 6 John Siracusa 2008-08-19 07:47:26 UTC
Both the workaround in comment #4 and the patch in comment #5 appear to work.  Neither are real "fixes" for the bug, however.  (But I'm glad I have something to work with until a real fix arrives, so thanks rahul!)
Comment 7 rahul 2008-08-21 08:13:07 UTC
Created attachment 22468 [details]
use ap_hook_post_read_request instead of input filter

Incorporate suggested changes by Nick
Comment 8 Sidharth Kshatriya 2008-09-18 08:23:02 UTC

I'm also facing the same problem -- I had been scratching my head for sometime before I found this bug report. I have to reduce my page size urgently and not having gzip on is not really an option for me.

I'm not really comfortable having a customized compiled version of apache2 on my system and the regular expression "hack" will also slow things down (will it?).

Any alternative ideas? When might this bug be fixed?


Comment 9 Sidharth Kshatriya 2008-09-18 08:27:57 UTC
I had a look at ap_hook_post_read_request attachment...seems like I can get away with just compiling mod_deflate.c

I'm using Apache 2.2.8. Can anyone help me on where I can get the correct source for the corresponding mod_deflate and how I can compile mod_deflate?

Thanks for your help!

Comment 10 Sidharth Kshatriya 2008-09-18 09:19:28 UTC

Sorry for flooding this mailing list.

In this thread there is Rahul's mod_deflate patch and there are some patches in this bug thread:

(mostly by Nick Kew)

Which one is the best solution?



Comment 11 rahul 2008-09-18 10:27:15 UTC
You will need both. This patch expects the other to have been applied (this is already applied on trunk).
Also please use the dev@httpd.apache.org for questions.

Comment 12 Sidharth Kshatriya 2008-09-18 11:03:49 UTC
Thanks for your response Rahul.

Unfortunately I am new to all of this. I applied Nick's patch to my mod_deflate.c but I got patching errors...his patch is assuming a slightly different mod_deflate.c from mine.

Would really appreciate if you were able to direct me to the place (or even better just gave me the patched mod_deflate.c version) that I could just compile.

[ASIDE: I am using apxs2 -c mod_deflate.c

with LDFLAGS="-lz" set in /usr/bin/aprconfig

I notice that my original mod_deflate.so depends on libpthread. When I apply your patch to my file and compile, I don't get dependency with libpthread.]

Please tell me if I would need to do any additional transformations to the file you would provide me (assuming you can :-) ).

Thanks a Ton!

[Attaching the mod_deflate that I have. This I obtained by issuing
sudo apt-get install apache2-src 
on ubuntu. This is version 2.2.8 Apache]
Comment 13 Sidharth Kshatriya 2008-09-18 11:10:32 UTC
Created attachment 22605 [details]
mod_deflate as found in apache2-src package on ubuntu version 2.2.8-1ubuntu0.3
Comment 14 Roy T. Fielding 2009-04-03 18:41:47 UTC
This bug was added in 2.2.8 in a failed attempt to address Bug 39727.
Since a real fix will require extensive changes, I have reverted the
change in 


Note: if you are patching a released version of the source code,
then extract the patch from the original change


and use patch -R to apply it in reverse.

Meanwhile, the original bug will still be tracked as issue 39727.
Comment 15 hackeron 2009-04-13 11:25:53 UTC
I'm still experiencing this with 2.2.11-2ubuntu2 -- but adding these 2 lines are suggested by rahul works:

RequestHeader  edit "If-None-Match" "^(.*)-gzip$" "$1"
Header  edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip"
Comment 16 Eric Covener 2013-06-23 17:41:44 UTC
Was this ever resolved in trunk/2.4?

Alias /deflate /home/covener/SRC/httpd-trunk/built/htdocs/index.html
<Location /deflate>
SetOutputFilter DEFLATE

$ wget  --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1 |grep ETag
  ETag: "58c5-4b26c6f28ce80-gzip"

$ wget --header='If-None-Match: "58c5-4b26c6f28ce80-gzip"'  --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1
--2013-06-23 13:39:47--  http://localhost/deflate
Resolving localhost (localhost)...
Connecting to localhost (localhost)||:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Sun, 23 Jun 2013 17:39:47 GMT
  Server: Apache/2.5.0-dev (Unix) OpenSSL/1.0.1c
  Last-Modified: Wed, 23 Nov 2011 20:04:58 GMT
  ETag: "58c5-4b26c6f28ce80-gzip"
  Accept-Ranges: bytes
  Vary: Accept-Encoding
  Content-Encoding: gzip
  Content-Length: 206
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: text/html

covener@cov-w520:~/SRC/httpd-trunk$ wget --header='If-None-Match: "58c5-4b26c6f28ce80"'  --header="Accept-Encoding: gzip" -S http://localhost/deflate -O/dev/null 2>&1|grep HTTP/1.1
  HTTP/1.1 304 Not Modified
Comment 17 Eric Covener 2014-04-11 02:43:20 UTC
It's a copout, but I've made this configurable in r1586542 with a default for now of maintaining the 2.4 behavior.
Comment 18 Yann Ylavic 2014-04-16 11:27:53 UTC
*** Bug 56354 has been marked as a duplicate of this bug. ***
Comment 19 Jesús Cea 2014-04-23 21:27:46 UTC
Eric, I am graceful for the new directive, but why not implementing a better ETAG verification instead of altering the ETAG generation?.

If you get an etag with a "-gzip" suffix, you just verify the etag without that suffix. You could link this to the "accept-encoding" header, maybe.
Comment 20 Eric Covener 2014-04-23 21:38:26 UTC
(In reply to Jesús Cea from comment #19)
> Eric, I am graceful for the new directive, but why not implementing a better
> ETAG verification instead of altering the ETAG generation?.

My immediate concern is the unnecessary difference between 2.2 and 2.4, allowing the 2.2 behavior in 2.4 is a lot simpler to tackle then adding some third behavior (I admitted before it was a copout)
Comment 21 Martin Heide 2015-02-13 08:10:40 UTC
(In reply to Eric Covener from comment #17)
> It's a copout, but I've made this configurable in r1586542 with a default
> for now of maintaining the 2.4 behavior.

Hi Eric,
will your workaround of r1586542 make it into some Apache 2.4 release? I saw that it 
is integrated in 2.5, but it is not mentioned in the 2.4 docs.
I tried the DeflateAlterETag directive, but it does not seem to available in Ubuntu 14.04 LTS (Apache 2.4.7).

Since you only added a directive, this cannot break any existing server configurations, so it should be available in 2.4, too!
Comment 22 joost.dekeijzer 2015-11-08 22:00:51 UTC
Am I correct to read in http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.26 that the If-None-Match header can contain mulitple ETAG values?

If so, maybe an alternative version of rahuls workaround could be:

RequestHeader edit "If-None-Match" '^"((.*)-gzip)"$' '"$1", "$2"'

With above line, I think there is no need to modify the outgoing Header and the modefied RequestHeader works in with or without deflation
Comment 23 Rainer Jung 2018-02-25 19:56:08 UTC
Undo spam change
Comment 24 Rainer Canavan 2018-09-11 16:11:16 UTC
Undo spam change.