Bug 43191 - compressableMimeType attribute ignored
Summary: compressableMimeType attribute ignored
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 5
Classification: Unclassified
Component: Connector:HTTP (show other bugs)
Version: 5.5.20
Hardware: All Windows XP
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
: 37834 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-08-22 15:02 UTC by Dave Wood
Modified: 2008-07-30 13:24 UTC (History)
3 users (show)



Attachments
patch vs. tc6.0.x/trunk (598 bytes, patch)
2007-08-24 11:05 UTC, Len Popp
Details | Diff
patch for 5.5 (631 bytes, patch)
2007-08-27 06:39 UTC, Len Popp
Details | Diff
GZIP compression filter (9.75 KB, text/x-java)
2008-07-01 10:12 UTC, Jonathan Leech
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dave Wood 2007-08-22 15:02:44 UTC
I have set compression="on" and compressableMimeType="text/html" in server.xml.
It appears that the value of compressableMimeType is being ignored. My XML
responses are all coming back gzipped. If I switch to compression="off" the
responses are not gzipped, but as soon as I turn it on, regardless of the value
of compressableMimeType, the XML responses are coming back gzipped.

A full Connector tag follows. Note that this is just an example -- I have tried
many values for compressableMimeType (that do NOT include text/xml) and the
responses are still compressed.

      <Connector port="7630" address="${jboss.bind.address}"
         maxThreads="20" minSpareThreads="4" maxSpareThreads="10" strategy="ms"
maxHttpHeaderSize="8192"
         emptySessionPath="true" enableLookups="false" redirectPort="7643"
acceptCount="16"
         connectionTimeout="20000" disableUploadTimeout="true"
         compression="off"
        
compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript"
/>
Comment 1 Len Popp 2007-08-24 08:13:19 UTC
The compressableMimeType isn't exactly ignored. It lets you add new MIME types
for compression, but there doesn't seem to be any way to remove the default set
of types.

That is, if you specify compressableMimeType="text/html" you effectively end up
with compressableMimeType="text/html,text/xml,text/plain,text/html".

I have verified the problem in 5.5.23 and 6.0.trunk. It looks easy to fix, I'll
give it a try.
Comment 2 Len Popp 2007-08-24 11:05:29 UTC
Created attachment 20702 [details]
patch vs. tc6.0.x/trunk

Patch for 6.0.trunk. Connector attribute compressableMimeType is the complete
list of compressable MIME types, instead of being added to the default list.
Comment 3 Len Popp 2007-08-27 06:39:56 UTC
Created attachment 20713 [details]
patch for 5.5

Here's the same patch for 5.5, since that's what the bug was reported against.
Comment 4 Peter Rossbach 2007-08-28 06:10:38 UTC
If seems that this is a good idea, but can be a regression with older tomcat versions :-(

Peter
Comment 5 Mark Thomas 2007-08-29 04:48:14 UTC
*** Bug 37834 has been marked as a duplicate of this bug. ***
Comment 6 Len Popp 2007-08-31 06:37:53 UTC
(In reply to comment #4)

As it is, there is no way to turn off gzip compression for one file type while
keeping it for others. That seems more serious than the possible regression,
which is easy to fix by setting compressableMimeType correctly. (And are there
really a lot of users that are using it incorrectly? The documentation is quite
clear.)

Alternatively, I guess you could add a new attribute that does what
compressableMimeType was supposed to do...
Comment 7 Mark Thomas 2008-05-03 03:26:17 UTC
I have applied your patch to trunk (expanded to include the other connectors) and proposed it for 6.0.x and 5.5.x
Comment 8 Mark Thomas 2008-05-15 12:35:20 UTC
This has been applied to 6.0.x and will be included in 6.0.17 onwards.
Comment 9 Mark Thomas 2008-05-16 10:56:35 UTC
Opps - this is still open for 5.5.x
Comment 10 Richard Mundell 2008-06-16 05:44:43 UTC
I would like to share an alternative approach for this problem I just implemented which avoids having to update Tomcat (or patch it).

Grab CompressionFilter.java, CompressionResponseStream.java and CompressionServletResponseWrapper.java from the Tomcat source code distribution (I did this with the 5.5.25 source).

Open up CompressionServletResponseWrapper.java and modify the writetoGZip() method. On line 305, replace...

    response.addHeader("Content-Encoding", "gzip");
    gzipstream = new GZIPOutputStream(output);

...with...

    if ((response.getContentType()!=null) &&
         (response.getContentType().equals("application/pdf")) {

        // I want to compress this content type
        response.addHeader("Content-Encoding", "gzip");
        gzipstream = new GZIPOutputStream(output);
    } else {
        // Don't want to compress this content type
        gzipstream = output;
    }

Compile all three Java classes, then add the filter into your web.xml file:

    <filter>
        <filter-name>Compressing Filter</filter-name>
        <filter-class>compressionFilters.CompressionFilter</filter-class>
        <init-param>
            <param-name>compressionThreshold</param-name>
            <param-value>1</param-value>
        </init-param>
    </filter>
    <filter-mapping>
        <filter-name>Compressing Filter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>


This is very handy if you don't want to upgrade Tomcat, or for Tomcat 5.5.x users for who there isn't an upgrade with the patch available yet. Also useful if you don't want to compress server-wide for all of your web applications.

Enjoy!
Comment 11 Jonathan Leech 2008-07-01 10:11:24 UTC
For such a simple concept, every compression filter I have ever seen has serious problems. Mostly the problems are not addressed, and instead all kinds of finely grained mechanisms are created to more selectively apply compression. For example, if a response sets the content-length header, I have seen compression filters break. The browser either hangs waiting for the rest of the response, or the response is truncated to the length set. I don't remember if Tomcat's compression filter is guilty of this behavior or not, but I suspect the latter due to the existence of the compressionThreshold setting. 

I have created my own GZIPFilter which addresses all the problems I have seen in various other compression filter implementations. I am posting it as an attachment to this bug, feel free to use it, or not. Its advantages are captured in the comments, in summary, it streams rather than makes a copy, it handles content-length correctly, it detects client support of gzip compression, and it doesn't double compress. It is not problem-free, the known problems and limitations are also captured in the comments.

My theory on filter mapping is to keep it simple. So my filter doesn't care about mime-types, or any other qualification other compression filters use to decide whether to compress or not. As I stated before, it is my experience that these features exist to "fix" other problems in the filters.

The original poster can use my GZIPFilter in one of a few ways:
1) Use filter-mapping to selectively compress content.
2) Extend GZIPFilter, look at the mime-type header, and compress or not by calling super.doFilter().
3) Modify my GZIPFilter to look at mime-types.
Comment 12 Jonathan Leech 2008-07-01 10:12:26 UTC
Created attachment 22203 [details]
GZIP compression filter
Comment 13 Mark Thomas 2008-07-30 13:24:26 UTC
This is fixed for 5.5.x and will be included in 5.5.27 onwards.