Bug 33106 - SSI Processing Enhancements (patch provided)
SSI Processing Enhancements (patch provided)
Status: RESOLVED FIXED
Product: Tomcat 5
Classification: Unclassified
Component: Servlets:SSI
5.5.4
PC Windows XP
: P2 enhancement with 1 vote (vote)
: ---
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2005-01-14 22:31 UTC by David Becker
Modified: 2005-08-17 15:15 UTC (History)
1 user (show)



Attachments
diff -u results for all modified files (36.26 KB, patch)
2005-01-14 22:36 UTC, David Becker
Details | Diff
Fixed a null pointer bug if you failed to specify the contentTypes initParam (1.11 KB, patch)
2005-01-20 19:05 UTC, David Becker
Details | Diff
Final set of diffs (10.51 KB, patch)
2005-04-11 04:08 UTC, David Becker
Details | Diff
Full set of modified files (28.32 KB, application/x-bzip2)
2005-04-11 04:09 UTC, David Becker
Details
Final set of diffs (in gzip) (12.10 KB, patch)
2005-04-12 22:11 UTC, David Becker
Details | Diff
Full set of modified files (in gzip) (33.35 KB, application/x-gzip)
2005-04-12 22:11 UTC, David Becker
Details
catch exceptions for non-file based urls (848 bytes, patch)
2005-08-16 19:38 UTC, David Becker
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Becker 2005-01-14 22:31:50 UTC
I've made some enhancements to the SSI processing package as follows:

1) Created an SSIFilter to use instead of SSIServlet and modified support
classes accordingly.  SSIFilter takes a contentTypes initParam to filter out
results which shouldn't be processed and removes the buffered initParam since it
isn't relevant.

2) Modified support classes to allow SSIFilter and SSIServlet to update the
Last-Modified header with dates from processing directives and include files. 
(This is better than using the Expires header, which is still supported.)

3) Moved Globals.SSI_FLAG_ATTR into SSIProcessor to make it easier to use the
ssi package in other web-servers that don't support SSI processing.  Now there
are only three external dependencies (IOTools, Strftime, URLEncoder) which I
recommend be included in servlets-ssi.jar so someone can just grab that jar and
be done with it.  You may want to consider removing the SSI_FLAG_ATTR from
Globals since it doesn't seem to be used anywhere else.

Patches (diff -u) Follow:
--- ./ResponseIncludeWrapper.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/ResponseIncludeWrapper.java	Fri Jan 14 10:34:00 2005
@@ -13,6 +13,10 @@
 
 import java.io.IOException;
 import java.io.PrintWriter;
+import java.text.DateFormat;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+
 import javax.servlet.ServletOutputStream;
 import javax.servlet.http.HttpServletResponse;
 import javax.servlet.http.HttpServletResponseWrapper;
@@ -24,12 +28,17 @@
  * @version $Revision: 1.5 $, $Date: 2004/09/01 18:33:33 $
  */
 public class ResponseIncludeWrapper extends HttpServletResponseWrapper {
+    private static final String CONTENT_TYPE = "content-type";
+    private static final String LAST_MODIFIED = "last-modified";
+    private DateFormat lmParser = new SimpleDateFormat("EEE, d MMM yyyy
HH:mm:ss z");
     /**
      * Our ServletOutputStream
      */
     protected ServletOutputStream originalServletOutputStream;
     protected ServletOutputStream servletOutputStream;
     protected PrintWriter printWriter;
+    protected long lastModified = 0;
+    private String contentType = null;
 
 
     /**
@@ -98,5 +107,85 @@
             return servletOutputStream;
         }
         throw new IllegalStateException();
+    }
+
+
+    /**
+     * Returns the value of the <code>last-modified</code> header field. The
result is
+     * the number of milliseconds since January 1, 1970 GMT.
+     * 
+     * @return the date the resource referenced by this
<code>ResponseIncludeWrapper</code> was last modified, or 0 if not known.
+     */
+    public long getLastModified() {
+        return lastModified;
+    }
+    
+    
+    /**
+     * Returns the value of the <code>content-type</code> header field.
+     * 
+     * @return the content type of the resource referenced by this
<code>ResponseIncludeWrapper</code>, or <code>null</code> if not known.
+     */
+    public String getContentType()
+    {
+        return contentType;
+    }
+
+    
+    public void addDateHeader(String name, long value) {
+        super.addDateHeader(name, value);
+        String lname = name.toLowerCase();
+        if (lname.equals(LAST_MODIFIED)) {
+            lastModified = value;
+        }
+    }
+
+    
+    public void addHeader(String name, String value) {
+        super.addHeader(name, value);
+        String lname = name.toLowerCase();
+        if (lname.equals(LAST_MODIFIED)) {
+            try {
+                lastModified = lmParser.parse(value).getTime();
+            } catch (ParseException e) {
+                ;
+            }
+        }
+        else if (lname.equals(CONTENT_TYPE))
+        {
+            contentType = value;
+        }
+    }
+
+    
+    public void setDateHeader(String name, long value) {
+        super.setDateHeader(name, value);
+        String lname = name.toLowerCase();
+        if (lname.equals(LAST_MODIFIED)) {
+            lastModified = value;
+        }
+    }
+ 
+    
+    public void setHeader(String name, String value) {
+        super.setHeader(name, value);
+        String lname = name.toLowerCase();
+        if (lname.equals(LAST_MODIFIED)) {
+            try {
+                lastModified = lmParser.parse(value).getTime();
+            } catch (ParseException e) {
+                ;
+            }
+        }
+        else if (lname.equals(CONTENT_TYPE))
+        {
+            contentType = value;
+        }
+    }
+
+    public void setContentType(String value)
+    {
+        super.setContentType(value);
+        contentType = value;
     }
 }
--- ./SSICommand.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSICommand.java	Fri Jan 14 11:37:00 2005
@@ -34,10 +34,11 @@
      *            The parameter values
      * @param writer
      *            the writer to output to
+     * @return the most current modified date resulting from any SSI commands
      * @throws SSIStopProcessingException
      *             if SSI processing should be aborted
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer)
             throws SSIStopProcessingException;
 }
--- ./SSIConditional.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIConditional.java	Fri Jan 14 11:47:00 2005
@@ -23,9 +23,10 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer)
             throws SSIStopProcessingException {
+        long lastModified = System.currentTimeMillis();
         // Retrieve the current state information
         SSIConditionalState state = ssiMediator.getConditionalState();
         if ("if".equalsIgnoreCase(commandName)) {
@@ -33,7 +34,7 @@
             // except count it
             if (state.processConditionalCommandsOnly) {
                 state.nestingCount++;
-                return;
+                return lastModified;
             }
             state.nestingCount = 0;
             // Evaluate the expression
@@ -48,12 +49,12 @@
         } else if ("elif".equalsIgnoreCase(commandName)) {
             // No need to even execute if we are nested in
             // a false branch
-            if (state.nestingCount > 0) return;
+            if (state.nestingCount > 0) return lastModified;
             // If a branch was already taken in this if block
             // then disable output and return
             if (state.branchTaken) {
                 state.processConditionalCommandsOnly = true;
-                return;
+                return lastModified;
             }
             // Evaluate the expression
             if (evaluateArguments(paramNames, paramValues, ssiMediator)) {
@@ -68,7 +69,7 @@
         } else if ("else".equalsIgnoreCase(commandName)) {
             // No need to even execute if we are nested in
             // a false branch
-            if (state.nestingCount > 0) return;
+            if (state.nestingCount > 0) return lastModified;
             // If we've already taken another branch then
             // disable output otherwise enable it.
             state.processConditionalCommandsOnly = state.branchTaken;
@@ -80,7 +81,7 @@
             // one level on the nesting count
             if (state.nestingCount > 0) {
                 state.nestingCount--;
-                return;
+                return lastModified;
             }
             // Turn output back on
             state.processConditionalCommandsOnly = false;
@@ -93,6 +94,7 @@
             //throw new SsiCommandException( "Not a conditional command:" +
             // cmdName );
         }
+        return lastModified;
     }
 
 
--- ./SSIConfig.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIConfig.java	Fri Jan 14 11:48:00 2005
@@ -24,8 +24,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         for (int i = 0; i < paramNames.length; i++) {
             String paramName = paramNames[i];
             String paramValue = paramValues[i];
@@ -33,10 +34,13 @@
                     .substituteVariables(paramValue);
             if (paramName.equalsIgnoreCase("errmsg")) {
                 ssiMediator.setConfigErrMsg(substitutedValue);
+                lastModified = System.currentTimeMillis();
             } else if (paramName.equalsIgnoreCase("sizefmt")) {
                 ssiMediator.setConfigSizeFmt(substitutedValue);
+                lastModified = System.currentTimeMillis();
             } else if (paramName.equalsIgnoreCase("timefmt")) {
                 ssiMediator.setConfigTimeFmt(substitutedValue);
+                lastModified = System.currentTimeMillis();
             } else {
                 ssiMediator.log("#config--Invalid attribute: " + paramName);
                 //We need to fetch this value each time, since it may change
@@ -46,5 +50,6 @@
                 writer.write(configErrMsg);
             }
         }
+        return lastModified;
     }
 }
--- ./SSIEcho.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIEcho.java	Fri Jan 14 11:49:00 2005
@@ -28,8 +28,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         String encoding = DEFAULT_ENCODING;
         String errorMessage = ssiMediator.getConfigErrMsg();
         for (int i = 0; i < paramNames.length; i++) {
@@ -42,6 +43,7 @@
                     variableValue = MISSING_VARIABLE_VALUE;
                 }
                 writer.write(variableValue);
+                lastModified = System.currentTimeMillis();
             } else if (paramName.equalsIgnoreCase("encoding")) {
                 if (isValidEncoding(paramValue)) {
                     encoding = paramValue;
@@ -54,6 +56,7 @@
                 writer.write(errorMessage);
             }
         }
+        return lastModified;
     }
 
 
--- ./SSIExec.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIExec.java	Fri Jan 14 11:51:00 2005
@@ -15,6 +15,7 @@
 import java.io.IOException;
 import java.io.InputStreamReader;
 import java.io.PrintWriter;
+
 import org.apache.catalina.util.IOTools;
 /**
  * Implements the Server-side #exec command
@@ -33,16 +34,17 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         String configErrMsg = ssiMediator.getConfigErrMsg();
         String paramName = paramNames[0];
         String paramValue = paramValues[0];
         String substitutedValue = ssiMediator.substituteVariables(paramValue);
         if (paramName.equalsIgnoreCase("cgi")) {
-            ssiInclude.process(ssiMediator, "include",
-                    new String[]{"virtual"}, new String[]{substitutedValue},
-                    writer);
+            lastModified = ssiInclude.process(ssiMediator, "include",
+                               new String[]{"virtual"}, new
String[]{substitutedValue},
+                               writer);
         } else if (paramName.equalsIgnoreCase("cmd")) {
             boolean foundProgram = false;
             try {
@@ -57,6 +59,7 @@
                 IOTools.flow(stdErrReader, writer, buf);
                 IOTools.flow(stdOutReader, writer, buf);
                 proc.waitFor();
+                lastModified = System.currentTimeMillis();
             } catch (InterruptedException e) {
                 ssiMediator.log("Couldn't exec file: " + substitutedValue, e);
                 writer.write(configErrMsg);
@@ -68,5 +71,6 @@
                 ssiMediator.log("Couldn't exec file: " + substitutedValue, e);
             }
         }
+        return lastModified;
     }
 }
--- ./SSIFlastmod.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIFlastmod.java	Fri Jan 14 11:43:00 2005
@@ -28,8 +28,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         String configErrMsg = ssiMediator.getConfigErrMsg();
         StringBuffer buf = new StringBuffer();
         for (int i = 0; i < paramNames.length; i++) {
@@ -41,7 +42,7 @@
                 if (paramName.equalsIgnoreCase("file")
                         || paramName.equalsIgnoreCase("virtual")) {
                     boolean virtual = paramName.equalsIgnoreCase("virtual");
-                    long lastModified = ssiMediator.getFileLastModified(
+                    lastModified = ssiMediator.getFileLastModified(
                             substitutedValue, virtual);
                     Date date = new Date(lastModified);
                     String configTimeFmt = ssiMediator.getConfigTimeFmt();
@@ -58,6 +59,7 @@
                 writer.write(configErrMsg);
             }
         }
+        return lastModified;
     }
 
 
--- ./SSIFsize.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIFsize.java	Fri Jan 14 11:44:00 2005
@@ -30,8 +30,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         String configErrMsg = ssiMediator.getConfigErrMsg();
         for (int i = 0; i < paramNames.length; i++) {
             String paramName = paramNames[i];
@@ -42,6 +43,8 @@
                 if (paramName.equalsIgnoreCase("file")
                         || paramName.equalsIgnoreCase("virtual")) {
                     boolean virtual = paramName.equalsIgnoreCase("virtual");
+                    lastModified = ssiMediator.getFileLastModified(
+                        substitutedValue, virtual);
                     long size = ssiMediator.getFileSize(substitutedValue,
                             virtual);
                     String configSizeFmt = ssiMediator.getConfigSizeFmt();
@@ -56,6 +59,7 @@
                 writer.write(configErrMsg);
             }
         }
+        return lastModified;
     }
 
 
--- ./SSIInclude.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIInclude.java	Fri Jan 14 11:53:00 2005
@@ -25,8 +25,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         String configErrMsg = ssiMediator.getConfigErrMsg();
         for (int i = 0; i < paramNames.length; i++) {
             String paramName = paramNames[i];
@@ -37,6 +38,8 @@
                 if (paramName.equalsIgnoreCase("file")
                         || paramName.equalsIgnoreCase("virtual")) {
                     boolean virtual = paramName.equalsIgnoreCase("virtual");
+                    lastModified = ssiMediator.getFileLastModified(
+                        substitutedValue, virtual);
                     String text = ssiMediator.getFileText(substitutedValue,
                             virtual);
                     writer.write(text);
@@ -51,5 +54,6 @@
                 writer.write(configErrMsg);
             }
         }
+        return lastModified;
     }
 }
--- ./SSIMediator.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIMediator.java	Fri Jan 14 11:26:00 2005
@@ -41,7 +41,7 @@
     protected String configSizeFmt = DEFAULT_CONFIG_SIZE_FMT;
     protected String className = getClass().getName();
     protected SSIExternalResolver ssiExternalResolver;
-    protected Date lastModifiedDate;
+    protected long lastModifiedDate;
     protected int debug;
     protected Strftime strftime;
     protected SSIConditionalState conditionalState = new SSIConditionalState();
@@ -64,7 +64,7 @@
 
 
     public SSIMediator(SSIExternalResolver ssiExternalResolver,
-            Date lastModifiedDate, int debug) {
+            long lastModifiedDate, int debug) {
         this.ssiExternalResolver = ssiExternalResolver;
         this.lastModifiedDate = lastModifiedDate;
         this.debug = debug;
@@ -315,7 +315,7 @@
             setVariableValue("DATE_LOCAL", null);
             ssiExternalResolver.setVariableValue(className + ".DATE_LOCAL",
                     retVal);
-            retVal = formatDate(lastModifiedDate, null);
+            retVal = formatDate(new Date(lastModifiedDate), null);
             setVariableValue("LAST_MODIFIED", null);
             ssiExternalResolver.setVariableValue(className + ".LAST_MODIFIED",
                     retVal);
--- ./SSIPrintenv.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIPrintenv.java	Fri Jan 14 11:46:00 2005
@@ -24,8 +24,9 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer) {
+        long lastModified = 0;
         //any arguments should produce an error
         if (paramNames.length > 0) {
             String errorMessage = ssiMediator.getConfigErrMsg();
@@ -46,7 +47,9 @@
                 writer.write('=');
                 writer.write(variableValue);
                 writer.write('\n');
+                lastModified = System.currentTimeMillis();
             }
         }
+        return lastModified;
     }
 }
--- ./SSIProcessor.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIProcessor.java	Fri Jan 14 11:37:00 2005
@@ -15,9 +15,9 @@
 import java.io.PrintWriter;
 import java.io.Reader;
 import java.io.StringWriter;
-import java.util.Date;
 import java.util.HashMap;
 import java.util.StringTokenizer;
+
 import org.apache.catalina.util.IOTools;
 /**
  * The entry point to SSI processing. This class does the actual parsing,
@@ -36,7 +36,16 @@
     protected SSIExternalResolver ssiExternalResolver;
     protected HashMap commands = new HashMap();
     protected int debug;
-
+    
+    /**
+     * The servlet context attribute under which we store a flag used
+     * to mark this request as having been processed by the SSIServlet.
+     * We do this because of the pathInfo mangling happening when using
+     * the CGIServlet in conjunction with the SSI servlet. (value stored
+     * as an object of type String)
+     */
+     public static final String SSI_FLAG_ATTR =
+         "org.apache.catalina.ssi.SSIServlet";
 
     public SSIProcessor(SSIExternalResolver ssiExternalResolver, int debug) {
         this.ssiExternalResolver = ssiExternalResolver;
@@ -76,11 +85,12 @@
      *            the reader to read the file containing SSIs from
      * @param writer
      *            the writer to write the file with the SSIs processed.
+     * @return the most current modified date resulting from any SSI commands
      * @throws IOException
      *             when things go horribly awry. Should be unlikely since the
      *             SSICommand usually catches 'normal' IOExceptions.
      */
-    public void process(Reader reader, Date lastModifiedDate,
+    public long process(Reader reader, long lastModifiedDate,
             PrintWriter writer) throws IOException {
         SSIMediator ssiMediator = new SSIMediator(ssiExternalResolver,
                 lastModifiedDate, debug);
@@ -142,8 +152,11 @@
                             // command is not conditional
                             if
(!ssiMediator.getConditionalState().processConditionalCommandsOnly
                                     || ssiCommand instanceof SSIConditional) {
-                                ssiCommand.process(ssiMediator, strCmd,
-                                        paramNames, paramValues, writer);
+                                long lmd = ssiCommand.process(ssiMediator, strCmd,
+                                               paramNames, paramValues, writer);
+                                if (lmd > lastModifiedDate) {
+                                    lastModifiedDate = lmd;
+                                }                                    
                             }
                         }
                         if (errorMessage != null) {
@@ -160,6 +173,7 @@
             //If we are here, then we have already stopped processing, so all
             // is good
         }
+        return lastModifiedDate;
     }
 
 
--- ./SSIServlet.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIServlet.java	Fri Jan 14 11:34:00 2005
@@ -19,13 +19,12 @@
 import java.io.StringWriter;
 import java.net.URL;
 import java.net.URLConnection;
-import java.util.Date;
+
 import javax.servlet.ServletContext;
 import javax.servlet.ServletException;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
-import org.apache.catalina.Globals;
 /**
  * Servlet to process SSI requests within a webpage. Mapped to a path from
  * within web.xml.
@@ -166,7 +165,7 @@
             res.setDateHeader("Expires", (new java.util.Date()).getTime()
                     + expires.longValue() * 1000);
         }
-        req.setAttribute(Globals.SSI_FLAG_ATTR, "true");
+        req.setAttribute(SSIProcessor.SSI_FLAG_ATTR, "true");
         processSSI(req, res, resource);
     }
 
@@ -174,7 +173,7 @@
     protected void processSSI(HttpServletRequest req, HttpServletResponse res,
             URL resource) throws IOException {
         SSIExternalResolver ssiExternalResolver = new SSIServletExternalResolver(
-                this, req, res, isVirtualWebappRelative, debug);
+                getServletContext(), req, res, isVirtualWebappRelative, debug);
         SSIProcessor ssiProcessor = new SSIProcessor(ssiExternalResolver,
                 debug);
         PrintWriter printWriter = null;
@@ -189,8 +188,10 @@
         InputStream resourceInputStream = resourceInfo.getInputStream();
         BufferedReader bufferedReader = new BufferedReader(
                 new InputStreamReader(resourceInputStream));
-        Date lastModifiedDate = new Date(resourceInfo.getLastModified());
-        ssiProcessor.process(bufferedReader, lastModifiedDate, printWriter);
+        long lastModified = ssiProcessor.process(bufferedReader,
resourceInfo.getLastModified(), printWriter);
+        if (lastModified > 0) {
+            res.setDateHeader("Last-Modified", lastModified);
+        }
         if (buffered) {
             printWriter.flush();
             String text = stringWriter.toString();
--- ./SSIServletExternalResolver.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSIServletExternalResolver.java	Fri Jan 14 09:31:00 2005
@@ -12,16 +12,17 @@
 
 
 import java.io.IOException;
+import java.io.UnsupportedEncodingException;
 import java.net.URL;
 import java.net.URLConnection;
 import java.net.URLDecoder;
 import java.util.Collection;
 import java.util.Date;
 import java.util.Enumeration;
+
 import javax.servlet.RequestDispatcher;
 import javax.servlet.ServletContext;
 import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 /**
@@ -37,17 +38,17 @@
             "QUERY_STRING", "QUERY_STRING_UNESCAPED", "REMOTE_ADDR",
             "REMOTE_HOST", "REMOTE_USER", "REQUEST_METHOD", "SCRIPT_NAME",
             "SERVER_NAME", "SERVER_PORT", "SERVER_PROTOCOL", "SERVER_SOFTWARE"};
-    protected HttpServlet servlet;
+    protected ServletContext context;
     protected HttpServletRequest req;
     protected HttpServletResponse res;
     protected boolean isVirtualWebappRelative;
     protected int debug;
 
 
-    public SSIServletExternalResolver(HttpServlet servlet,
+    public SSIServletExternalResolver(ServletContext context,
             HttpServletRequest req, HttpServletResponse res,
             boolean isVirtualWebappRelative, int debug) {
-        this.servlet = servlet;
+        this.context = context;
         this.req = req;
         this.res = res;
         this.isVirtualWebappRelative = isVirtualWebappRelative;
@@ -60,9 +61,9 @@
         //is the same as Servlet.log( message ), since API
         //doesn't seem to say so.
         if (throwable != null) {
-            servlet.log(message, throwable);
+            context.log(message, throwable);
         } else {
-            servlet.log(message);
+            context.log(message);
         }
     }
 
@@ -160,7 +161,14 @@
         } else if (name.equalsIgnoreCase("QUERY_STRING_UNESCAPED")) {
             String queryString = req.getQueryString();
             if (queryString != null) {
-                retVal = URLDecoder.decode(queryString);
+                try
+                {
+                    retVal = URLDecoder.decode(queryString, "UTF-8");
+                }
+                catch (UnsupportedEncodingException e)
+                {
+                    retVal = queryString;
+                }
             }
         } else if (name.equalsIgnoreCase("REMOTE_ADDR")) {
             retVal = req.getRemoteAddr();
@@ -179,8 +187,7 @@
         } else if (name.equalsIgnoreCase("SERVER_PROTOCOL")) {
             retVal = req.getProtocol();
         } else if (name.equalsIgnoreCase("SERVER_SOFTWARE")) {
-            ServletContext servletContext = servlet.getServletContext();
-            retVal = servletContext.getServerInfo();
+            retVal = context.getServerInfo();
         }
         return retVal;
     }
@@ -250,26 +257,23 @@
                     + nonVirtualPath);
         }
         String path = getAbsolutePath(nonVirtualPath);
-        ServletContext servletContext = servlet.getServletContext();
         ServletContextAndPath csAndP = new ServletContextAndPath(
-                servletContext, path);
+                context, path);
         return csAndP;
     }
 
 
     protected ServletContextAndPath getServletContextAndPathFromVirtualPath(
             String virtualPath) throws IOException {
-        ServletContext servletContext = servlet.getServletContext();
-        String path = null;
         if (!virtualPath.startsWith("/") && !virtualPath.startsWith("\\")) {
-            path = getAbsolutePath(virtualPath);
+            return new ServletContextAndPath(context,
getAbsolutePath(virtualPath));
         } else {
             String normalized = SSIServletRequestUtil.normalize(virtualPath);
             if (isVirtualWebappRelative) {
-                path = normalized;
+                return new ServletContextAndPath(context, normalized);
             } else {
-                servletContext = servletContext.getContext(normalized);
-                if (servletContext == null) {
+                ServletContext normContext = context.getContext(normalized);
+                if (normContext == null) {
                     throw new IOException("Couldn't get context for path: "
                             + normalized);
                 }
@@ -277,19 +281,19 @@
                 // to remove,
                 // ie:
                 // '/file1.shtml' vs '/appName1/file1.shtml'
-                if (!isRootContext(servletContext)) {
-                    path = getPathWithoutContext(normalized);
-                    if (path == null) {
+                if (!isRootContext(normContext)) {
+                    String noContext = getPathWithoutContext(normalized);
+                    if (noContext == null) {
                         throw new IOException(
                                 "Couldn't remove context from path: "
                                         + normalized);
                     }
+                    return new ServletContextAndPath(normContext, noContext);
                 } else {
-                    path = normalized;
+                    return new ServletContextAndPath(normContext, normalized);
                 }
             }
         }
-        return new ServletContextAndPath(servletContext, path);
     }
 
 
--- ./SSISet.java	Fri Oct 29 16:13:00 2004
+++ ../ssi_new/SSISet.java	Fri Jan 14 11:52:00 2005
@@ -23,9 +23,10 @@
     /**
      * @see SSICommand
      */
-    public void process(SSIMediator ssiMediator, String commandName,
+    public long process(SSIMediator ssiMediator, String commandName,
             String[] paramNames, String[] paramValues, PrintWriter writer)
             throws SSIStopProcessingException {
+        long lastModified = 0;
         String errorMessage = ssiMediator.getConfigErrMsg();
         String variableName = null;
         for (int i = 0; i < paramNames.length; i++) {
@@ -39,6 +40,7 @@
                             .substituteVariables(paramValue);
                     ssiMediator.setVariableValue(variableName,
                             substitutedValue);
+                    lastModified = System.currentTimeMillis();
                 } else {
                     ssiMediator.log("#set--no variable specified");
                     writer.write(errorMessage);
@@ -50,5 +52,6 @@
                 throw new SSIStopProcessingException();
             }
         }
+        return lastModified;
     }
 }
--- SSIFilter.java	Fri Jan 14 13:25:44 2005
+++ ../ssi_new/SSIFilter.java	Fri Jan 14 13:30:06 2005
@@ -0,0 +1,189 @@
+/*
+ * Copyright 1999,2004 The Apache Software Foundation. Licensed under the
+ * Apache License, Version 2.0 (the "License"); you may not use this file
+ * except in compliance with the License. You may obtain a copy of the License
+ * at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable
+ * law or agreed to in writing, software distributed under the License is
+ * distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the specific language
+ * governing permissions and limitations under the License.
+ */
+package org.apache.catalina.ssi;
+
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.io.OutputStream;
+import java.io.OutputStreamWriter;
+import java.io.PrintWriter;
+import java.io.Reader;
+import java.io.Writer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import javax.servlet.Filter;
+import javax.servlet.FilterChain;
+import javax.servlet.FilterConfig;
+import javax.servlet.ServletException;
+import javax.servlet.ServletRequest;
+import javax.servlet.ServletResponse;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+/**
+ * Filter to process SSI requests within a webpage. Mapped to a content types from
+ * within web.xml.
+ *
+ * Based on code from <code>org.apache.catalina.ssi.SSIServlet</code>.
+ * 
+ * @author David Becker
+ * @version $Revision: 1.0 $, $Date: 2005/01/14 13:28:00 $
+ * @see org.apache.catalina.ssi.SSIServlet
+ */
+public class SSIFilter implements Filter {
+    /** Configuration for this filter. */
+    protected FilterConfig config = null;
+    /** Debug level for this filter. */
+    protected int debug = 0;
+    /** Expiration time in seconds for the doc. */
+    protected Long expires = null;
+    /** virtual path can be webapp-relative */
+    protected boolean isVirtualWebappRelative = false;
+    /** content types allowed for SSI processing */
+    protected List contentTypes = new ArrayList();
+    /** should all content types be allowed */
+    protected boolean allowAllContentTypes = false;
+
+
+    //----------------- Public methods.
+    /**
+     * Initialize this filter.
+     * 
+     * @exception ServletException
+     *                if an error occurs
+     */
+    public void init(FilterConfig config) throws ServletException {
+        this.config = config;
+        String value = null;
+        try {
+            value = config.getInitParameter("debug");
+            debug = Integer.parseInt(value);
+        } catch (Throwable t) {
+            ;
+        }
+        try {
+            value = config.getInitParameter(
+                    "isVirtualWebappRelative");
+            isVirtualWebappRelative = Integer.parseInt(value) > 0?true:false;
+        } catch (Throwable t) {
+            ;
+        }
+        try {
+            value = config.getInitParameter("expires");
+            expires = Long.valueOf(value);
+        } catch (NumberFormatException e) {
+            expires = null;
+            config.getServletContext().log("Invalid format for expires
initParam; expected integer (seconds)");
+        } catch (Throwable t) {
+            ;
+        }
+        try {
+            String types = config.getInitParameter("contentTypes");
+            allowAllContentTypes = (types.equals("*") || types.equals("*/*"));
+            if ((types != null) && !types.equals(""))
+            {
+                contentTypes = Arrays.asList(types.split(","));
+            }
+            else
+            {
+                contentTypes.add("text/html");
+                config.getServletContext().log("No contentTypes initParam
provided; defaulting to text/html");
+            }
+        } catch (Throwable t) {
+            ;
+        }
+        if (debug > 0)
+            config.getServletContext().log("SSIFilter.init() SSI invoker
started with 'debug'=" + debug);
+    }
+
+
+    public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
+        // cast once
+        HttpServletRequest req = (HttpServletRequest)request;
+        HttpServletResponse res = (HttpServletResponse)response;
+        
+        // indicate that we're in SSI processing
+        req.setAttribute(SSIProcessor.SSI_FLAG_ATTR, "true");           
+
+        // setup to capture output
+        ByteArrayServletOutputStream basos = new ByteArrayServletOutputStream();
+        ResponseIncludeWrapper responseIncludeWrapper = new
ResponseIncludeWrapper(res, basos);
+
+        // process remainder of filter chain
+        chain.doFilter(req, responseIncludeWrapper);
+
+        // we can't assume the chain flushed its output
+        responseIncludeWrapper.flushOutputStreamOrWriter();
+        byte[] bytes = basos.toByteArray();
+
+        // get content type
+        String contentType = responseIncludeWrapper.getContentType();
+        if ((contentType == null) || contentType.equals("")) {
+            contentType =
config.getServletContext().getMimeType(req.getRequestURI());
+            if ((contentType == null) || contentType.equals("")) {
+                contentType = "text/html";
+            }
+        }
+        if (contentType.indexOf(";") > -1)
+        {
+            contentType = contentType.substring(0, contentType.indexOf(";"));
+        }
+
+        // is this an allowed type for SSI processing?
+        if (allowAllContentTypes || contentTypes.contains(contentType)) {
+
+            // set up SSI processing 
+            SSIExternalResolver ssiExternalResolver = new
SSIServletExternalResolver(
+                    config.getServletContext(), req, res,
isVirtualWebappRelative, debug);
+            SSIProcessor ssiProcessor = new SSIProcessor(ssiExternalResolver,
+                    debug);
+            
+            // prepare readers/writers
+            Reader reader = new InputStreamReader(new ByteArrayInputStream(bytes));
+            ByteArrayOutputStream ssiout = new ByteArrayOutputStream();
+            PrintWriter writer = new PrintWriter(new OutputStreamWriter(ssiout));
+            
+            // do SSI processing  
+            long lastModified = ssiProcessor.process(reader,
responseIncludeWrapper.getLastModified(), writer);
+            
+            // set output bytes
+            writer.flush();
+            bytes = ssiout.toByteArray();
+            
+            // override headers
+            if (expires != null) {
+                res.setDateHeader("Expires", (new java.util.Date()).getTime()
+                        + expires.longValue() * 1000);
+            }
+            if (lastModified > 0) {
+                res.setDateHeader("Last-Modified", lastModified);
+            }
+        }
+
+        // write output
+        try {
+            OutputStream out = res.getOutputStream();
+            out.write(bytes);
+        } catch (Throwable t) {
+            Writer out = res.getWriter();
+            out.write(new String(bytes));
+        }
+    }
+
+
+    public void destroy()
+    {
+    }
+}
Comment 1 David Becker 2005-01-14 22:36:39 UTC
Created attachment 14002 [details]
diff -u results for all modified files
Comment 2 David Becker 2005-01-14 22:38:57 UTC
(In reply to comment #0)
> Patches (diff -u) Follow:

Sorry, I didn't realize I'd be given the chance to add an attachment later --
I've never submitted before...
Comment 3 David Becker 2005-01-15 01:41:51 UTC
Re #3 above, there is also a dependency on org.apache.catalina.util classes
DateTool, StringManager and RequestUtil.
Comment 4 David Becker 2005-01-20 19:05:16 UTC
Created attachment 14053 [details]
Fixed a null pointer bug if you failed to specify the contentTypes initParam

The contentTypes initParam in SSIFilter was supposed to default to text/html if
you failed to specify it, but it was generating a null pointer exception due to
a misplaced statement.
Comment 5 Yoav Shapira 2005-03-23 16:32:17 UTC
Mark, as the custodian of the SSI stuff, please let us know if you're
interesting in committing this at some point.  Thanks ;)
Comment 6 Mark Thomas 2005-03-26 22:25:36 UTC
I'll look at this now, so it should be in place for 5.5.10. I won't back port
this to 4.1.x of 5.0.x

David, you might want to consider providing some documentation patches otherwise
people won't be aware of these enhancements.
Comment 7 Mark Thomas 2005-03-27 14:10:39 UTC
I have reviewed the patch and have found one area I would like to see a change.
The character encoding used to decode the query string has changed from platform
default to UTF-8. Such a change is likely to break things for existing users.

FYI the Tomcat Coyote connector has a number of configuration options for URI
decoding including:
- use platform default
- use body encoding
- use specified encoding

This could be fixed either by adding a configuration option or by reading the
settings from the connector. The downside to using the connector settings is it
obviously isn't portable.

I'll leave this a week or so and if an alternative approach to the query string
decoding isn't submitted I'll commit this patch less that one change.
Comment 8 David Becker 2005-03-28 19:15:26 UTC
(In reply to comment #7)
Thanks for the feedback and for being so kind to a new submitter.  :-)  I'll
work on the areas you suggest.  Look for updated patches, etc., by next Monday.

Thanks!
Comment 9 David Becker 2005-04-04 04:27:57 UTC
Sorry guys, but I need more time.  I had a busy week at work and a computer failure this weekend.  I do 
plan on submitting a new set of patches with the issues you raised addressed, doc patches and some 
other minor enhancements.  Can you give me another week?  Thanks!
Comment 10 Mark Thomas 2005-04-04 19:15:05 UTC
Not a problem. Take as long as you need. 
Comment 11 David Becker 2005-04-11 04:08:09 UTC
Created attachment 14675 [details]
Final set of diffs

Here it is at long last.  A new set of diffs based on 5.5.9 with the fixes you
proposed.  I've also included web.xml, build.xml and documentation patches. 
Please review the section on the query string decoding you had concerns about
and see if you think I handled it properly.

I've also taken the liberty of making the SSIFilter the default behavior and
deprecating the SSIServlet.  I hope this is OK with everyone.
Comment 12 David Becker 2005-04-11 04:09:24 UTC
Created attachment 14676 [details]
Full set of modified files

Here are the same set of files, but the full files in case you don't feel like
manually applying the patches.
Comment 13 David Becker 2005-04-11 04:12:20 UTC
BTW, I also made a few other minor fixes based on my experiences working with
these changes in a production environment.  The date parsing of the
last-modified header is a bit more robust and the contentType init parameter was
changed to a regex pattern for more flexibility.
Comment 14 Mark Thomas 2005-04-12 22:04:46 UTC
David,

I can't read you patches in bzip2 format (winzip doesn't recognise it).

Can you re-add the patches in an alternative format (tar.gz would be ok)?
Comment 15 David Becker 2005-04-12 22:11:13 UTC
Created attachment 14693 [details]
Final set of diffs (in gzip)
Comment 16 David Becker 2005-04-12 22:11:54 UTC
Created attachment 14694 [details]
Full set of modified files (in gzip)
Comment 17 Mark Thomas 2005-04-14 19:56:53 UTC
I have almost completed reviewing and applying your changes. So far, I have only
made some minor changes (typos, line lengths etc) and have also re-worked the
query string encoding algorithm a bit.

There is one part of your patch I haven't yet worked out. In
ResponseIncludeWrapper you have intercepted the last-modified and content-type
headers. I understand why last-modified but not content-type. Can you explain
please?
Comment 18 David Becker 2005-04-14 20:08:04 UTC
The SSIFilter uses that same wrapper to capture the normal output.  The filter
applies SSI processing by content type not file extension.
Comment 19 Mark Thomas 2005-04-14 22:23:39 UTC
But why override getContentType()? I don't see what your code gives you that the
standard wrapper method doesn't.
Comment 20 David Becker 2005-04-14 22:31:34 UTC
(In reply to comment #19)
> But why override getContentType()? I don't see what your code gives you that the
> standard wrapper method doesn't.

Ahh.  Not all resources (servlets, etc.) explicitly set a mime type, and this
method sometimes returns null, leaving it up to the container to set the content
type on the way out.  If this is the case, my method will attempt to look it up
from the container before hand, because I need it *now*...  Not 100% sure on
Tomcat, but I know iPlanet does that.
Comment 21 Mark Thomas 2005-04-21 23:33:04 UTC
Almost there, but one slight glitch. After the SSI processing, the
content-length header and the actual content length are different. This causes
problems for both FireFox and IE. The easiest way to see it is to do an SSI
include of a file containing a single character.

When I do commit your patch, I intend to include the following changes unless
you know of a reason not to:
SSIServlet
  - Not deprecated. I'd like to give people the option.
SSIFilter
  - Format changes for 80 character width.
ResponseIncludeWrapper
  - Default for lastModified changed to -1 from 0.
Globals
  - Keep the old flag
  - I need to check why the CGI servlet isn't using it any more
build.xml
  - Keep jar file name the same for consistency for current users
web.xml
  - Keep servlet mappings
SSIHowTo
  - Keep servlet configuration
SSIServletExternalResolver
  - Align behaviour with standard Tomcat. Fix posisble NPEs. Code is now:

        } else if (name.equalsIgnoreCase("QUERY_STRING_UNESCAPED")) {
            String queryString = req.getQueryString();
            if (queryString != null) {
                // Use default as a last resort
            	String queryStringEncoding =
                    org.apache.coyote.Constants.DEFAULT_CHARACTER_ENCODING;
                
                String uriEncoding = null;
                boolean useBodyEncodingForURI = false;
                
                // Get encoding settings from request / connector if possible
                String requestEncoding = req.getCharacterEncoding();
                if (req instanceof Request) {
                    uriEncoding = ((Request)req).getConnector().getURIEncoding();
                    useBodyEncodingForURI =
                        ((Request)req).getConnector().getUseBodyEncodingForURI();
                }
                
                // If valid, apply settings from request / connector
                if (uriEncoding != null) {
                	queryStringEncoding = uriEncoding;
                } else if(useBodyEncodingForURI) {
                    if (requestEncoding != null) {
                    	queryStringEncoding = requestEncoding;
                    }
                }
                
                try {
               	    retVal = URLDecoder.decode(queryString,
queryStringEncoding);                       
				} catch (UnsupportedEncodingException e) {
					retVal = queryString;
				}
            }

Comment 22 David Becker 2005-04-21 23:50:43 UTC
Thank you for your follow up on this.  You're right about the content-length
header, I didn't notice that in our environment because our server sets the
content-length on the way out to address some other issues.  This could be
simply resolved by explicitly resetting the content-length header to the site of
the byte wrapper array.  Feel free to make this (and the other changes you
proposed) unless you'd prefer me to do it.  Thanks a ton for your help in
working with a newbie contributor...  :-)
Comment 23 Mark Thomas 2005-04-23 12:24:51 UTC
Many thanks for the patch.
Comment 24 David Becker 2005-05-05 06:05:36 UTC
Looking in the web cvs view at
http://cvs.apache.org/viewcvs.cgi/jakarta-tomcat-catalina/catalina/src/share/org/apache/catalina/ssi/
I don't see the SSIFilter.  Am I missing something?  Or does that lag a little
bit behind the actual CVS checkins?
Comment 25 Mark Thomas 2005-05-05 19:31:15 UTC
My fault. I committed the other changes but forgot to add the new file. It is
done now.
Comment 26 David Becker 2005-08-16 19:38:02 UTC
Created attachment 16065 [details]
catch exceptions for non-file based urls

While testing with the official code merges, I noticed that some exception
catching was removed from two file based methods.  If these exceptions are not
caught, non-file based includes won't work.  In my case, we do a <!--#include
virtual="/some/servlet"--> but because that's not a file, it can't figure out
the last modified date because the servlet doesn't set one, and so it throws an
exception.  This exception needs to be caught, or the include won't work.
Comment 27 Mark Thomas 2005-08-17 23:15:41 UTC
Fixed in CVS for 5.5.x and will be included in next release.