Bug 36905

Summary: Tomcat WebDAV characters bug
Product: Tomcat 5 Reporter: Roland Rabben <roland.rabben>
Component: Servlets:WebDAVAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED FIXED    
Severity: major CC: christoph
Priority: P2    
Version: 5.5.12   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Attachments: Screen dump of directorylisting with filename containing semi-colon.
Screen-dump of error-msg when I click the filename with semi-colon.

Description Roland Rabben 2005-10-04 09:19:13 UTC
When writing files to the server using Tomcats WebDAV I am not able to write
files that contain theese characters in the filename (or create folders):

; (semicolon)
+ (plus)
# (pound)

I know these are reserved characters, so I have tried to URL encode them, but
Tomcat ignores the URL encoding and writes the URL encoded filenames to disk.
Eg. "myfile;01.txt" is saved like "myfile%B301.txt" on the server. Tomcat is
configured to do URL Encoding. If I don't do URL Encoding it simply cuts the end
of the filename from semicolon like "myfile".

I also tested this in Tomcat 5.5.12 with same problems. I have tested several
clients. IE 6.0, DavExplorer and own code.

This is my Connector configuration in server.xml:
 <Connector port="80" 
 maxHttpHeaderSize="8192" useBodyEncodingForURI="true" URIEncoding="UTF-8"
 maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
 enableLookups="false" redirectPort="8443" acceptCount="100"
 connectionTimeout="20000" disableUploadTimeout="true" />

This is what Tomcat logs in stdout.log:
09:14:36,040  INFO MediaServiceImpl: destination = Scala/content/
09:14:36,056  INFO MediaServiceImpl: MediaItem saved! Identifier: 7
09:14:36,072  INFO MediaServiceImpl: MediaItemFile saved! Identifier: 1
09:14:36,290  INFO [/rome-planning]: webdav: DefaultServlet.init:  input buffer
size=2048, output buffer size=2048
09:14:36,290  INFO [/rome-planning]: webdav: [PROPFIND] /
[Fatal Error] :-1:-1: Premature end of file.
09:14:36,384  INFO [/rome-planning]: webdav: [PUT] /data/Scala/content/myfile

stderr.log reports nothing.
Comment 1 Mark Thomas 2005-10-04 18:36:51 UTC
Taking me off the CC list since I get see all these via tomcat-dev anyway.

Please do not add other people's e-mail addresses to a bugzilla entry unless
they ask you to.
Comment 2 Mark Thomas 2005-10-04 23:50:05 UTC
Testing files with 5.5.x from CVS, IE6, XP(SP2+all patches), URIEncoding="UTF-8"
and port="8080":

'+', ' ' and '#' are OK for files, ';' gets truncated

Looking at the headers using ieHTTPHeaders (I was very impressed that this works
with web folders as well as normal web pages) the fault lies with IE. You can
see from the headers that '#' is encoded but ';' is not. If you try forcing IE
to send the correct headers by using a file name with '%B3' it correctly encodes
the '%' and you see '%25B3' in the headers and '%B3' in your webdav directory.

If you have a *simple* test client that shows that any of these characters are
not correctly handled by Tomcat's WebDAV servlet please attach it so I can
investigate further but all indications so far are that for files this is a
client issue not a Tomcat problem. Best of luck getting MS to fix it...

I also suggest you look at bug36303 which you may well run into if you haven't
already.

I'll look at collections next...
Comment 3 Mark Thomas 2005-10-05 00:18:58 UTC
With IE as the client the same behaviour is seen for collections as for files
and the cause is the same - IE does not encode ';' in file names.

Just to see what would happen, I tried the folder creation tests with DAV
Explorer 0.91. Using TCP Mon from the Apache Axis project (a very handy utility)
DAV Explorer is also getting the headers wrong but not in the same way as IE.

DAV tuncates files with '#' in the name internally before they ar even sent to
Tomcat. For file names that contain ';' DAV explorer does not truncate but
neither does it encode.

Since you also reported your own client was having problems, I am going to set
this a NEEDINFO. The information I need is a sample client program that
demonstrates a problem with Tomcat.

As I have seen no sign of Tomcat misbehaviour here, if there is no further
update after a few weeks I will close this report as invalid.
Comment 4 Roland Rabben 2005-10-05 09:48:22 UTC
The error was produced with IE, DavExplorer and our own client code. Our own 
client code uses apache webdav libs. The client was run from a Windows XP SP2 
and MAC OSX 10.4.

Our Tomcat server (5.5.12) was running on port 80.

We tested the same clients on Apache 2.0.54 with mod_dav without any problems. 
This leads me to think that the problem lies with Tomcat and not our client.

Test code:
---------------------------------


import java.io.File;
import java.io.IOException;

import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpURL;
import org.apache.commons.httpclient.URIException;
import org.apache.webdav.lib.WebdavResource;

public class ModDavTestClient {

	public static final String TEST_FILENAME = "myTestFile;testing.txt";
	public static final String TEST_FILEPATH = "C:/temp/";
	public static final String TEST_CONTEXT 
= "http://localhost:8080/planning/webdav/data/";
	public static final String TEST_USERNAME = "upload";
	public static final String TEST_PASSWORD = "test123";
	
	public static void main(String args[]) {
		HttpURL homeUrl = null;
		WebdavResource webDavRes = null;
		boolean response = false;
		HttpURL dest = null;

		// Preparing File to be upload with webdav resource
		File inputFile = new File(TEST_FILEPATH + TEST_FILENAME);

		if (!inputFile.exists()) {
			System.err.println("Test file not exists: " + 
TEST_FILEPATH
					+ TEST_FILENAME);
			System.exit(0);
		}

		// Creating HttpURL for WebdavResource
		try {
			homeUrl = new HttpURL(TEST_CONTEXT);
			homeUrl.setUserinfo(TEST_USERNAME, TEST_PASSWORD);
		} catch (URIException e) {
			e.printStackTrace();
		}

		if (homeUrl == null) {
			System.err.println("Error creating HomeURL");
			System.exit(0);
		}

		// Creating WebdavResource
		try {
			webDavRes = new WebdavResource(homeUrl);
		} catch (HttpException e) {
			e.printStackTrace();
			System.exit(0);
		} catch (IOException e) {
			e.printStackTrace();
			System.exit(0);
		}

		StringBuilder destUrl = new StringBuilder(TEST_CONTEXT);
		destUrl.append(TEST_FILENAME);

		try {
			dest = new HttpURL(destUrl.toString());
			dest.setUserinfo(TEST_USERNAME, TEST_PASSWORD);
		} catch (URIException e1) {
			e1.printStackTrace();
			try {
				webDavRes.close();
			} catch (IOException e) {
			}
			System.exit(0);
		}

		try {
			System.out.println("Prepare writing to destination: " 
+ dest.getPath());
		} catch (URIException e2) {
			e2.printStackTrace();
			try {
				webDavRes.close();
			} catch (IOException e) {
			}
			System.exit(0);
		}

		try {
			response = webDavRes.putMethod(dest.getPath(), 
inputFile);
		} catch (URIException e1) {
			e1.printStackTrace();
		} catch (HttpException e1) {
			e1.printStackTrace();
		} catch (IOException e1) {
			e1.printStackTrace();
		}

		System.out.println("respond: " + response);

		try {
			webDavRes.close();
		} catch (IOException e) {
			e.printStackTrace();
		}
		System.exit(0);
	}

}


Running output:
--------------------
Prepare writing to destination: /planning/webdav/data/myTestFile;testing.txt
respond: true

Checking uploaded file:
-----------------------
[tomcat path]/webapps/planning/webdav/data/myTestFile

It suppose to be:
[tomcat path]/webapps/planning/webdav/data/myTestFile;testing.txt


Testing with same code, but upload to an Apache ModDav: it works.
Comment 5 Mark Thomas 2005-10-05 22:25:20 UTC
Thanks for the client app. Having debugged my way through it looking at TCP Mon
as I go, the request sent to Tomcat is not correctly encoded. However, the much
simpler test finally dawned on me. Telnet with MKCOL /webdav/test%3Bcol doesn't
work either so I will investigate further. However, once this is fixed (assuming
a fix is possible) there will still be a lot of things broken on the client side.
Comment 6 Roland Rabben 2005-10-11 13:07:11 UTC
Created attachment 16655 [details]
Screen dump of directorylisting with filename containing semi-colon.
Comment 7 Roland Rabben 2005-10-11 13:07:56 UTC
Created attachment 16656 [details]
Screen-dump of error-msg when I click the filename with semi-colon.
Comment 8 Roland Rabben 2005-10-11 13:10:40 UTC
I have discovered a simular problem that seems to be related to this bug.

I place a file with semi-colon in the filename under the Tomcat http server. 
When I am linking to that file (test;01.png) i get "the requested resource is 
not available.

Take a look at the attached screendumps.

Screen1 shows a directory listing of the file in question.
Screen2 shows the error-msg when i click the filename.

This is not a problem in Apache 2.0.53
I tested and found this problem using IE 6.0 and Firefox 1.0.7 against Tomcat 
5.5.12
Comment 9 Mark Thomas 2006-07-28 01:38:41 UTC
The handling of encoded ; characters has now been fixed in the SVN and will be
included in 5.5.18 and later.

I have tested MKCOL /webdav/test%3Btest and the file test;test is created. All
of the other issues I saw whilst investigating this bug were client related.
These issues are likely to continue to cause problems in this area. (For
example, IE doesn't encode the ; when requesting test;test and throws a fit if
you try.) On this basis I am closing this bug.

If you believe this bug is still valid please test using telnet before
re-opening to make 100% sure you are seeing a Tomcat issue rather than a client one.
Comment 10 Mark Thomas 2006-08-30 03:18:03 UTC
*** Bug 33806 has been marked as a duplicate of this bug. ***