Bug 47609

Summary: Failsafe EOL conversion
Product: Tomcat 5 Reporter: Sebb <sebb>
Component: Native:PackagingAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED FIXED    
Severity: normal    
Priority: P2    
Version: 5.5.27   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Attachments: List of extensions used in 5.5.28
List of files with no extension
fixcrlf use include rather than exclude
Improved patch - detab

Description Sebb 2009-07-30 11:52:49 UTC
Created attachment 24066 [details]
List of extensions used in 5.5.28

The builds/build.xml file has the following code:

  <target name="package-src-tgz">
    <fixcrlf srcdir="${tomcat.dist}/src"
        excludes="**/*.bin,**/*.bmp,**/*.dia,**/*.exe,**/*.gif,**/*.ico,**/*.jar,**/*.jpg,**/*.pdf,**/*.rtf,**/*.war"
        eol="lf"
        encoding="ISO-8859-1" fixlast="false" />

The intention is to fix all the source files so that they are readable on Unix (LF) systems.

However, the command is inherently unsafe - if a new binary file is added with an extension that is not listed, then the file will be damaged. In general it is impossible to reverse LF-conversion on a binary file.

It would be much safer to use includes instead. If a new source file type is added, then the worst that can happen is that the file is awkard to read on Unix. Unlike the case with binary files, the file can still be converted successfully.

The current list of extensions that are present in the 5.5.28 source archive are included in the file "extensions.txt".

At least two binary extensions (.db, .keystore) are currently missing from the excludes list.

There are quite a few files with no extension. These appear to be all text files, and are listed in the file "no-extension.txt" (duplicate names have been removed).
Comment 1 Sebb 2009-07-30 11:53:24 UTC
Created attachment 24067 [details]
List of files with no extension
Comment 2 Filip Hanik 2009-07-31 16:04:23 UTC
So the only safe way to do it, would be to specify an includes list instead of using an excludes.
Patches welcome
Comment 3 Sebb 2009-08-02 07:26:27 UTC
OK, I will create a patch shortly
Comment 4 Sebb 2009-08-02 08:24:37 UTC
Created attachment 24080 [details]
fixcrlf use include rather than exclude
Comment 5 Sebb 2009-08-02 08:29:53 UTC
Created attachment 24081 [details]
Improved patch - detab
Comment 6 Mark Thomas 2009-11-22 11:31:59 UTC
Modified patch applied to trunk and proposed for 6.0.x and 5.5.x
Comment 7 Konstantin Kolinko 2010-03-04 20:30:20 UTC
Fixed in 6.0.22, 5.5.29.