Bug 57344 - [PATCH] Provide sha1 checksum files for Tomcat downloads
[PATCH] Provide sha1 checksum files for Tomcat downloads
Status: RESOLVED FIXED
Product: Tomcat 6
Classification: Unclassified
Component: Native:Packaging
6.0.43
PC All
: P2 enhancement (vote)
: ----
Assigned To: Tomcat Developers Mailing List
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2014-12-12 00:38 UTC by Konstantin Kolinko
Modified: 2015-02-18 22:05 UTC (History)
0 users



Attachments
2014-12-12_tc9_57344_sha1.patch (9.90 KB, patch)
2014-12-12 00:40 UTC, Konstantin Kolinko
Details | Diff
2014-12-14_tc6_57344_sha1.patch (2.20 KB, patch)
2014-12-14 16:13 UTC, Konstantin Kolinko
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Konstantin Kolinko 2014-12-12 00:38:18 UTC
I prepared a patch that adds *.sha1 files to Tomcat release besides the *.md5 ones.

Are there any objections?

Minor issue fixed by the way:
due to missing <exclude name="*.asc"> the apache-tomcat-8.0.15-embed.tar.gz archive contains apache-tomcat-8.0.15-embed.zip.asc file that does not belong there.


Is it worth to add sha2 (sha-512)? I have some questions:

My issues with sha-512:
=========================
1) Is it worth adding sha-512?
Generally users shall use PGP signatures for verification, not the checksums.

2) Not many ASF projects are providing sha2 checksums.
One that I know is Apache Ant,

http://ant.apache.org/bindownload.cgi

Looking at Apache Ant, I have several notes:

a) *.sha512 files are not copied to mirrors, so ASF mirror system knows how to skip these files. (Good)

b) when I click on the link to a file, browser (Firefox 34) does not display the file, but prompts to download it. (Unexpected)

Looking at response headers in Firefox, the server sends Content-Type: application/zip for *.zip.sha512, application/x-gzip for *.tar.gz.sha512, application/x-bzip2 for *.tar.bz2.

So the server is not configured for these files. It sends text/plain for *.sha1 files.


3) I do not have a command-line tool to automatically verify sha-512.

There exist sha512sum from GNU,
http://www.gnu.org/software/coreutils/manual/coreutils.html#sha2-utilities

but GnuWin32 CoreUtils do not have this tool.


4) What file format shall we use?

Apache Ant downloads are using  hashsum + LF.

We are using  hashsum + " *" + filename, which is the format supported by md5sum and sha1sum GNU utilities.

`openssl dgst -sha512 filename` generates "SHA512(" + filename + ")= " +hashsum +LF, but I think that openssl does not read this format.

I think it would better to print just the hashsum value, but I wonder if that is supported by sha512sum tool.
Comment 1 Konstantin Kolinko 2014-12-12 00:40:10 UTC
Created attachment 32286 [details]
2014-12-12_tc9_57344_sha1.patch
Comment 2 Christopher Schultz 2014-12-12 04:03:12 UTC
(In reply to Konstantin Kolinko from comment #0)
> 3) I do not have a command-line tool to automatically verify sha-512.
> 
> There exist sha512sum from GNU,
> http://www.gnu.org/software/coreutils/manual/coreutils.html#sha2-utilities
> 
> but GnuWin32 CoreUtils do not have this tool.

The 'sha512sum' command exists on both Debian and Amazon Linux (CentOS/RHEL compatible), and so I would imagine it's available on most Linux distributions. Mac OS X has 'shasum -a 512' which "mimics the behavior of GNU shaXsum" (according to the man page).

As we use Apache Ant for building, why not just use <checksum>?
https://ant.apache.org/manual/Tasks/checksum.html

It can be configured to use any of the proposed formats...

> 4) What file format shall we use?
> 
> Apache Ant downloads are using  hashsum + LF.
> 
> We are using  hashsum + " *" + filename, which is the format supported by
> md5sum and sha1sum GNU utilities.
> 
> `openssl dgst -sha512 filename` generates "SHA512(" + filename + ")= "
> +hashsum +LF, but I think that openssl does not read this format.
> 
> I think it would better to print just the hashsum value, but I wonder if
> that is supported by sha512sum tool.

I don't believe shaXsum can do that. I can't get the Mac version to do it, either. If sed/awk/etc can be relied upon, we can always cobble-together whatever combination of strings we need to make the tools happy.

In what environments do we think that checksums will be verified?

Ant's <checksum> can verify a signature as well as generate one. Is Ant/JVM any more/less trustworthy than shaXsum/openssl?
Comment 3 Mark Thomas 2014-12-12 09:58:36 UTC
+0 to the patch. No objections but what is the benefit?

Re sha2:
1. As above. What is the benefit.

2. I'm less concerned about what other ASF projects are doing and more concerned about what the benefit of is doing it is.
2b) I'll see if I can get that fixed.

3. I use cyohash. It doesn't support the exact formats but it is good enough for validating.

4. The same format as we do for sha1 unless there is a good reason not to.
Comment 4 Konstantin Kolinko 2014-12-13 21:23:48 UTC
> No objections but what is the benefit?

My concern is that there have been actual malware that exploited weakness in MD5 (Flame, as mentioned in Wikipedia article on MD5). As such I think that md5 is not enough to verify a file integrity.

https://en.wikipedia.org/wiki/MD5

> Re sha2:
> 1. As above. What is the benefit.

I am neutral on sha2.
I just think that it is easier to add it now while this task is in our scope.


> 4. The same format as we do for sha1 unless there is a good reason not to.

Ack.
I am opting for "{hash} *{filename}" format then.

Apache Ant can be used to validate it, among other options.


Thank you for your review.
Comment 5 Konstantin Kolinko 2014-12-13 21:58:08 UTC
Fixed in Tomcat trunk, 8, 7 (r1645357, r1645360, r1645361) to be in 8.0.16, 7.0.58.
Comment 6 Konstantin Kolinko 2014-12-14 00:09:40 UTC
A note on backporting to Tomcat 6:

1) GPG support (<target name="sign") does not exist in Tomcat 6. I think it makes sense to backport that as well.

Revisions for this feature are r1231923, r1231947 and r1232368 (January 2012).

2) md5sum is calculated both in extras.xml and dist.xml
Comment 7 Konstantin Kolinko 2014-12-14 16:13:18 UTC
Created attachment 32287 [details]
2014-12-14_tc6_57344_sha1.patch

Patch for Tomcat 6.
Add sha1 checksums. I am not backporting GPG signing.

(In reply to Konstantin Kolinko from comment #6)
> A note on backporting to Tomcat 6:

To avoid special-casing the extras, it is possible to implement signing differently from Tomcat 7: sign all files in one step. In Ant there exists <apply/> task, that runs an external executable over a set of files.
Comment 8 Konstantin Kolinko 2015-02-18 22:05:49 UTC
The patch applied to 6.0 in r1660738, will be in 6.0.44 onwards.