Issue 127661 - SHA1/MD5 in sal/rtl/source/digest.c give invalid results for some inputs (ODF interop issues)
Summary: SHA1/MD5 in sal/rtl/source/digest.c give invalid results for some inputs (ODF...
Alias: None
Product: General
Classification: Code
Component: security (show other issues)
Version: 3.3.0 or older (OOo)
Hardware: All All
: P5 (lowest) Major (vote)
Target Milestone: ---
Assignee: AOO security list
QA Contact:
Depends on:
Reported: 2018-01-15 22:17 UTC by JimF
Modified: 2019-07-27 08:21 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Note You need to log in before you can comment on or make changes to this issue.
Description JimF 2018-01-15 22:17:52 UTC
There are 2 bugs in sal/rtl/source/digest.c  The bug will give incorrect results for SHA1 and MD5 when the last limb of the hashed data is between 52 and 55 bytes long.  So using that code to hash any data which is   52 <= length(data)%64 <= 55 will give bogus hash results.

The bug is seen on lines 710 and line 1160.  The bug is in the endMD5 and endSHA1 functions.  The bug has been reported to Libre Office, (same source base), and is being actively handled there.  This is the bug report made on Libre office, which lists findings to date, along with the code changes and plans for that product.  The bug there stems from the source being forked from Apache at some time, and the bug already being in that code base.

The real problem is that the SHA1 is used in the encryption validation of ODF encrypted data files.  Since this broken hash has been used to create these encrypted files, then the broken code must be preserved to be used as a fall-back when opening existing ODF files.    If the code for the SHA1 in this spot is simply 'fixed', then there will be many documents rendered non usable. Also, if this bug is left alone with no change, then documents created by Libre office (or others) which use correct SHA1 will not be usable.  The method of using both hash algorithms (the correct one, and the buggy one), and using the first that 'works', is probably the only way to proceed, without impacting existing customer data files.
Comment 1 oooforum (fr) 2019-07-22 13:03:00 UTC
>The bug has been reported to Libre Office, (same source base), and is being
>actively handled there.
Well, maybe you could ask to the developer to publish his patch under AL v.2. We'll happy to commit it into AOO trunk.
Comment 2 JimF 2019-07-24 19:02:22 UTC
I am not part of either the AOO group (in any way) or in the Libra office group.  I simply found this bug while building other tools to process the ODF data files, doing data recovery.   There were files created by both suites which were NOT correctly finding known passwords using FIPS compliant SHA1 algorithm.  I simply investigated the raw source, found the problem, was fully able to recreate the problem, AND to recreate the broken values found in the file (thus matching the hash), by adding an extra 64 bytes of NULL data to the data being hashed.

I simply brought the bug forward to both Apache and to Libre.  So far, Libra has taken steps to correct their SHA1 and MD5. I believe they may have removed the MD5 code, since it was deprecated and unused.  For the SHA1, they corrected the hash, BUT kept the older buggy code.  Upon import of ODF file, if the correct hash did not correctly work for the user password, the buggy code was used.  If that buggy code produced a correct match, then the ODF file was allowed to be properly opened, BUT when re-saved, the ODF file was done so using proper SHA1, so that the non-compliant saved file was not carried forward.

NOTE, Libre actually HAD test vectors which they had commented out (for longer hashes), due to those test vectors failing.  However, once the bug was fixed as I recommended, those test vectors worked properly. The test vectors were FIPS compliant, so with the buggy code, those test vectors were failing.  Now they can be included due to proper functionality and the hashes produced all now match.

How the Apache group chooses to deal with this bug is beyond the scope of my involvement.  It certainly is not the worst issue ever.  A ODF file saved by AOO which triggers the bug will be validated and opened by AOO with the buggy SHA1.  But it will not be usable in any other suite.  I am simply the messenger, finding and documenting the bug, and trying my best to document how to do the 'raw' fix, and also posting warnings about fixing this and then having users already saved files (in the non-compliant format) NOT being able to be opened properly using only the new correct and compliant code.

Comment 3 oooforum (fr) 2019-07-25 07:28:06 UTC
(In reply to JimF from comment #2)
> I am simply the messenger, finding and documenting the bug
Well, AOO project is only composed of volunteers
And a messenger can involved and become one of them.
Comment 4 Peter 2019-07-27 08:21:59 UTC
I have asked on LO dev List and received no answer since.

The discussion on OpenOffice dev List surfaced a nice alternative by using the code from APR. This is maybe more effort but is in general a more interesting direction to follow.