I have generated 2 MSI files 3 GB in size (one with a single large document entry and one with three smaller 1 GB entries). Trying to add an additional document entry in the file system, I get: Caused by: java.lang.IllegalArgumentException: Negative position at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:755) [rt.jar:1.8.0_121] at org.apache.poi.poifs.nio.FileBackedDataSource.write(FileBackedDataSource.java:120) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.createBlockIfNeeded(NPOIFSFileSystem.java:505) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422) [poi-3.15.jar:3.15] at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.createDocument(NPOIFSFileSystem.java:689) [poi-3.15.jar:3.15] at org.signserver.module.msauthcode.signer.MSAuthCodeSigner.signMSI(MSAuthCodeSigner.java:570) [SignServer-Module-MSAuthCode-4.1.0-SNAPSHOT.jar:] I think the culprit here is the prevBlock and nextBlock index "pointers" in StreamBlockByteBuffer being ints causing an overflow.
I can't seem to find the source code for MSAuthCodeSigner, so it's hard to tell if this is a POI bug or a mistake in the calling code. (It seems to be missing from the module list in https://svn.cesecore.eu/svn/signserver/trunk/signserver/modules/ ) Are you able to share the POI-related lines of code from there, so we can take a look? (If it is a bug, we'll need to write a unit test, which'll require the code steps taken to reproduce)
Right, missed that, sorry about that. This is in a module in-development. The code looks something like this: final NPOIFSFileSystem fsOut = new NPOIFSFileSystem(responseData.getAsFile(), false)) { // Add the signature file fsOut.createDocument(new ByteArrayInputStream(signedbytes), "\05DigitalSignature"); where the getAsFile() call would return a File instance pointing to the resulting file (which is just a copy of the original file). and the signedbytes is a byte array with the signature data (I don't think the content should really matter in this case. I could try to write a simple stand-alone program to expose the issue as well.
Btw. I've received a contribution from Sebastian S. a while ago, which I eventually plan to integrate ... maybe it's a help to you ... http://people.apache.org/~kiwiwings/20140908_MSIAPI_JUnitTests.zip
If you could knock up a failing unit test based on a temp File and some huge empty starting entries, that'd be great! If not, I can try later
Created attachment 34697 [details] Stand-alone program to expose bug The supplied program can be run with an input file as commandline argument. The file is overwritten (if successful).
(In reply to Marcus Lundblad from comment #5) > Created attachment 34697 [details] > Stand-alone program to expose bug > > The supplied program can be run with an input file as commandline argument. > The file is overwritten (if successful). https://www.update.uu.se/~ml/large.zip An archive containing a very large MSI file. NOTE: the file will decompress to a very large file. The MSI file was created with msibuilder (from msitools) and only contains document streams filled with zeros. It will not actually work as a real installer in Windows.
I've just tried with your program and your 3gb msi file, but I think the file might be corrupt? When I run the program, it fails with: Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 8, Size: 8 at java.util.ArrayList.rangeCheck(ArrayList.java:635) at java.util.ArrayList.get(ArrayList.java:411) at org.apache.poi.poifs.property.PropertyTableBase.populatePropertyTree(PropertyTableBase.java:128) at org.apache.poi.poifs.property.PropertyTableBase.<init>(PropertyTableBase.java:63) at org.apache.poi.poifs.property.NPropertyTable.<init>(NPropertyTable.java:66) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.readCoreContents(NPOIFSFileSystem.java:440) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:235) at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:168) at Main.main(Main.java:18) I've done a fix for PropertyTableBase to skip broken entries. That then lets me run your program without error
(In reply to Nick Burch from comment #7) > I've just tried with your program and your 3gb msi file, but I think the > file might be corrupt? When I run the program, it fails with: > > Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 8, > Size: 8 > at java.util.ArrayList.rangeCheck(ArrayList.java:635) > at java.util.ArrayList.get(ArrayList.java:411) > at > org.apache.poi.poifs.property.PropertyTableBase. > populatePropertyTree(PropertyTableBase.java:128) > at > org.apache.poi.poifs.property.PropertyTableBase.<init>(PropertyTableBase. > java:63) > at > org.apache.poi.poifs.property.NPropertyTable.<init>(NPropertyTable.java:66) > at > org.apache.poi.poifs.filesystem.NPOIFSFileSystem. > readCoreContents(NPOIFSFileSystem.java:440) > at > org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem. > java:235) > at > org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem. > java:168) > at Main.main(Main.java:18) > > I've done a fix for PropertyTableBase to skip broken entries. That then lets > me run your program without error I can still reproduce the "Negative position" exception problem with a file generated by the WiX toolset. I only managed to generate a very large file with random data (so it won't compress, will see if I can still provide a link to download it). Also, it did actually work with my sample test program, but when I increased the size of the byte array to add as an extra document to 10000 bytes I got the same error again, so I guess it might be due to padding (when a new block needs to be created).