Our webapp is using bouncycastle security provider, which was packed within the WEB-INF/lib folder. Furthermore our webapp run within a tomcat environment, which uses unpackWars="false". This woked fine with Tomcat6 and Tomcat7. After switching to Tomcat8 we got an exception: Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC at javax.crypto.Cipher.getInstance(Cipher.java:642) at javax.crypto.Cipher.getInstance(Cipher.java:580) at de.balvi.xmljobs.job.domain.JobParameterType$PasswordJobParameterType.createCipher(JobParameterType.java:200) at de.balvi.xmljobs.job.domain.JobParameterType$PasswordJobParameterType.toPersistent(JobParameterType.java:169) ... 96 more Caused by: java.util.jar.JarException: jar:file:/D:/Java_Bin/BALVI_TOMCAT/webapps/ROOT.war!/WEB-INF/lib/bcprov-jdk15on-1.51.jar has unsigned entries - WEB-INF/css/bootstrap-responsive.min.css at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:462) at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:322) at javax.crypto.JarVerifier.verify(JarVerifier.java:250) at javax.crypto.JceSecurity.verifyProviderJar(JceSecurity.java:161) at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:187) at javax.crypto.Cipher.getInstance(Cipher.java:638) at javax.crypto.Cipher.getInstance(Cipher.java:580) at de.balvi.xmljobs.job.domain.JobParameterType$PasswordJobParameterType.createCipher(JobParameterType.java:200) The exception "java.util.jar.JarException: jar:file:/D:/Java_Bin/BALVI_TOMCAT/webapps/ROOT.war!/WEB-INF/lib/bcprov-jdk15on-1.51.jar has unsigned entries - WEB-INF/css/bootstrap-responsive.min.css" states to an entry of the WAR itself instead of an entry of the bcprov-jdk150-1.51.jar. Extract from JobParameterType$PasswordJobParameterType.createCipher(JobParameterType.java:200) states to private Cipher createCipher(int encryptMode) throws NoSuchAlgorithmException, NoSuchProviderException, NoSuchPaddingException, InvalidKeyException, InvalidAlgorithmParameterException { // Register BouncyCastleProvider Security.addProvider(new BouncyCastleProvider()); // Create the IV Key AlgorithmParameterSpec IVspec = new IvParameterSpec(getIV128Key()); // Create the Cipher für Decrypting Cipher encryptCipher = Cipher.getInstance("AES/CBC/PKCS5Padding", "BC"); // <== line 200 SecretKey keyValue = new SecretKeySpec(getAES128Key(), "AES"); encryptCipher.init(encryptMode, keyValue, IVspec); return encryptCipher; } As workaround, i removed the bouncycastle provider from the WEB-INF/lib and put it to CATALINA_BASE/lib. A second problem occured, when a uri resource is loaded from a jar inside a unpacked war: com.mycila.xmltool.XMLDocumentException: Validation failed: Illegal character in opaque part at index 55: jar:war:file:/D:/Java_Bin/BALVI_TOMCAT/webapps/ROOT.war^/WEB-INF/lib/xmljobs-1.1.1.jar!/xmljobs/groovy-flow-1.0.xsd The RFC 2396 "URI Generic Syntax" says, the char ^ is an "unwise" uri char, which causes a java.net.URISyntaxException in our environment. As workaround, we changed the resource loader to return a Stream instead of the uri string. Both problems are reproducable with any java platform. (Tried Java 7/8 on Windows X86_64, Linux X86_64) with Tomcat 8. Tomcat 7 has no problem. Please check the changes form 7 to 8.
Neither of these are issues with Tomcat. The first is an invalid assumption in javax.crypto about the location of the provided classes and the second is overly strict URI validation in your environment. Another workaround would be to run with unpackWARs. A further advantage will be a performance improvement in class loading. For a fuller discussion of the performance impact of unpackWARs see Bug 57251.
Hi again, i'm sorry to bother you again, but the slight difference between Tomcat 7 and Tomcat 8 causes many support trouble for me. To use unpackWars="true" is a problem for us. As you suggested, unpackWars="true" worked, but unpackWars="false" seems to be broken with Tomcat 8. So i digged a little deeper into the problem. I found out that Tomcat 7 and Tomcat 8 both extract classes and libs from WEB-INF to CATALINA_BASE/work/[app]/, even if unpackWars="false" is set. The difference seems to be that Tomcat 7 delives resource or class loader requests from the "work" direktory. So there is at least a file to open and all things worked fine. The javax.crypto.JarVerifier.verifySingleJar is happy and bouncycastle provider gets registered. With Tomcat 8 the things changed, so the classloader(?) returns an URI to the relativ path within the war instead of the jar in "work". This causes javax.crypto.JarVerifier.verifySingleJar to verify the WAR instead of the jar. The same behavior seems to match for the resource loader, which delived a relative path for our xsd stored within a jar. As Tomcat 7 worked fine i assume that many other webapps would have the same problem, if they running with unpackWars="false" So i beg you, please take a look at this behavior change from Tomcat 7 to Tomcat 8 Kind regards, Frank
(In reply to Frank Holler from comment #2) > To use unpackWars="true" is a problem for us. Why? > So i digged a little deeper into the problem. > I found out that Tomcat 7 and Tomcat 8 both extract classes and libs from > WEB-INF to CATALINA_BASE/work/[app]/, even if unpackWars="false" is set. That is not correct. Tomcat 8 does not extract JARs or classes to the work directory under any circumstances. One of the options considered to address the performance problems is to restore the extraction of the JARs to the work directory but that creates complexity in an area where we have been trying to reduce it. This is still an option of last resort but my preference - where possible - is to address the reasons why people can't use unpackWARs="true".
(In reply to Mark Thomas from comment #3) > (In reply to Frank Holler from comment #2) > > To use unpackWars="true" is a problem for us. > > Why? > > > So i digged a little deeper into the problem. > > I found out that Tomcat 7 and Tomcat 8 both extract classes and libs from > > WEB-INF to CATALINA_BASE/work/[app]/, even if unpackWars="false" is set. > > That is not correct. Tomcat 8 does not extract JARs or classes to the work > directory under any circumstances. > > One of the options considered to address the performance problems is to > restore the extraction of the JARs to the work directory but that creates > complexity in an area where we have been trying to reduce it. This is still > an option of last resort but my preference - where possible - is to address > the reasons why people can't use unpackWARs="true". You're rigt, my fault. i did not clean up the work directory after changing form unpackWars="true" to "false". So the remaining files was leftover from the last restart. The reason why we don't use unpackWars="true" is, that the unpacked directory won't get updated, if the war changes. For security reasons we (and our customers) don't deploy the "manager" or any other webapp, only our war. So the deployment process with unpackWars="false" is: 1.) drop the new war into webapps 2.) restart tomcat With unpackWars="true" and without using "manager"-app the steps to update would be: 1.) stop tomcat 2.) delete the unpacked folder => otherwise it won't work!!! 3.) drop the new war into webapps 4.) start tomcat again To check, if this behavior changed with tomcat 8 we did the following: Our webapp version 2.6.6 is deployed with unpackwars="true", the unpacked folder exists. Afterwards we replaced the WAR with version 2.6.7 and restart tomcat. As any version before, the unpacked directory wasn't updated, so the version 2.6.6 is started again, the new war with version 2.6.7 is ignored. After this problem occured often, we told our customers to set unpackWars="false", so our customers (and our support of course) has less problems when delvierung updates of our app. So perhaps you understand, why we won't like to tell our customers to change unpackWars back when updating to tomcat 8. I understand, that a revert of the changes isn't easy, but for my opinion, unpackWars="false" isn't usable with tomcat 8 any more without refactoring many of our logic to bypass the new loader behavior. The problem is, that we don't know, which of the used libs also have a problem (like com.mycila.xmltool), so we would be happy, if the behavior of tomcat 8 keeps compatible tomcat 7. Kind regards, Frank
(In reply to Frank Holler from comment #4) > The reason why we don't use unpackWars="true" is, that the unpacked > directory won't get updated, if the war changes. For security reasons we > (and our customers) don't deploy the "manager" or any other webapp, only our > war. > > So the deployment process with unpackWars="false" is: > 1.) drop the new war into webapps > 2.) restart tomcat You must have also set autoDeploy="false" otherwise dropping an updated WAR into a running Tomcat instance would trigger a redploy. > With unpackWars="true" and without using "manager"-app the steps to update > would be: > 1.) stop tomcat > 2.) delete the unpacked folder => otherwise it won't work!!! > 3.) drop the new war into webapps > 4.) start tomcat again Not the case as of Tomcat 8.0.21. On restart, Tomcat will now detect the updated WAR, delete the expanded directory and expand the updated WAR (assuming deployOnStartup="true"). > I understand, that a revert of the changes isn't easy, but for my opinion, > unpackWars="false" isn't usable with tomcat 8 any more without refactoring > many of our logic to bypass the new loader behavior. > The problem is, that we don't know, which of the used libs also have a > problem (like com.mycila.xmltool), so we would be happy, if the behavior of > tomcat 8 keeps compatible tomcat 7. Tomcat's general policy for adding workarounds for bugs in other projects is that we won't do it. Fixing the issue at the source means the fix is more widely available and we have more time to spend on Tomcat bugs and new feature requests. We do make occasional exceptions - usually when the bug is likely to affect a large proportion of the Tomcat user based and the 3rd party vendor has indicated that they have no intention of fixing the bug. The new ability to detect an updated WAR while Tomcat has stopped looks to be sufficient to remove the need for your application to run with unpackWARs="false". The behaviour of javax.crypto with an unpacked WAR is a concern although we don't see as much bouncy castle usage as we used to. The issue with com.mycila.xmltool falls into the category that we'd expect the vendor to fix. As per the discussion on bug 57251 and as stated above, the preference remains to fix issues people have with unpackWARs="true". I haven't seen anything in this issue that explains why - as of 8.0.21 - running with unpackWARs="true" is not a viable option for you.
Created attachment 32597 [details] example war for getResource() from lib see README.txt embedded within the zip
(In reply to Mark Thomas from comment #5) > (In reply to Frank Holler from comment #4) > The issue with com.mycila.xmltool falls into the category that we'd expect > the vendor to fix. I created a quiet simple test project to demonstrate the problem of getClass().getResource(), which returns an URL. So in my opinion, this is no "vendor specific" problem, because i only used Java Runtime imports, no additional imports are needed. Kind regards, Frank
(In reply to Frank Holler from comment #7) > So in my opinion, this is no "vendor specific" problem, because i only used > Java Runtime imports, no additional imports are needed. Accepted. The format we use for WAR URLs fails if toURI() is called on it. That needs to get fixed. I'll look at that today. You still haven't responded to this: "I haven't seen anything in this issue that explains why - as of 8.0.21 - running with unpackWARs="true" is not a viable option for you."
From Tomcat 8.0.22 onwards (the 8.0.21 release is currently in progress so 8.0.22 should be within the next month) */ is used rather than ^/ which is valid as per RFC 2396. The old format is still supported if it is used but any URLs returned by Tomcat will use the new format. I've also added some test cases to validate these changes. By my count that leaves the javax.crypto issue - which will looks like a JRE bug - that you should be able to work-around with unpackWARs="true".
(In reply to Mark Thomas from comment #9) > From Tomcat 8.0.22 onwards (the 8.0.21 release is currently in progress so > 8.0.22 should be within the next month) */ is used rather than ^/ which is > valid as per RFC 2396. The old format is still supported if it is used but > any URLs returned by Tomcat will use the new format. I've also added some > test cases to validate these changes. > > By my count that leaves the javax.crypto issue - which will looks like a JRE > bug - that you should be able to work-around with unpackWARs="true". Sounds good from me. We will set the pom dependy to "provided" an put the bouncy castle provider to CATALINA_BAS/lib. If our customers won't use unpackWars="true", i'll expect that our app will work onTomcat 8 using 8.0.21 and abouve. Thanks for the fast and qualified help. Kind regards, Frank
Just to be clear, the URLs for WAR resources fix will be in 8.0.22 onwards.
Apologies for re-opening this case. I'm using the RSA crypto jars and have the same behavior with 8.0.30. I think the thread below misses the point of where the actual bug is. When not unpacking the war file, the jar signature check is seeing that the jar is signed (in my case the RSA jar, in the previous user's case, the BC jar), but is then checking the signature on the war file itself, instead of the jar. It seems that the functionality of leaving a war file packed, where the war file has a signed jar in its WEB-INF/lib, is simply broken. Of course there are workarounds - unpacking the war, or moving the signed jars into the tomcat/lib folder - but these are both workarounds for something that is broken. If it is the intent of the tomcat community to no longer allow/support signed jars inside packed war files, then there is no choice but to use a workaround, and this limitation should be called out. But I suspect that this behavior is probably inadvertent and should be fixed, rather than insist that war files with signed jars must be unpacked in tomcat8 and beyond. What do you think?
Let me be more specific - the jar verifier is complaining that the signed jar contains other unrelated resources (a JSP page), which are actually in the war, not the jar at all: Caused by: java.util.jar.JarException: jar:file:/D:/IDFC/SSORest/app/gateway-2.11.3-SNAPSHOT.war!/WEB-INF/lib/cryptoj-12.52.0.0.jar has unsigned entries - publicauth.jsp at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464) at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:322) at javax.crypto.JarVerifier.verify(JarVerifier.java:250) at javax.crypto.JceSecurity.verifyProviderJar(JceSecurity.java:160)
I can't repeat the problem you are reporting. Please provide a test case that reproduces this issue with one of: - 8.0.x trunk - 9.0.x trunk - latest 8.0.x release - latest 9.0.x release The test case should be as simple as an index.jsp page to package with the Bouncy Castle provider JAR (I don't have easy access to Crypto-J but if this is the same issue it will be reproducible with BC) in a packed WAR that triggers the exception. The test case I am currently using that loads the BC provider works without error.
No further information provided. If you still observe the problem described in comment #12 then please create a new issue and provide a test case that can be used to reproduce the problem.