Bug 64074

Summary: getResource() for directories started to return nulls instead of list of files that directory contains
Product: Tomcat 9 Reporter: Konrad <kcichocki>
Component: CatalinaAssignee: Tomcat Developers Mailing List <dev>
Severity: regression    
Priority: P2    
Version: 9.0.30   
Target Milestone: -----   
Hardware: PC   
OS: All   
Attachments: a test case that reproduces a bug

Description Konrad 2020-01-13 23:18:58 UTC
Created attachment 36963 [details]
a test case that reproduces a bug

In tomcat 9.0.27 when you invoke getClass().getResource(<directory>).getInputStream() you would get an input stream containing all files from the directory. In tomcat 9.0.30 that logic has changed and you get a null instead of an input stream.

We created a small test project to reproduce this error (added to this ticket as an attachment). When you deploy it to the tomcat 9.0.27 (under namespace test_tomcat9_war) and access http://localhost:8080/test_tomcat9_war/DemoServlet you will get 

"Full: blah.txt blah2.txt test" message

while when you deploy the same artifact to the tomcat 9.0.30 you will get 

"Full was null"
We did some investigation on our side about what has changed and it looks like

When you, in a DemoServlet, invoke getClass().getResource("/full/") you get an URL object back in both cases. 
in tomcat 9.0.30:
you will get an URL object with a handler of class org.apache.catalina.webresources.CachedResource$CachedResourceURLStreamHandler
then when you call openConnection on that URL object it will delegate the openConnection to the mentioned handler. That will return org.apache.catalina.webresources.CachedResource$CachedResourceURLConnection object. Then when you call getInputStream on it it will return you a null because it doesn't handle properly a case when the resource is a directory. 
This is a simplified description as there are more layers of delegation here - eventually the org/apache/catalina/webresources/FileResource.java doGetInputStream() method is invoked and it fails when it tries to return FileInputStream(resource) - because the resource is a directory. 

in tomcat 9.0.27
you will get an URL object with a handler of class sun.net.www.protocol.file.Handler then when you call openConnection on that RUL object it will delegate the openConnection to the mentioned handler again. That will return class sun.net.www.protocol.file.FileURLConnection object. Then when you call getInputStream() method on it it will return you an input stream that has a list of files in that directory (the FileURL connection object has a special logic to handle directories.

Parts of our system rely on this functionality and because of this bug we can't update to the newest version of tomcat which may be an issue especially when there is some security fix to be applied.
Comment 1 Remy Maucherat 2020-01-14 08:24:59 UTC
This is some custom behavior that worked by accident, getInputStream applies to files, not directories.
This will probably be WONTFIX.
Comment 2 Mark Thomas 2020-01-14 08:29:22 UTC
I'll see how easy it is to restore the directory listing behaviour.
Comment 3 Mark Thomas 2020-01-14 17:23:23 UTC
It wasn't too bad. And as a nice side-effect it takes account of resourceJARS and packedWARs when caching is enabled.

Fixed in:
- master for onwards
- 9.0.x for 9.0.31 onwards
- 8.5.x for 8.5.51 onwards

Unless we completely re-implement the JNDI resource handling from 7.0.x (not something I want to do) there is always going to be some edge case that isn't supported. At the moment the edge cases that are emerging are simple to implement. At some point we will reach the point we may reach the point where we decide the implementation is too complex.
Comment 4 Konrad 2020-01-15 01:11:58 UTC
Whoa, that was quick, thank you very much!