Hi everybody Since tomcat 9.0.13 and the ability to monitor "tomcat-users.xml" was introduced we encounter quite strange behaviour: we run into "too many files open" exception after a certain time when tomcat is running. nothing changed on the apps which run on this instance, except the version of tomcat (from 9.0.12 to 9.0.13) I found now the following strange thing: when doing "lsof" on the PID runnning tomcat, I see: [gtt@tgttcn0:~/tomcat/logs:1061> lsof -p 16829 | grep tomcat-users | wc -l 260 a couple of seconds later: [gtt@tgttcn0:~/tomcat/logs:1062> lsof -p 16829 | grep tomcat-users | wc -l 262 and about 20 seconds later: [gtt@tgttcn0:~/tomcat/logs:1062> lsof -p 16829 | grep tomcat-users | wc -l 264 and nothing is really going on on the tomcat at the moment. It looks like something is opening tomcat-users.xml all the time and does not close it anymore.... Here the same "lsof" command on one of our servers running tomcat 9.0.12: [gtt@pgttcn1:~:1008> lsof -p 26572 | grep tomcat-users | wc -l 0 thanks for having a look into this Erich
here a full listing of the "lsof" command to better illustrate the issue.... [gtt@tgttcn0:~/tomcat/logs:1064> lsof -p 16829 | grep tomcat-users java 16829 gtt 68r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 71r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 75r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 77r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 78r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 79r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 80r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml java 16829 gtt 81r REG 259,5 241 268435594 /home/gtt/apache-tomcat-9.0.13/conf/tomcat-users.xml
ok, I digged into this a little bit deeper -- I figured out that there is a new flag "watchSource=false" (default is true) which can be set: <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" watchSource="false" /> when I set the flag to false, everything seems to be fine! I suspect the issue is somewhere here: MemoryUserDatabase.java in method "public void backgroundProcess()" which reads the file: [...] URI uri = ConfigFileLoader.getURI(getPathname()); try { URL url = uri.toURL(); URLConnection uConn = url.openConnection(); [...] but it looks like it remains open....
There is nothing that can be closed here, and the URL is a regular file URL. However, calling getLastModified on it indeed leaks a descriptor. There's a problem somewhere and more investigation is needed, but this is odd.
Thanks for the report. Fixed in: - trunk for 9.0.14 onwards
*** Bug 62958 has been marked as a duplicate of this bug. ***
I found a question about a possibly related bug in Tomcat https://stackoverflow.com/questions/53745188/tomcat-undeploys-wars-automatically-after-some-time. Could the log include details of the exception to easily confirm the cause of the issue? https://github.com/apache/tomcat/blob/40500131c71ec8116eb35628ddf6abd515c012ba/java/org/apache/catalina/users/MemoryUserDatabase.java#L447
(In reply to Eduardo Guadalupe Quintanilla from comment #6) > I found a question about a possibly related bug in Tomcat > https://stackoverflow.com/questions/53745188/tomcat-undeploys-wars- > automatically-after-some-time. > > Could the log include details of the exception to easily confirm the cause > of the issue? > > https://github.com/apache/tomcat/blob/ > 40500131c71ec8116eb35628ddf6abd515c012ba/java/org/apache/catalina/users/ > MemoryUserDatabase.java#L447 That is a network issue and unrelated to this bug.