ConfigurationSource is 5 years old.

TOMCAT_HOME has never been an environment variable used by Tomcat.

The system property "user.dir" is essentially the same as the CWD of the process when it was started. It's not the same as the user's home directory.

The only time "user.dir" is used is if the code initializing Tomcat doesn't give it any specific information about where to load the configuration files from.

Do you have a situation where Tomcat is actually doing something incorrect/unexpected or is this a complaint filed after reading the source code?

I will review next week. It is very convoluted so it's possible something is missing.

If you want to separate the binaries of tomcat from it's configuration folder, then user.dir complicates the matter immensely as initial read of server.xml where you can configure listener to register a source depends on DEFAULT ConfigurationSource.
Maybe ConfigurationSource is 5 years old, but behavior changed with tomcat 9. It was not used as default for tomcat 8

"The only time "user.dir" is used is if the code initializing Tomcat doesn't give it any specific information about where to load the configuration files from."
How do you give specific information about where to load the configuration files from initially?
I didn't find a way to do that on initial start except having 2 server.xml files, one in user.dir and one in my separate location for configuration, because CATALINA_BASE environment variable is not respected as in tomcat 8

(In reply to 7element from comment #4)
> "The only time "user.dir" is used is if the code initializing Tomcat doesn't
> give it any specific information about where to load the configuration files
> from."
> How do you give specific information about where to load the configuration
> files from initially?

The only place ConfigurationSource.DEFAULT is used is in (static) ConfigFileLoader.getSource. If you want to override the default, then call (static) ConfigFileLoader.setSource with whatever you want.

See o.a.catalina.Catalina.parseServerXml for an example of how to set it to something like what you are expecting.

> I didn't find a way to do that on initial start except having 2 server.xml
> files, one in user.dir and one in my separate location for configuration,
> because CATALINA_BASE environment variable is not respected as in tomcat 8

If you are using Tomcat embedded, then you need to tell Tomcat where to find your configuration files.

I think I can't explain it good, but I'll try
Install tomcat 8, remove conf folder, set CATALINA_BASE outside of tomcat 8 folder with valid (even default configuration). Start tomcat. Check the result.
Do the same with tomcat 9 and you'll understand what I mean.
In this case "If you want to override the default, then call (static) ConfigFileLoader.setSource with whatever you want", I don't want to call anything. just to move configuration in different location and set CATALINA_BASE with no coding whatsoever.

Your reproduction suggestion sounds easy, but it misses a lot of important precision. Since separating out CATALINA_BASE is a working recipe for many of us, could you please answer:

- which OS is this on? Linux? Windows?

- what exact Tomcat version do you use?

- how do you exactly set CATALINA_BASE?

- is your new conf folder underneath the set value for CATALINA_BASE, so something like $CATALINA_BASE/conf?

- which other directories and files do you have underneath CATALINA_BASE$
  - For example webapps and work?

- do you also set CATALINA_HOME to the original product directory?

- how do you exactly start Tomcat?
  For instance using the standartup.(sh|bat) script Tomcat provides?

- in case you do not start it as a service: in which directory is your shell process when you execute the start commands?

Thanks and regard,

Rainer

(In reply to Rainer Jung from comment #7)
> Your reproduction suggestion sounds easy, but it misses a lot of important
> precision. Since separating out CATALINA_BASE is a working recipe for many
> of us, could you please answer:
> 
> - which OS is this on? Linux? Windows?

Both Linux and Windows

> 
> - what exact Tomcat version do you use?

I try to seamlessly upgrade from 8.5 to 9/10

> 
> - how do you exactly set CATALINA_BASE?

export CATALINA_BASE=/home/user/.tomcat (conf/server.xml, conf/web.xml, conf/context.xml, conf/localhost.jks, lib)
set CATALINA_BASE=C:\Users\user\tomcat (conf\server.xml, conf/web.xml, conf/context.xml, conf/localhost.jks, lib)
From server.xml you can set work, webapps, lib folders if it finds proper one at startup

> 
> - is your new conf folder underneath the set value for CATALINA_BASE, so
> something like $CATALINA_BASE/conf?

Yes

> 
> - which other directories and files do you have underneath CATALINA_BASE$
>   - For example webapps and work?

Nothing else. From server.xml you can set work, webapps

> 
> - do you also set CATALINA_HOME to the original product directory?

Yes, but they point to the same place

> 
> - how do you exactly start Tomcat?
>   For instance using the standartup.(sh|bat) script Tomcat provides?

Yes. the standartup.(sh|bat) script Tomcat provides
> 
> - in case you do not start it as a service: in which directory is your shell
> process when you execute the start commands?

Any folder. User on the system can start it's own process and point to his configuration folder by using CATALINA_BASE. For example

On linux:
/var/lib/tomcat for binaries
/home/user/.tomcat/conf for server.xml, certificates, global web.xml, global context.xml
/home/user/.tomcat/workspace for webapps, work, etc. I can point to those via server.xml using appBase and workDir

On Windows: C:\Program Files\tomcat for binaries
C:\Users\user\tomcat\conf for server.xml, certificates, global web.xml, global context.xml
C:\Users\user\workspace for webapps, work. I can point to those via server.xml using appBase and workDir

> 
> Thanks and regard,
> 
> Rainer

Thank you very much for the patience

Best regards

ConfigurationSource.DEFAULT is a very bare bones default that is not part of Catalina. Catalina is the component that uses the CATALINA_BASE env variable (as the name implies). The ConfigurationSource implementation that implements this behavior is org.apache.catalina.startup.CatalinaBaseConfigurationSource which should be properly setup by the Catalina startup. If you are using some exotic environment, then you need to either setup Catalina properly or implement a ConfigurationSource that works for your environment.

I'm sorry, but I don't find it exotic to request CATALINA_BASE to be respected when tomcat initially boots up and not only user.dir to be used to search for server.xml
As I said already, all I want is backward compatibility as this was the case until 8.5
On top I don't want to change the logic and break currently released tomcat versions
All I want is to add a backward compatible way to search for the configuration, i.e. to search for additional place (CATALINA_BASE) for server.xml as a last resort, before failing and continue with the defaults.
I couldn't find a way to change with configuration org.apache.catalina.startup.CatalinaBaseConfigurationSource to my own ConfigurationSource and that is the reason for this request.

Best Regards

Please do not reopen the BZ. Please use the user mailing list to discuss this further instead. If a problem is found, then it will be addressed just the same.

The shell scripts will pass the CATALINA_BASE environment variable as the catalina.base Java system property, which will then be used to setup the paths that are used by CatalinaBaseConfigurationSource. I fail to see a behavior change there.

Note: 8.5 to 9.0 is a major version update, so some change of behavior is acceptable, and some configuration updates could be required.

[remm@feather build]$ export CATALINA_BASE=/tmp
[remm@feather build]$ ./bin/catalina.sh run
Using CATALINA_BASE:   /tmp
Using CATALINA_HOME:   /home/remm/Work/tomcat/apache-tomcat-trunk/output/build
Using CATALINA_TMPDIR: /tmp/temp
Using JRE_HOME:        /usr/lib/jvm/java-22
Using CLASSPATH:       /home/remm/Work/tomcat/apache-tomcat-trunk/output/build/bin/bootstrap.jar:/home/remm/Work/tomcat/apache-tomcat-trunk/output/build/bin/tomcat-juli.jar
Using CATALINA_OPTS:   
WARNING: java.io.tmpdir directory does not exist
Apr 23, 2024 10:18:24 AM org.apache.catalina.startup.Catalina parseServerXml
WARNING: Unable to load server configuration from [/tmp/conf/server.xml]
java.io.FileNotFoundException: /tmp/conf/server.xml (No such file or directory)
	at java.base/java.io.FileInputStream.open0(Native Method)
	at java.base/java.io.FileInputStream.open(FileInputStream.java:213)
	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:152)
	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:106)
	at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:84)
	at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:186)
	at org.apache.catalina.startup.CatalinaBaseConfigurationSource.getResource(CatalinaBaseConfigurationSource.java:120)
	at org.apache.tomcat.util.file.ConfigurationSource.getConfResource(ConfigurationSource.java:150)
	at org.apache.tomcat.util.file.ConfigurationSource.getServerXml(ConfigurationSource.java:127)
	at org.apache.catalina.startup.CatalinaBaseConfigurationSource.getServerXml(CatalinaBaseConfigurationSource.java:54)
	at org.apache.catalina.startup.Catalina.parseServerXml(Catalina.java:590)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:691)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:729)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:299)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:469)

Created attachment 39682 [details]
Patch

Propose a patch with the requested change

(In reply to 7element from comment #12)
> Created attachment 39682 [details]
> Patch
> 
> Propose a patch with the requested change

The patch does not seem to do anything, since ConfigurationSource.super.getServerXml() default implementation also calls getResource (the idea is to allow overriding).