Bug 47467

Summary: Deployment of the war file by URL when contextpath is not specified in Manager Application.
Product: Tomcat 9 Reporter: meetaarora3007
Component: CatalinaAssignee: Tomcat Developers Mailing List <dev>
Severity: enhancement CC: bkuker, jernst, uwe
Priority: P2    
Version: unspecified   
Target Milestone: -----   
Hardware: All   
OS: All   
Attachments: Toughts about ManagerServlet#deploy(.....) cases and which are getting wrong and which are doable
Generated Excel File when TestPoiPrintArea.java is run with poi-3.7-SNAPSHOT-20100610.jar

Description meetaarora3007 2009-07-02 05:50:13 UTC
I have seen a problem in Tomact manager application. According to the Tomcat documentation6.0 

Deploy a Directory or WAR by URL
Deploy a web application directory or ".war" file located on the Tomcat server. If no path is specified, the directory name or the war file name without the ".war" extension is used as the path. The war parameter specifies a URL (including the file: scheme) for either a directory or a web application archive (WAR) file. The supported syntax for a URL referring to a WAR file is described on the Javadocs page for the java.net.JarURLConnection class. Use only URLs that refer to the entire WAR file.

In this example the web application located in the directory /path/to/foo on the Tomcat server is deployed as the web application context named /footoo. 



In this example the ".war" file /path/to/bar.war on the Tomcat server is deployed as the web application context named /bar. Notice that there is no path parameter so the context path defaults to the name of the web application archive file without the ".war" extension. 


In the last paragraph, it is written that it is not mandatory to specify path in the manager application i.e it should be by default deployed. But it gives me an error as " Invalid context path null specified".

Steps to reproduce the scanerio:
1. Open manager application
2. Do not specify anything in ContextPath field.
3. Enter the path to the war file in the third field.

Result: It gives me the error reported above. Also my application contains an explicit path to it in context.xml.All the settings are default only. 
Here if I give an explicit path in ContextPath field in Manager application it will be deployed successfully.
Comment 1 Uwe Günther 2009-09-23 02:55:30 UTC
I just investigated a bit in this one and IMHO this bug is realy odd because the documentation in html-manager-howto.xml and o.a.catalina.manager.ManagerServlet is not in sync with what the code does or is designed for.

Let me point that out in detail:

1. The docs are talking about JarURLConnection like syntax for the 'WAR or Directory URL' field in the 'Deploy' section of HTMLManagerServlet* (aka known as war parameter there):

	jar:file:/c:/myWebapp.war!/ **

If you look in the code - you will find out - it can only handle java.io.File like file names (with an optional 'file:' prefix): 





Every other filename not ending with '.war' is handled by the code as already exploded directory and gets deployed as such!

Back to the point: The existing code in HTMLManagerServlet and ManagerServlet is unable to handle JarURLConnection like syntax like following one mentioned above: jar:file:/c:/myWebapp.war!/ To make it crystal clear here, the only type of URL they can handle are 'file:' URLs based on java.io.File(String) at all.

* Note: HTMLManagerServlet is a sub class of ManagerServlet which gets used by the Ant DeployTask as well. So there are common implications.

2.) On the one hand if you deploy a war file remote via the 'Select WAR file to upload' field in the 'Deploy' section of HTMLManagerServlet the 'Context Path' gets derived from the war file name. So you don't need a 'Context Path' to deploy a *.war file. For example if your war file is callled 'myWebapp.war', your 'Context Path' will be '/myWebapp'.
On the other hand if you try to deploy a webapp as a *.war or a directory locally via 'WAR or Directory URL' you have to provide a 'Context Path', but it would be possible to derive the 'Context Path' as well from the 'WAR or Directory URL' name like it is done for remote deployments.

Proposed Solution:

- Fix the documentaion and the HTMLManagerServlet UI to get number 1.) fixed. Especially remove the word 'URL' from the HTMLManagerServlet UI field descriptors and the jar:file:.... Urls from the javadoc and the html-manager-howto.xml

- Enhance ManagerServlet to derive the 'Context Path' (aka 'path' parameter) from an already given 'WAR or Directory' (aka 'war' parameter). For example if you have no 'Context Path' and the 'WAR or Directory' filed contains 'C:\myWebapp' or 'C:\myWebapp.war' the 'Context Path' will be '/myWebapp'.

I start to work on this from now on in the trunk and will append the patch as soon as I am finished (probably this week) to this bug, so it can merged in by one of the commiter.

Would be nice to get some feedback if the probosed solution fits or not.

Kind Regards
Uwe Günther
Comment 2 Mark Thomas 2009-09-25 14:38:52 UTC
+1 to making the docs and the functionality consistent.

Personally, I'd lean towards figuring out if the jar:file: syntax ever worked and it if was look at why/when it broke and how easy it would be to fix it before opting to remove it completely.
Comment 3 Uwe Günther 2009-09-25 16:25:48 UTC
Thats what I thought as well when I started with this bug. This jar:file: syntax is based on an URLConnection and there isn't one used in the whole MangerServlet or HTMLManagerServlet code. That shows to me  jar:file: syntax is only in the docs to document a function we simply do not have in the code at all.

I checked the svn history 2 years back and there was no jar:file:... URL syntax at all. Means I could not found any URLConnections in the code and its history. Maybe the original author could give us some historical inside knowledge, but I think this is late back to tomcat 4.x.x.

At the moment I can't even dream up a scenario where you could use a jar:file: URLConnection like syntax or where it would make sence, but thats me. :-) 

Lets go back to what does till now: There are basically two deployment strategies in the HTMLManagerServlet:

    1.) local deployment (webapp *.war or webapp dir must be locally accessable from tomcat to get copied via GET command to the webapps dir)
    2.) remote deployment (only a webapp *.war can be remotely copied to the webapps dir via POST resp. PUT)

Either you deploy a local dir or war as webapp or you deploy a remote war as webapp. In both cases the ManagerServlet does only simple copy operations to copy the local war / dir or the remotely uploaded war to the webapps dir.

Anyway, in both cases tomcat gets told via JMX in the ManagerServlet to use the new copied webapp dir resp. explode the new copied *.war file and use it.

I have taken the whole ManagerServlet#deploy(PrintWriter writer, String config, String path, String war, boolean update) method in to pieces to fully analyze and understand it, so I can say what is going on and what gets wrong here. 

There are even some other strange bugs in this method: Lets say you have 3 params, each of them can have a value or can be null, so this leads us to 2^3=8 diffrent combinations where 5 of them are complete valid ones and 3 are invalid ones. The existing code only works for 2 of 5 valid cases and does some odd thing in the other 3 cases. For more details see my attachment which is listing the cases.

I reworked only this single ManagerServlet#deploy(...) mehtod but created 4 new classes to handle complexisty / special cases, because the method had been grown in to a hard to handle beast. :-) 

Since today afternoon I have a working version running on my dev box together with the trunk code. I am currently setting up a local testing scenarios to seriously test that new code. Not only, but because it gets used via HTTP from the Ant DeployTask, the ManagerServlet and the HTMLManagerServlet webapp frontend. But it looks ok until now.

Kind Regards
Uwe Günther
Comment 4 Uwe Günther 2009-09-25 16:28:07 UTC
Created attachment 24313 [details]
Toughts about ManagerServlet#deploy(.....) cases and which are getting wrong and which are doable
Comment 5 Uwe Günther 2009-09-26 04:50:58 UTC
After some archaeological researches in older tomcat releases it came to daylight the general URL support has been gone starting with Tomcat 5.5.0. In this release o.a.c.Deployer has been removed from the Tomcat core. The Deployer interface has been used inside the ManagerServlet to drive the deployments. Deployer was a specialized Container into which web applications could be deployed and undeployed. The whole Deployer interface could only used together with java.net.URL. This was the main reason, why the enduser of the HTMLManager application had to deal with URL like syntaxtes like "file://yourFileOrDir" or "jar:file://yourWarFile.war!/"

Starting with Tomcat 5.5.0 o.a.c.Deployer interface got dropped from Tomcat and the ManagerServlet got migrated to do the deployment with the JMX mBeanServer from now on. If you look in the code starting from 5.5.0 the ManagerServlet.deploy(.....) got worse and worse and less and less maintainable because the migration hasn't done properly. For example:

    * Javadoc hasn't been migrated
    * URL support has been dropped (only a 'file:' prefix is supported from now on which isn't really need but this is a different topic) which was a good thing because the user from now on can use plain OS File paths like you would expect from an enduser point of view from the very beginning.
    * deployment of local war without context path has been dropped
    * deployment of local dir without context path has been dropped

There should be short term and a long term solution.

Short term solution should be 
    1.) to apply the fix I already did for this bug mentioned in my comment above, 
    2.) remove a the pseudo URLs like 'file:' we already have in the code and provide the user a consistent interface for native file- and path-names. This will effect the Ant DeployTask as well. 
    3.) The 'jar:file:' and 'file:' syntax needs to be removed/changed in the related javadocs and howtos.

Long term solution would be to create a new bug to go over the whole manager application and bring it back in to an consistant and maintainable state. I will create that bug as soon this one went into svn.

Kind Regards
Uwe Günther
Comment 6 Mark Thomas 2009-09-27 01:56:30 UTC
Thanks for the research. It helps enormously to be able to read through your summary and understand the background to this bug and how we got here.

I think path needs to be removed completely. The only place it is valid to use it is in server.xml and the manager app doesn't touch server.xml.

The leaves 4 cases:
1. config(no) war(no)
- invalid since nothing to deploy

2. config(yes) war(no)
- valid - copy appname.xml to $CATALINA_BASE/conf/<engine>/<host>

3. config(no) war(yes)
- valid - copy appname.war to Host's appBase

4. config(yes) war(yes)
- valid if appname is same for appname.xml and appname.war
- copy appname.xml to $CATALINA_BASE/conf/<engine>/<host>
- copy appname.war to Host's appBase

Any provide path can be ignored but I'd suggest logging a warning.

It is makes the fix easier, I'd be happy to see all the file URL support removed as well.
Comment 7 Uwe Günther 2009-09-27 03:44:40 UTC
Thanks for your input but this leads me to some further questions to discuss :-)

The 'path' parameter describes the context path of the web application. By providing a 'path' and a 'war' parameter you can tell the ManagerServlet for example ?path=/myWebapp&war=C:\\myWar.war. This adds the value to not having a hard wired relationship between the context name (aka '&path=/myWebapp') and the deployment source (webapp as *.war file or as an webapp conform directory structure aka '&war=C:\\myWar.war' or '&war=C:\\myDir')

Further more there is the case where you can provide a context.xml as config, but you have to know to which webapp this config belongs to.
A case would be 


In this case the context.xml would be end up in ${catalina.base}/conf/[Engine]/[Host]/myWebapp.xml

Basically you can say the '&path=' is used to get the Context (aka webapp) from the mBeanServer. If the '&path=' gets dropped you would need it inside the deploy(.....) method anyway but with the limitation the '&path=' would be always hardwired to the name of the '&war=' value (war or dir name) or the '&config=' value (config name

Ok this derivation of the '&path=' from '&war=' is done at the moment anyway if no '&path=' is provided, but you still have the possibility to override that, say you want to deploy the same war as 5 different webapps for testing purposes for example.

It works like this at the moment in the patch:

		if (!hasPath && !hasConfig && hasWar) { 
			this.war = new War(war);
			this.path = this.war.getPath();
			this.config = null;
		} else if (!hasPath && hasConfig && hasWar) { 
			//war, config
			this.war = new War(war);
			this.path = this.war.getPath();
			this.config = new Config(config);
		} else if (hasPath && !hasConfig && hasWar) {
			//war, path
			this.war = new War(war);
			this.path = new Path(path);
			this.config = null;
		} else if (hasPath && hasConfig && !hasWar) {
			//config, path
			this.war = null;
			this.path = new Path(path);
			this.config = new Config(config);
		} else if (hasPath && hasConfig && hasWar) {
			//war, config, path
			this.war = new War(war);
			this.path = new Path(path);
			this.config = new Config(config);
		} else {
			//!hasPath && !hasConfig && !hasWar
			//!hasPath &&  hasConfig && !hasWar
			// hasPath && !hasConfig && !hasWar
			throw new IllegalArgumentException(
					"Need one of the following combinations" +
	    			": war " +
	    			"| war, path " +
	    			"| war, config " +
	    			"| war, config, path " +
	    			"| config, path");

We could think about to derive the '&path=' (aka webapp context) from the '&config=' (config name) for the following case:

	//!hasPath &&  hasConfig && !hasWar

for example: 


would be end up in:


Thats what you suggested as well in comment #6 case 2.

BTW: In the pre JMX version of ManagerServlet (all pre Tomcat 5.5.0 versions) we had o.a.c.Deployer to encapsulate all the deployment related tasks. Starting with Tomcat 5.5.0 all these tasks got moved as 'JMX Agent Level like' access methods into the Manager Servlet which is a bit ugly from an OO point of view. Instead of  that solution we should think about having a 'JMX Agent Level like' Deployer object in the 'manager' webapp to provide encapsulated services for the tasks to perform an deploy, undeploy, ... from within the ManagerServlet. But thats possibly beautifying and should go with an different Bug.

Kind Regards
Uwe Günther
Comment 8 Mark Thomas 2009-09-27 09:11:53 UTC
If path is kept then I'd suggest something along the lines of treating it as an optional parameter that is used to rename the war/dir/context.xml file.
Comment 9 Uwe Günther 2009-09-27 09:22:31 UTC
That sounds perfect to me. I will check all the related interfaces to the ManagerServlet like the Ant DeployTask to make that clear and of course the html howto.

BTW: How would you deal with the localized properties if something new comes along? Is it ok to just enhance the non English property files with the new property together with its English value or is there any other procedure?

Kind Regards
Uwe Günther
Comment 10 Mark Thomas 2009-09-27 09:26:24 UTC
If you set the English property value then that automatically gets used if the non-English value isn't provided. You can leave the non-English files as is. Of course, if your language skills are better than mine and you can provide any of the translations that that would be great - but don't worry if you can't.
Comment 11 Uwe Günther 2009-10-04 07:22:57 UTC
I need a bit of an advice here from the commiters here. Mark maybe you could give me some infos. But let me first ask the question:

If have fixed and tested the deploy method in the ManagerServlet and I am in the process changing the javadoc for the ManagerServlet to what it does.
But I think this whole bug/change starts to mushroom in to something bigger the deeper I dig in the code because there are alot of inconsistencies. I just want to list here some of them which needs to be cleaned up in that area:

1.) Go over the Ant DeployTask and remove some deprecated Tasks and make the DeployTask more consistent (&localWar &war mess). There are also some invariants of the params which would cause odd behavior there.
2.) Go over the manager/text/<commands> and remove some deprecated tasks and make it more consistent compared with its subclass HTMLManagerServlet. There are odd things as well, I don't want to list them all here in detail.
3.) Go over the whole manager-howto.xml and describe the behavior we have really at the moment. This is a task where I would need someone whose mother tounge is english to double check the stuff I plan to rewrite.

I don't want to do that all in this Bug number, because I think the changes I did to ManagerServlet#deploy(.....) and the HTMLManagerServlet already are not that small at all and I do want to get them in first.

Possibly I should create 3 new Bugs for 1.) - 3.) and should put all the stuff I fixed in to a patch and attach it to this one so you guys can commit it.

What do you mean?

BTW: I have done all in the trunk (Tomcat 7). How do you bring this usually back into Tomcat 6.0.x or 5.5.x ?

Cheers Uwe
Comment 12 Uwe Günther 2009-10-06 05:32:06 UTC
I am still working on this. Code changes are done and tested for Tomcat trunk (7.0.0) and Tomcat 6.0.x trunk. I am in the process of doing the doc changes in html-manager-howto.xml and manager-howto.xml in addition to the code changes.

Kind Regards
Uwe Günther
Comment 13 Mark Thomas 2009-12-01 15:04:35 UTC
To answer your questions, create separate bugs to do the clean-up and attach the patches there.

How close are you to having the patch ready for this issue?
Comment 14 Mark Thomas 2009-12-11 05:31:28 UTC
*** Bug 47544 has been marked as a duplicate of this bug. ***
Comment 15 Konstantin Kolinko 2009-12-22 19:20:31 UTC
Lower importance, classify this as an enhancement, set Tomcat version as 6.0.20.

In essence, this is a documentation issue, and we can fix it as such,
but enhancements for the ManagerServlet are proposed as well.
Comment 16 Yohan Yudanara 2010-06-10 07:34:16 UTC
Created attachment 25576 [details]
Generated Excel File when TestPoiPrintArea.java is run with poi-3.7-SNAPSHOT-20100610.jar
Comment 17 Yohan Yudanara 2010-06-10 07:35:50 UTC
Sorry, I'm sending attachment to wrong bug...
It should be for bug #46664
Comment 18 Mark Thomas 2011-08-29 14:21:38 UTC
*** Bug 51727 has been marked as a duplicate of this bug. ***
Comment 19 Mark Thomas 2011-08-29 15:16:48 UTC
*** Bug 51727 has been marked as a duplicate of this bug. ***
Comment 20 Mark Thomas 2017-04-05 13:44:24 UTC
The content of attachment 25576 [details] has been deleted for the following reason:

Attached to wrong bug
Comment 21 Mark Thomas 2017-05-31 19:10:16 UTC
Since there are behavioural changes at a minimum, and probably changes to protected methods in the Manager and HTMLManager, I think this is one for 9.0.x only.

My general idea is to try and reduce the number of deploy methods and duplicate (ish) code but no firm plans as yet.
Comment 22 Mark Thomas 2018-02-28 16:13:32 UTC
The clean-up has been completed over the years. There may still be a few opportunities to reduce duplication.

The main remaining part was to derive the path name if it was not explicitly defined. This has now been implemented in 9.0.x for 9.0.6 onwards.