Start and stop are synchronous. Calling start() then stop() should be sufficient. Doing that with the current startup scripts is a little trickier because you need to wait for the shutdown port to open.

Something like:
./catalina.sh start && sleep 1 && ./catalina.sh stop

should work however long the web application takes to start up.

(In reply to Mark Thomas from comment #1)
> Start and stop are synchronous. Calling start() then stop() should be
> sufficient. Doing that with the current startup scripts is a little trickier
> because you need to wait for the shutdown port to open.
> 
> Something like:
> ./catalina.sh start && sleep 1 && ./catalina.sh stop
> 
> should work however long the web application takes to start up.

On my laptop the servlet takes about 3 seconds to initialize (though on some cloud deployments I have seen it take as much as 15s).

From testing the above compounded command it seems to me that the shutdown port only opens after the servlet initialization, so waiting 1 or 2s does not work [1].

If we could start listening the shutdown port at the beginning of the process, and then it can be a blocking operation until start completes, then that would be a great a solution.

Thoughts?

[1] /workspace/src/tomcat-master$ output/build/bin/catalina.sh start && sleep 2 && output/build/bin/catalina.sh stop
Using CATALINA_BASE:   /workspace/test/catalina-base
Using CATALINA_HOME:   /workspace/src/tomcat-master/output/build
Using CATALINA_TMPDIR: /workspace/test/catalina-base/temp
Using JRE_HOME:        /opt/java/jdk1.8.0_202
Using CLASSPATH:       /workspace/src/tomcat-master/output/build/bin/bootstrap.jar:/workspace/src/tomcat-master/output/build/bin/tomcat-juli.jar
Tomcat started.
Using CATALINA_BASE:   /workspace/test/catalina-base
Using CATALINA_HOME:   /workspace/src/tomcat-master/output/build
Using CATALINA_TMPDIR: /workspace/test/catalina-base/temp
Using JRE_HOME:        /opt/java/jdk1.8.0_202
Using CLASSPATH:       /workspace/src/tomcat-master/output/build/bin/bootstrap.jar:/workspace/src/tomcat-master/output/build/bin/tomcat-juli.jar
Apr 29, 2019 10:58:34 AM org.apache.catalina.startup.Catalina stopServer
SEVERE: Could not contact [localhost:8005] (base port [8005] and offset [0]). Tomcat may not be running.
Apr 29, 2019 10:58:34 AM org.apache.catalina.startup.Catalina stopServer
SEVERE: Error stopping Catalina
java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at java.net.Socket.<init>(Socket.java:434)
	at java.net.Socket.<init>(Socket.java:211)
	at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:513)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:390)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:480)

I'm definitely missing something, here. Launching Tomcat and then shutting it down saves zero time on the next launch except maybe to create a directory or two and unpack any WAR archives being deployed.

If you want to warm the application, you'll want to pre-compile all the JSPs either using jspc or by just spamming the application with web requests.

So what exactly is the point, here?

I can see use-cases for building something like this (some people want to know when Tomcat is "really started"), but for triggering a shutdown to "warm" an app server? I don't get it.

(In reply to Christopher Schultz from comment #3)
> I'm definitely missing something, here. Launching Tomcat and then shutting
> it down saves zero time on the next launch except maybe to create a
> directory or two and unpack any WAR archives being deployed.

This is not for creating the Tomcat directory structure, but all of the initialization of the servlet itself.
 
> If you want to warm the application, you'll want to pre-compile all the JSPs
> either using jspc or by just spamming the application with web requests.
> 
> So what exactly is the point, here?

This is not a JSP per-se, even though it follows the JSP interface.  In this case it is the Lucee Application Server that implements a CFML engine.

> I can see use-cases for building something like this (some people want to
> know when Tomcat is "really started"), but for triggering a shutdown to
> "warm" an app server? I don't get it.

The Application Server can take a while on first run because according to configuration settings it might download extensions and install them, etc.  Without downloading extensions it takes about 3.5s to launch on my latptop.  With extensions it can take up to 60s.

In monolith deployments that might have been acceptable, but in Docker or server-less deployments it is not.

You can find more information in the Lucee ticket system [1], where I actually resolved the issue for Lucee but it feels like a hack and would be much better if resolved on the Tomcat level as 1) it will be cleaner, and 2) it will help other projects who might need this feature.


[1] https://luceeserver.atlassian.net/browse/LDEV-1196

(In reply to Igal Sapir from comment #4)
> (In reply to Christopher Schultz from comment #3)
> > I'm definitely missing something, here. Launching Tomcat and then shutting
> > it down saves zero time on the next launch except maybe to create a
> > directory or two and unpack any WAR archives being deployed.
> 
> This is not for creating the Tomcat directory structure, but all of the
> initialization of the servlet itself.

So if the servlet does something like downloading a bunch of files or whatever? I mean, the JVM stops, so reading db info into a cache or something doesn't help. Just making sure that this has nothing to do with Tomcat itself, which won't benefit at all.

> > If you want to warm the application, you'll want to pre-compile all the JSPs
> > either using jspc or by just spamming the application with web requests.
> > 
> > So what exactly is the point, here?
> 
> This is not a JSP per-se, even though it follows the JSP interface.  In this
> case it is the Lucee Application Server that implements a CFML engine.

Gotcha. I'm curious what the Lucee engine does on startup that happens ONE TIME instead of each time the application is launched.

> > I can see use-cases for building something like this (some people want to
> > know when Tomcat is "really started"), but for triggering a shutdown to
> > "warm" an app server? I don't get it.
> 
> The Application Server can take a while on first run because according to
> configuration settings it might download extensions and install them, etc. 
> Without downloading extensions it takes about 3.5s to launch on my latptop. 
> With extensions it can take up to 60s.

So these are extensions for Lucee?

> In monolith deployments that might have been acceptable, but in Docker or
> server-less deployments it is not.

I have to admit that I find "serverless servlets" an amusing construct.

(In reply to Christopher Schultz from comment #5)
> (In reply to Igal Sapir from comment #4)
> > (In reply to Christopher Schultz from comment #3)
> > > I'm definitely missing something, here. Launching Tomcat and then shutting
> > > it down saves zero time on the next launch except maybe to create a
> > > directory or two and unpack any WAR archives being deployed.
> > 
> > This is not for creating the Tomcat directory structure, but all of the
> > initialization of the servlet itself.
> 
> So if the servlet does something like downloading a bunch of files or
> whatever? I mean, the JVM stops, so reading db info into a cache or
> something doesn't help. Just making sure that this has nothing to do with
> Tomcat itself, which won't benefit at all.

Right.  Not Tomcat itself.

> 
> > > If you want to warm the application, you'll want to pre-compile all the JSPs
> > > either using jspc or by just spamming the application with web requests.
> > > 
> > > So what exactly is the point, here?
> > 
> > This is not a JSP per-se, even though it follows the JSP interface.  In this
> > case it is the Lucee Application Server that implements a CFML engine.
> 
> Gotcha. I'm curious what the Lucee engine does on startup that happens ONE
> TIME instead of each time the application is launched.

In addition to Extensions (more below), some of the initial setup is to create directories and files for the application during initialization.  That only happens once so the first time Lucee is launched takes much longer than subsequent launches.  

This is exactly the issue that this enhancement proposes to fix.  Do the first launch during the `docker build` process so that `docker run` will start much faster.

> 
> > > I can see use-cases for building something like this (some people want to
> > > know when Tomcat is "really started"), but for triggering a shutdown to
> > > "warm" an app server? I don't get it.
> > 
> > The Application Server can take a while on first run because according to
> > configuration settings it might download extensions and install them, etc. 
> > Without downloading extensions it takes about 3.5s to launch on my latptop. 
> > With extensions it can take up to 60s.
> 
> So these are extensions for Lucee?

Yes.  Lucee has many different extensions, e.g. database drivers, cache providers, PDF generators, Chart generators, etc.  Some extensions are provided by the Lucee project [1] while others provided by 3rd party vendors.  Developers can configure which extensions their application requires to control the image size.

> 
> > In monolith deployments that might have been acceptable, but in Docker or
> > server-less deployments it is not.
> 
> I have to admit that I find "serverless servlets" an amusing construct.

;-)

[1] https://github.com/search?p=3&q=org%3Alucee+ext&type=Repositories

Thinking about this some more, I think the best way to handle this is with a new "warmup" command. Implemented in a similar manner to configtest it would call server.start() followed by server.stop().

I have a slight objection to the term "warmup" being used, here.

Can we call it something else?

Nothing brilliant comes to mind.

"updown"?
"startstop"?
"launchtest"?

Perhaps such a special launch command could provide some output such as "OK" if everything is okay, or some kind of error words/codes which describe what happened. For example, if one of the connectors failed to start properly, we could report that failure. Perhaps the same for any configured auto-deploy application that failed to start cleanly.

Since this is synchronous, we can use the exit code to signal success or failure.

(In reply to Igal Sapir from comment #0)
> Currently, one hack that many use is to launch Tomcat during the container
> build process, wait for an arbitrary length of time, e.g. 10s or 20s, and
> then shut Tomcat down.

From your question and some comments, if the problem is with the "arbitrary length of time", I suggest to rely on JMX particularly on the Connector stateName value. It is changing from INITIALIZED to STARTED once Tomcat has finished deploying. If you can bring ANT within your docker, the effort is not that huge, you just need 2 targets, one for monitoring the Connector state, and one for stopping Tomcat.

This is the result that I get with a servlet called at the startup and that sleeps 60 during its initialization :

catalina.out
09-Jul-2019 22:10:55.804 INFO [localhost-startStop-5] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/home/eadell/tests/apache-tomcat-8.5.15/webapps/host-manager] has finished in [27] ms
09-Jul-2019 22:10:55.879 FINE [localhost-startStop-4] org.apache.catalina.authenticator.AuthenticatorBase.startInternal No SingleSignOn Valve is present
09-Jul-2019 22:10:56.055 INFO [localhost-startStop-4] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/home/eadell/tests/apache-tomcat-8.5.15/webapps/examples] has finished in [793] ms
09-Jul-2019 22:11:55.779 INFO [localhost-startStop-3] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/home/eadell/tests/apache-tomcat-8.5.15/webapps/slowstart] has finished in [60,518] ms
09-Jul-2019 22:11:55.790 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8280"]
09-Jul-2019 22:11:55.811 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8281"]
09-Jul-2019 22:11:55.815 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 60719 ms
09-Jul-2019 22:11:57.411 INFO [main] org.apache.catalina.core.StandardServer.await A valid shutdown command was received via the shutdown port. Stopping the Server instance.
09-Jul-2019 22:11:57.412 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8280"]
09-Jul-2019 22:11:57.469 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8281"]
09-Jul-2019 22:11:57.521 INFO [main] org.apache.catalina.core.StandardService.stopInternal Stopping service [Catalina]
destroying LongInitServlet

shell output
$ time (bin/startup.sh ; ant -f anted-jmx/project.xml buildAndStop)
Using CATALINA_BASE:   /home/eadell/tests/apache-tomcat
Using CATALINA_HOME:   /home/eadell/tests/apache-tomcat
Using CATALINA_TMPDIR: /home/eadell/tests/apache-tomcat/temp
Using JRE_HOME:        /usr/local/java/1.9
Using CLASSPATH:       /home/eadell/tests/apache-tomcat/bin/bootstrap.jar:/home/eadell/tests/apache-tomcat/bin/tomcat-juli.jar
Tomcat started.
Buildfile: /home/eadell/tests/apache-tomcat-8.5.15/anted-jmx/project.xml

waitMBEAN:
     [echo] Server url alive

shutdown:
     [echo] shutting down ${tomcat.catalina_base}

buildAndStop:

BUILD SUCCESSFUL
Total time: 1 minute 3 seconds

real    1m4.673s
user    0m5.157s
sys     0m0.600s


Here are the ANT targets :

<target name="waitMBEAN">
  <jmxOpen host="${jmx.server.name}" port="${jmx.server.port}" username="controlRole" password="******" />
  <waitfor maxwait="1200" maxwaitunit="second" timeoutproperty="server.timeout" >
      <jmxCondition
        operation="=="
        name="Catalina:type=Connector,port=8281"
        attribute="stateName" value="STARTED"
      />
  </waitfor>
  <fail if="server.timeout" message="Server url don't answer inside 1200 sec" />
  <echo message="Tomcat finished starting" />
</target>

<target name="shutdown" description="stop tomcat">
    <echo message="shutting down ${tomcat.catalina_base}" />
    <exec executable="/bin/bash">
        <arg line="${catalina.home}/bin/shutdown.sh" />
    </exec>
</target>

<target name="buildAndStop" depends="waitMBEAN,shutdown" />

If required you could also add targets to call servlets, and stop once you have finished these calls.

Of course, remember to use a random JMX password then you would not compromise all of your Tomcats if something bad happened.