Bug 28003

After 2 days I've finally managed to get Tomcat 5.0.19/mod_jk2/JNI working with
Apache 2.0.48. This threw up a number of issues.

Firstly, the connector documentation ought to be clearly linked off the main
Tomcat page. It's currently hidden under documentation and linked from a couple
of other obscure places - since it's shipped as a separate module, and the
configuration is rather specific, it ought to be easier to find the main JK page
at http://jakarta.apache.org/tomcat/tomcat-4.1-doc/jk2/index.html. (And it would
be nice if that page was updated to reflect some of the new information in this
report.) Even once I'd found that page, getting back to it is too involved.

Second, although using the socket channel works fine in general, I couldn't get
the standalone mod_jk2 2.0.4 to work in JNI mode at all. I'm currently running a
mod_jk2 compiled from the Tomcat 5.0.19 source archive, and have got that to
work, but switching to the precompiled jk2 fails with exactly the same
configuration.

Third, often error messages are put to the Apache log of the form "[error]
lb.service() worker failed 120000 for ajp13:jni" or "[error] mod_jk.handler()
Error connecting to tomcat 120000". 120000 is the error value available at the
point in the code where those messages are logged. It is the value JK_ERR, which
is defined as APR_OS_START_USEERR. (My apr_errno.h says this symbol is
deprecated and ought to be APR_OS_START_USERERR instead.) This seems to be the
only error code produced internally, so is totally useless for figuring out what
went wrong. It would be much better if at each point in the code where JK_ERR
was introduced, a clear statement of *why* was logged out. I had to do much
chasing through the source to figure this out. Often the result is propagated
through several layers and by the time a log statement is encountered the error
could have come from anywhere for any reason.

Fourth, the jk2 documentation is under the Tomcat 4.1 docs and hasn't been
updated for the 5.0 world. It recommends the following worker.jni config:

# Define the parameters for the Java Virtual Machine
[vm:]
OPT=-Djava.class.path=${TOMCAT_HOME}/lib/tomcat-jni.jar;${TOMCAT_HOME}/lib/tomcat.jar
...
# JNI worker startup handler
[worker.jni:onStartup]
class=org/apache/jk/apr/TomcatStarter
ARG=start
stdout=${serverRoot}/logs/stdout.log
stderr=${serverRoot}/logs/stderr.log

# JNI worker shutdown handler
[worker.jni:onShutdown]
class=org/apache/jk/apr/TomcatStarter
ARG=stop

This no longer works with 5.0.19: the jar files have moved/changed, and start
and stop appear to be used to run a standalone Tomcat/Catalina instance, which
continues until terminated. However the JNI channel expects the start request to
return and marks itself "starting" until it does so. It only gets marked "done"
through AprImpl.jkSetAttribute() on succesful return from the Catalina startup
function, which never happens. As a result any requests fail with "lb.service()
worker failed 120000 for ajp13:jni". In fact the ARG parameters should now read
"startd" and "stopd" to run Catalina in the background.

Fifth, the whole option setting system is a little strange. The placement of
colons or dots in the option name seems haphazard, and I was often uncertain
whether I'd managed to direct a value to the right component, even if no error
had been produced. This was compounded by mod_jk2 logging JK_LOG_DEBUG_LEVEL
messages at APLOG_DEBUG to Apache. Meaning I can ramp up my JK logging level as
much as I like but won't see anything unless I also increase my Apache log
level. I think it would be better to log at APLOG_NOTICE, or make the output
level configurable, so there is only one point of control. Regardless, this
should be clearly documented. It's quite disheartening to be making changes and
having them apparently ignored with no indication why. I'm also currently using
JkSet to configure "TOMCAT_HOME" and "CATALINA_HOME", which appears to work (and
the values I specify get used as replacements for ${TOMCAT_HOME} in the
remaining config) but spews an initial error out. There must be a way to do this
cleanly without error.

My final working config is (all in httpd.conf, workers.properties is empty):

<IfModule mod_jk2.c>

  JkSet shm.info "Scoreboard. Requried for reconfiguration and status with
multiprocess servers."
  JkSet shm.file "D:/Program Files/Apache Group/Apache2/logs/jk2.shm"
  JkSet shm.size 1048576
  
  JkSet lb:lb.info "a load balancer named lb"

  JkSet TOMCAT_HOME "D:\PROGRA~1\APACHE~1\TOMCAT~1.0"
  JkSet CATALINA_HOME "D:\PROGRA~1\APACHE~1\TOMCAT~1.0"

  JkSet channel.jni:jni.info "The jni channel, used if tomcat is started inprocess"
    
  JkSet vm:.info "Parameters used to load a JVM in the server process"
  JkSet vm:.JVM "D:/Program Files/Sun/AppServer/jdk/jre/bin/client/jvm.dll"
  JkSet vm:.OPT "-Djava.endorsed.dirs=${TOMCAT_HOME}\common\endorsed"
  JkSet vm:.OPT "-Xmx128M"
  JkSet vm:.OPT "-Xrs"
  JkSet vm:.OPT
"-Djava.class.path=${TOMCAT_HOME};${TOMCAT_HOME}\server\lib\tomcat-jni.jar;${TOMCAT_HOME}\common\lib\jmx.jar;${TOMCAT_HOME}\server\lib\commons-modeler.jar;${TOMCAT_HOME}\bin\bootstrap.jar"
  JkSet vm:.OPT "-Dtomcat.home=${TOMCAT_HOME}"
  JkSet vm:.OPT "-Dcatalina.home=${TOMCAT_HOME}"
  
  JkSet worker.jni:onStartup.info "Command to be executed by the VM on startup.
This one will start tomcat."
  JkSet worker.jni:onStartup.class org/apache/jk/apr/TomcatStarter
  JkSet worker.jni:onStartup.ARG startd
  JkSet worker.jni:onStartup.stdout "${serverRoot}/logs/stdout.log"
  JkSet worker.jni:onStartup.stderr "${serverRoot}/logs/stderr.log"
  
  JkSet worker.jni:onShutdown.info "Command to be executed by the VM on
shutdown. This one will stop tomcat."
  JkSet worker.jni:onShutdown.class org/apache/jk/apr/TomcatStarter
  JkSet worker.jni:onShutdown.ARG stopd
  
  JkSet status:.info "Status worker, displays runtime information"
  
  <Location /jkstatus>
    JkUriSet group status:
  </Location>

  <Location /jsp-examples>
    JkUriSet group lb:lb
  </Location>
  <Location /servlets-examples>
    JkUriSet group lb:lb
  </Location>
</IfModule>

Other points in the above:

1) The classpath starts with an inital component which is never used. It seems
that the first component is interpreted as ${serverRoot}/<path>, causing the
first jar file to be ignored as an invalid path. Later components are
interpreted correctly, hence the dummy component to protect the later ones.

2) All the path values above are Win32 short filenames if the real component
contains a space. java.exe accepts paths containing spaces, as quoted strings,
on the command line, but apparently this doesn't work with mod_jk2 calling
jvm.dll. Using non-quoted short names (to eliminate spaces) does work though.
Even quoting the short names breaks. Under Windows, paths can contain spaces -
this is a fact of the platform, and being told (as various HOWTOs have done) to
rearrange you file system to appease software that can't deal with this is
really not appropriate. The software should be able to deal with it (java.exe
can, on the command-line, so I suspect the fault lies in mod_jk2 somewhere.) I'd
also point out that UNIX path names can contain spaces too - these need quoting
or escaping, but they can and do exist. Software that can't deal is therefore
broken cross-platform, not just failing to implement a Win32 oddity.

3) The OPT parameter is multi-valued, but when you look at the config under
/jkstatus/, only the last value is shown. Concatenating them all onto one line
"-D... -D... -X... -X... -D..." breaks mod_jk2 too. They ought to all be visible
(again, it's confusing. It made me think for a while that it was just ignoring
the earlier values completely.) Multiple, space separated values in the same OPT
value should also be supported.

4) The jar files listed are the minimal set required to get the Bootstrap class
loaded. Once that starts up it reconfigures the classpath to find all the rest
of the stuff needed. Until that point you need to pass at least that set of jar
files or you get NoClassDefFoundErrors. None of which are visible under mod_jk2
until the system has started up sufficiently to replace System.out and
System.err (I found them by running TomcatStarter under java.exe.) It might be
nice to find a way of getting tomcat-jni to do a similar thing to Bootstrap so
that only that one jar file need be specified. At least I didn't have to specify
every single jar file in Tomcat, which would have made the config just horrible.

5) I have 3 JREs installed. The default is a normal JRE. I want it to use the
JDK, hence the JVM paramater. Sun's latest J2EE JDK apparently doesn't register
itself in the same way as the JRE, but you still need a compiler to run JSP
pages, don't you?