Bug 41538

Summary: Unable to run Tomcat as a Windows service under JDK 1.6
Product: Tomcat 6 Reporter: Fabio Grassi <fabio.grassi>
Component: CatalinaAssignee: Tomcat Developers Mailing List <dev>
Severity: blocker CC: andrewolter89, bdziedzic, daniel.armbrust.list, donnygat, gp, jeff.muus, johnvdenley, mario.escobar, mzko
Priority: P2    
Version: 6.0.0   
Target Milestone: default   
Hardware: Other   
OS: Windows Server 2003   

Description Fabio Grassi 2007-02-05 03:08:27 UTC
Under jdk1.6.0 and on some machines only, I'm not able to run tomcat as a
(Windows) service. When I try to start the service (installation goes all right)
I get the following error in jakarta_service:

[2007-02-05 10:45:06] [info] Debugging Service...
[2007-02-05 10:45:06] [info] Starting service...
[2007-02-05 10:45:06] [173  javajni.c] [error] Impossibile trovare il modulo
[2007-02-05 10:45:06] [924  prunsrv.c] [error] Failed creating java
[2007-02-05 10:45:06] [1179 prunsrv.c] [error] ServiceStart returned 1
[2007-02-05 10:45:06] [info] Debug service finished.
[2007-02-05 10:45:06] [info] Procrun finished.

If I start Tomcat with start.bat everything goes well. If I use jdk1.5
everything goes well.

Thanks in advance
Comment 1 Mark Thomas 2007-02-06 17:43:26 UTC
This works for me on XP Home SP2.

That it fails for you on some machines and works on others points to a
configuration issue with those machines (maybe the Java 6 install?) rather than
a Tomcat issue or an issue with the service wrapper.

The tomcat users list is the best place to get further help with this issue.
Comment 2 Fabio Grassi 2007-02-08 00:29:02 UTC
If you Google a little or browse tomcat-user mailing list you'll discover that
I'm not the only one to experience this very same problem. It is for sure a
configuration-related issue, but the fact that it involves only Tomcat and only
on jdk 1.6 (with jdk 1.5 it works fine on the same machine) and that at least a
few people who are long-time Tomcat users have not been able so far to fix it,
should suggest to keep the bug open and investigate further. Bear in mind that
not many people have started using jdk 1.6, and those who are may be using
Tomcat 6, so it may end up that many more cases show up in the next weeks.

Ciao, FG.
Comment 3 Christophe Pierret 2007-02-08 03:53:08 UTC
I had the exact same issue on a W2K3 server running 5.5.20 (due to a tester
installing JDK1.6 by mistake instead of JDK1.5 since it is now the default JDK
choice on Sun web site).
I was on a urge for a "CEO" demo, and did not investigate much (and went back to
JDK 1.5), but it seems to be related to configuration of tomcat windows service
using jvm.dll and some changes in JDK1.6 related to usage of jvm.dll from a
windows service (PATH related ?)
So the problem is very likely tomcat windows service installer when using jdk1.6.
I guess that if one does the following steps, he can easily reproduce the issue:
- install a clean W2K3 server OS : no previous JDK or tomcat installation or
environment variable changes like PATH or JAVA_HOME
- install JDK1.6 (and JRE 1.6 inside JDK setup)
- install tomcat 5.5.20 as a windows service
- start the tomcat service

I really think this is a "blocker" issue as it simply prevents running tomcat as
a service on some installations with current recommended JDK/JRE versions (1.6).
I should be kept open until tomcat service installation works with JDK1.6 on
clean servers or at least a documented work-around is found for those installations.
Comment 4 Stefan Negrea 2007-02-08 11:24:54 UTC
The tomcat service needs to be relinked to jvm.dll. This can be changed from the
service properties in the Services window. 

Comment 5 Mark Thomas 2007-02-08 19:17:45 UTC
I have tested this again, this time with a completely fresh install of the
 - Windows 2003 R2
 - JDK 1.6.0
 - Tomcat 5.5.20

The Tomcat service installs without issue and works as expected.

The varying reports of success from different users point to a configuration or
incomplete installation of Tomcat and/or the 1.6 JDK.

If someone can provide a series of tested installation steps for a clean system
that reproduces this problem then I will investigate further.
Comment 6 Christophe Pierret 2007-02-09 02:14:53 UTC
Some information on why this issue may occur on some systems...

The jakarta_service code uses a LOAD_WITH_ALTERED_SEARCH_PATH option when
loading jvm.dll (and all its dependent DLLs).
See the following link for details on what is done when the jvm.dll is loaded by
the jakarta_service code:
  Dynamic-Link Library Search Order, 

The behavior depends on:
- the OS version
- the registry value HKLM\System\CurrentControlSet\Control\Session
- the content of the directory specified for jvm.dll, the current directory, the
system directory, the 16-bit system directory, the Windows directory, the
directories that are listed in the PATH environment variable. 

There are many ways it can go wrong ...
So I suggest that anyone encountering this issue, use the Filemon tool from:
Launch it before starting the tomcat service, stopping capture after the service
start has failed then save the log and have a look at the log to guess which DLL
fails to load and if the search path used is correct.
I heavily suspect issues to come from DLL search path including bad versions of
some DLLs (and very likely JDK1.5 DLLs while trying to load JDK1.6 DLLs).
Comment 7 Fabio Grassi 2007-02-10 04:34:06 UTC
I investigated further using Filemon as cleverly suggested by Christophe and
found that Tomcat5.exe was unable to load msvcrt71.dll.

On that machine that particular dll was not present in any of the directories on
the search path. In fact it was present in JAVA_HOME\bin which is not added to
the path by default. I added JAVA_HOME\bin to Path env variable and now
everything works fine (although I had to reboot the server in order to have the
search path modified, merely modifying the Path env variable was not enough).

On the other machine where Tomcat service worked from the beginning with jdk1.6,
it turned out that msvcr71.dll had already been installed by software other than
the JDK in directories different from JAVA_HOME\bin and that one of those many
copies was the one to be loaded.

Whether this is a Tomcat bug, a JDK bug, a Windows bug or no bug at all I don't
 know, but I think other people will hit it particularly if msvcr71.dll is not
installed by default by at least some Windows versions, so I reopen this issue
in order for it to be further scrutinized.

Ciao, FG.
Comment 8 Mark Thomas 2007-02-10 09:20:44 UTC
Thanks for the additional information. A quick Google shows that this is a Java
bug. See http://forums.java.net/jive/thread.jspa?messageID=78158

The short-term fix is to copy msvcr71.dll from %JAVA_HOME%\bin to
%SYSTEM_ROOT%\system32 or any other location on the path.

Sorry I missed this in my previous test. I was using VMWare and installed the
tools so I could access the JDK install files on the local machine. The tools
installation installed the required dll in system32 so it was on the path -
hence the install wasn't quite as clean as I thought it was and everything worked.
Comment 9 Christophe Pierret 2007-02-11 00:57:18 UTC
The Java bug is here:

Workaround suggestion: 
Tomcat service installer could copy msvcr71.dll from %JAVA_HOME%\bin to
%SYSTEM_ROOT%\system32 when JDK version is 1.6 where JAVA_HOME is taken from
windows registry (not the env var as it may not exist).
The NSIS installer script could be modified to do that.

Comment 10 Mladen Turk 2007-02-11 09:16:19 UTC
The suggested workaround will not be implemented.
The solution is to properly install JDK, setup JAVA_HOME
and add the %JAVA_HOME%\bin to the System PATH.
Comment 11 Mark Thomas 2007-02-16 16:19:22 UTC
*** Bug 41639 has been marked as a duplicate of this bug. ***
Comment 12 Gili 2007-03-02 14:10:06 UTC
Mladen, I think you are mistaking two different bugs here.

The first issue is that in older builds of JDK6 could not find msvcr71.dll and
poped up a dialog box when you ran java from command-line. This has been fixed
in JDK6 final. See: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6372216

The second issue (totally unrelated to the first) is that Tomcat does not ship
msvcr71.dll as part of its installer even though Microsoft's deployment guide
clearly requires this:

I quote: "An application should use and redistribute msvcr71.dll, and it should
avoid placing a copy or using an existing copy of msvcr71.dll in the system
directory. Instead, the application should keep a copy of msvcr71.dll in its
application directory with the program executable."

Just because the JRE happens to redistribute msvcr71.dll itself does not mean
that Tomcat is not required to do the same. The point of an installer is to
unpack the software and fully configure it. By asking users to manually add
JAVA_HOME/bin to their PATH (where is this documented even?) you are completely
missing this point.

In my humble view you have two options:

1) Follow the Microsoft deployment guide and redistribute msvcr71.dll with
Tomcat. Every application I know does this.

2) Alternatively, if you insist on reusing msvcr71.dll shipped with the JRE,
Tomcat's installer should either copy it into the tomcat/bin directory or modify
tomcat5.exe to explicitly look for it in jre/bin.

The existing error message is extremely cryptic and very hard to figure out.
Please reconsider fixing this as soon as possible.
Comment 13 Marco Wayop 2007-03-16 09:38:07 UTC

In my case I had uninstalled JDK 1.5 and installed JDK 1.6

For me the Tomcat service would not start even after copying the msvcr71.dll in
the system folder and checking JAVA_HOME and PATH variable.

For me the solution was to change tomcat properties (Start->Programs->Apache
Tomcat->Configure Tomcat

And point the JVM and Classpath to the new JDK folders.

Hope this helps.

Marco Wayop

Comment 14 Mark Thomas 2007-08-16 17:05:14 UTC
*** Bug 43127 has been marked as a duplicate of this bug. ***
Comment 15 Mark Thomas 2007-08-29 04:45:13 UTC
*** Bug 43221 has been marked as a duplicate of this bug. ***
Comment 16 Jonathan Leech 2007-08-29 08:45:14 UTC
Java 6 breaks https anyway, by using only 768 bits in the Diffie-Hellman key
exchange, and thus insecure. 
Comment 17 Mark Thomas 2007-09-06 19:01:40 UTC
*** Bug 41868 has been marked as a duplicate of this bug. ***
Comment 18 Mark Thomas 2007-09-06 19:09:20 UTC
*** Bug 42915 has been marked as a duplicate of this bug. ***
Comment 19 Mark Thomas 2008-04-07 16:24:57 UTC
*** Bug 44764 has been marked as a duplicate of this bug. ***
Comment 20 Mark Thomas 2008-04-07 16:31:01 UTC
Just to summarise:

Short term fix options:
 - add %JAVA_HOME%\bin to the PATH
 - copy %JAVA_HOME%\bin\mscvr71.dll to a folder on the path

A quick check shows that it is jvm.dll that requires msvcr71.dll, not Tomcat or the service.

Re-reading the comments above, particularly comment #6, suggests that this is probably an issue with procrun, part of commons-daemon.

Commons uses JIRA. new issues can be created via:
Comment 21 John V Denley 2008-04-08 02:16:55 UTC
Mark Thomas stated:

"Short term fix options:
 - add %JAVA_HOME%\bin to the PATH
 - copy %JAVA_HOME%\bin\mscvr71.dll to a folder on the path"

surely if you add %JAVA_HOME%\bin to the PATH environment variable, then the mscvr71.dll is NOW in a folder on the path, so the second step is not necessary, or am i being really dim and missing something obvious?!
Comment 22 Mark Thomas 2008-04-08 09:20:30 UTC
Those were meant to be either or options.
Comment 23 Carlos 2008-04-10 14:59:01 UTC

In my case, I tried: 
*adding C:\j2sdk1.4.2_17\bin to the PATH environmental variable
*copying the mscvr71.dll file to C:\WINDOWS\system32 directory
*Adding an environmental variable with name JAVA_HOME whcih pointed to C:\j2sdk1.4.2_17\bin
         -After all of which I restarted the machine and/or re-installed Tomcat

Non of the above worked to get the Tomcat service started and I keep getting the same error reported in the logs...
Did I do something wrong? Did I miss something?

Comment 24 Ingolf Magnus 2008-04-21 02:21:55 UTC
It may be that the run-time error occurs in the commons-daemon codebase.

But the bug is in the Tomcat installer, not in the installed code.

According to OS (Windows) policy, each application requiring msvcrt71.dll should bring its own copy (see comment above), independently of other applications (e.g. the JDK).

If the Tomcat installer is written by the Tomcat team, this bug should not be RESOLVED:INVALID. I do not see how the commons-daemon team could fix Tomcat's problem here.

--Ingolf Magnus
Comment 25 Mark Thomas 2008-04-21 03:13:36 UTC
Based on the analysis so far, the issue appears to be related to how daemon loads jvm.dll. This makes is a daemon issue not a Tomcat one.
Comment 26 Mark Thomas 2008-04-21 14:05:01 UTC
To be clear it is jvm.dll that requires mscvr71.dll. This is a JVM component. It is neither a Tomcat nor a daemon component.

Looking a bit further it appears to be a JVM issue, but one that Sun don't view as an issue. http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6509291

There is a nice summary here:

The fix at the end looks like an approach that would work in daemon.
Comment 27 Dan Armbrust 2008-07-10 14:04:23 UTC
Ok, I really don't follow the logic here.

Most people seem to agree that this is a JVM bug.  <quick rant> And Sun has done what they do best, ignore the bug and pretend it isn't their fault - they closed the bug as invalid. </quick rant>

So, now we back up to tomcat - which is the app I'm trying to use - and it doesn't work.  But tomcat also closed the bug as invalid?  With a vague hope that the sun bug can be worked around by changes in the daemon project?

Where does the buck stop?  In a  brief look, I haven't found any bugs in the daemon project that seem to indicate that this was even passed on to them.

This bug should be reopened - and not closed until:

The Tomcat installer is modified to either ship its own copy of msvcr71.dll, and drop it beside the tomcat.exe file, or, the installer is modified to copy  the file from the jvm bin directory into the tomcat bin directory.


A workaround is implemented in the daemon project, and tomcat starts shipping a version of daemon which includes the workaround.  

I really don't see how this should become my problem as a user of tomcat.  It sucks that tomcat or the daemon launcher has to work around sun bugs - but I don't see what other choice there is in situations like this. 
Comment 28 Mark Thomas 2008-07-10 14:20:15 UTC
If you must rant, go do it on the Sun bug tracker.

If you want to do something productive, create an enhancement request in JIRA for commons daemon to work-around the Sun bug. As I said in a previous comment, all the pointers are there for a competent C coder to fix this. For the record, my C code sucks.

If you want to do something *really* helpful, write a patch for commons-daemon that  implements the necessary enhancement. If your C is as bad as mine, find someone who can write C and get them to write the patch.
Comment 29 William A. Rowe Jr. 2008-07-10 14:49:37 UTC
As Mark points out, this is a JAVA bug.  It's contingent on the Sun Java installer
to provide msvcr71.

This is *NOT* a system library (msvcrt.dll is).  In fact MS expects everyone who
consumes msvc*71 to install them privately, which is why they are flagged redist.

Please, don't waste your time and ours by reopening.
Comment 30 Dan Armbrust 2008-07-11 06:29:40 UTC
(In reply to comment #29)
> As Mark points out, this is a JAVA bug.  It's contingent on the Sun Java
> installer
> to provide msvcr71.


You are missing the important detail that sun DOES provide msvcr71.  So, in their mind, they have done nothing wrong.

Tomcat is the provider of tomcat.exe - so Tomcat is the app provider which has not provided the msvcr71 according to Sun's flawed logic. 

Now, tomcat shouldn't have to provide this library, because (if I understand the issue correctly) if Sun hadn't messed up, then the jvm.dll would be able to find the msvcr71 which sun provides.  But, Sun messed up.  So jvm.dll can't find the msvcr71 which sun provides when you call the jvm.dll from a different folder - the folder where tomcat.exe is located in this case.

> Please, don't waste your time and ours by reopening.

And that comment sounds an awful lot like Sun's response.  More buck passing.  Seems pretty clear to me that if tomcat won't start as a service, and tomcat is providing the service code, then we have a bug in tomcat.  It doesn't matter to me as a user of tomcat where the source of the problem is - this bug should still be open for tracking until the source of the problem is fixed, or a workaround is put into the tomcat installer - either via the installer, or by using a new version of commons-daemon which has a workaround.  

With the bug open, maybe then this bug would attract a developer who knows c well enough to implement a workaround in commons-daemon to actually fix this problem.  Unfortunately, I'm pretty sure I know even less c than Mark, so that developer won't be me.

I have opened a bug in commons-daemon:

And, I'll risk wasting your time by reopening this bug - because I strongly feel that this bug shouldn't be closed until the DAEMON bug is resolved, if that is the path that is taken.  At worst, maybe the churn of reopening and closing this bug will attract another developer who has the skills necessary to actually fix this issue for the rest of the tomcat users.
Comment 31 Mladen Turk 2008-07-11 07:14:39 UTC
The solution is to add %JAVA_HOME%\jre\bin to the system PATH environment variable reboot and then start the service.

I'm closing this issues as WONTFIX, but I left the DEAMON-110 issue opened.
See how it will be solvable for XP SP1+ windows versions, so it wont be fixed
for every OS we support.
There is a hack to try to preload msvcrt71.dll directly, but this might not
work because of additional dependencies. So, the only solution is to set
the path from which dependent dll's will be loaded, either via system PATH or
use version having the SetDllPath kernel32 function.
Comment 32 Mario 2008-08-16 07:39:09 UTC
I find a temporary solution for the problem with JVM_1.6 (May be for You that is obviously)

Copy  msvcr71.dll (find it on ...\Java\jre6\bin or Java\jre6\bin\new_plugin) to 
Java\jre6\bin\client or where your JVM is located, try to start your server again

Good Luck!!!
Comment 33 Mark Thomas 2008-10-14 09:55:52 UTC
*** Bug 45999 has been marked as a duplicate of this bug. ***
Comment 34 Konstantin Kolinko 2009-07-07 04:31:23 UTC
procrun 2.0.5 has a fix/workaround for this issue

It will be in TC 5.5.28, TC 6.0.21.