|Summary:||tomcat chooses wrong host if using mod_jk|
|Product:||Tomcat Connectors||Reporter:||Sven <sven.koehler>|
|Component:||Common||Assignee:||Tomcat Developers Mailing List <dev>|
testcase for the problem i described
patch for this bug
Description Sven 2005-08-12 05:08:52 UTC
Hi, my config is as follows: tomcat 5.5.9 knows two hosts, let's say the host "host1" and the host "host2". http://host1/ is basically a jsp-page echoing the string host1 - http://host2/ echos the string host2. apache 2.0.54 is configured with two vhost with ServerName host1 and host2. Both servers have a ServerAlias too - the one has ServerAlias alias1, the other has ServerAlias alias2 mod-jk 1.2.14 only knows one worker. My worker.properties looks like this: worker.list=worker1 worker.worker1.type=ajp13 worker.worker1.host=127.0.0.1 A very simple config. Both apache vhosts contain a "JkMount *.jsp worker1" Apache's UseCanonialName is on and Tomcats AJP-connector has useIPVHosts turned on. Therefor tomcat will chose the host by the server-name that mod_jk sends. So calling http://alias1/ results in a page containing the string host1, and calling http://alias2/ results in a page containing the string host2. The problem: Sometimes (not always and very selden) http://alias1 returns the string host2 and http://alias2 returns the string host1. I'm not sure, where the problem is, but i'm almost sure it's mod_jk - although i don't know, what should go wrong. I now defined a second worker with the same address and each of apache's vhosts use a different worker. The error didn't come up yet.
Comment 1 patrick ong 2005-12-25 18:23:29 UTC
Dont know whether related or not. Migrated some old 4.0 serlvet to 4.1 with mod_jk - 1.2.0 After I upgraded mod_jk to 1.2.15 my tomcat-4.1.31 start to point to wrong host. (the serlvet binaries are copied from another host, my point is Previous version did not give me problem) email@example.com
Comment 2 Sven 2005-12-25 19:13:55 UTC
So the problem still exists with mod_jk 1.2.15 and Tomcat 5.5.12. If two VirtualHosts use the same worker, Tomcat receives the wrong ServerName. IMHO, it has something todo with some cache or some other flaw. (The caches are worker-wise, aren't they?) Does somebody care about this? @Patrick: i don't know if your problem is related. For me, the problem is reproducable, but it does not go wrong always: first HTTP-request works, second doesn't, third works again, some random ...
Comment 3 Yoav Shapira 2006-01-10 20:12:50 UTC
Patrick, I don't think your problem is related. Try firstname.lastname@example.org for help. Sven, I can't reproduce your problem. If you're still experiencing it and can post the configuration files for Apache, Tomcat, mod_jk, please do.
Comment 4 Sven 2006-01-10 20:29:35 UTC
I'll do my best, to reproduce it with a small testcase.
Comment 5 Sven 2006-01-12 20:58:04 UTC
Created attachment 17404 [details] testcase for the problem i described There are two hosts in tomcat: host1 and host2 They point to /tmp/host1 and /tmp/host2 (which is also in the tgz) There are two hosts in apache: host1/alias1 and host2/alias2 There are two workers defined, but only one is in use from both virtualhosts Put the line "127.0.0.1 host1 host2 alias1 alias2" in /etc/hosts If you run "./test.sh alias1" and "./test.sh alias2" in two terminals you should see only "host1" in one window, and "host2" in the other window. What i experience is, that "./test.sh alias2" output "host1" sometimes - or "./test.sh alias1" outputs "host2". When i change apache's virtualhosts to use two different workers, the problem is gone.
Comment 6 Sven 2006-01-12 21:43:42 UTC
you can also start "./test.sh host1" and "./test.sh host2" with the same strange result: "./test.sh host1" sometimes outputs "host2", and "./test.sh host2" sometimes outputs "host1"
Comment 7 Dan Carwin 2006-01-17 19:24:16 UTC
Sven, Both workers on the same port? Why not match the "host" name in workers.properties to the "host" container name in server.xml? Why point both at 127.0.0.1? Seems the proper config is to use names for the workers' host values instead of the same (port and) ip addy in both cases. -Dan
Comment 8 Sven 2006-01-18 01:15:40 UTC
@Dan: i want to use only one worker - and the config attached to this bug only uses one worker - though two are declared. Using two workers that point to the same tomcat is a workaround for me, so that the bug described here doesn't rise. I don't understand, what you mean with "using names" in the worker-file. That doesn't make sense to me since the proper DNS-name would be "localhost" in both cases. The AJP-Connector only listenes on localhost. The names used in the worker-file are _not_ sent to Tomcat by mod_jk, AFAIK.
Comment 9 Sven 2006-01-18 17:51:38 UTC
The comment by Dan sounded much like "your config cannot work at all". So i think i need to explain a little more: The config does work, if you: - restart apache, then run only "./test.sh alias1": you will only see host1 on the console - restart apache, then run only "./test.sh alias2": you will only see host2 on the console - use worker1 for the one virtualhost, and use worker2 for the other, then run "./test.sh alias1" and "./test.sh alias2" in parallel: both scripts return the right results. I hope that you agree with me, that tomcat chooses the hosts like it should. Note, that the server.xml says useIPVHosts="true" - so tomcat does AFAIK not look at the host-header, but at some other information that mod_jk sends. Since i also use "UseCanonicalName On" for apache, mod_jk is supposed to send the ServerName declared for the virtualhost. The config does not work, if you use the same worker for both virtualhosts - which is "the right thing" IMHO since there is also only one tomcat running. What do i mean, with "it does not work" - well: Start "./test.sh alias1" and "./test.sh alias2" in parallel. If you're lucky, it works for 10 requests and you see what you should: "host1" returned by the first command, and "host2" returned by the second command. The result changes for me, and the at least one command "goes crazy" and returns something else: either the command only returns host1 instead of host2 (or vice versa of course) or sometimes i even get an alternating sequence of host1 and host2 by one of the commands - or even by both. Especially the alternating output is a case, that should not occur. The choice made by tomcat is deterministic AFAIK and tomcat is only considering the data that it gets by mod_jk, so how can there be an alternating sequence? My conclusion is, that mod_jk sends the wrong servername, or perhaps sending the servername is optional and tomcat just "reuses" the information it has been given by a former AJP-request on the same connection - but i have no clue about those internal things.
Comment 10 Mladen Turk 2006-03-17 09:25:47 UTC
I was not able to reproduce the problem. With your test files and mod_jk from SVN it works OK.
Comment 11 Sven 2007-03-02 20:13:11 UTC
I found the bug. It's still present in Tomcat 6.0.10. I will supply details soon, as well as a patch.
Comment 12 Sven 2007-03-02 21:59:03 UTC
So the bug is in the class org.apache.tomcat.util.buf.MessageBytes. The method setBytes doesn't clear the char-buffer. And the method setChars doesn't clear the byte-buffer. If useIPVHosts is enabled, then the parameter "host" of the org.apache.tomcat.util.http.mapper.map()-method is a MessageBytes-Object of the type T_BYTES. The method MessageBytes.toChars()-method is called. But this method exists, if the charbuffer is already filled. And in deed, the char-buffer is already filled. While the byte-buffer contains the value "host1", the char-buffer may still contain the value "host2" because the MessageBytes objects are reused. And instead of converting the byte-buffer-value "host1" to chars, the old value "host2" is used. This resulted in the bug. So the following patch fixes the problem. With your permission, i would like to clean up the class MessageBytes. It really needs it - IMHO.
Comment 13 Sven 2007-03-02 22:00:42 UTC
Created attachment 19659 [details] patch for this bug The patch should apply cleanly to Tomcat 6.0.10
Comment 14 Sven 2007-03-03 00:40:45 UTC
I discovered, that calling byteC.recycle() also resets the Encoding that might have been set. But actually the setEncoding() method is used nowhere. And calling byteC.reset() doesn't work, because the reset()-method leaves the ByteChunk in an illegal state: the array-pointer is set to null, but the internal "isSet" which controlls the result of isNull() is not reset to false. I'm quite unhappy with the class MessageBytes. For example setInt/setLong also forget to clear the char-buffer. toBytes() ignores the charset. In many cases i discovered strange handeling of non-ASCII characters. So i don't aim to clean it up. There are too many side-effects that might be hurt by any modification.
Comment 15 Remy Maucherat 2007-03-03 02:00:04 UTC
-1 for the patch. If the problem exists (I really doubt it), a better solution will have to be found.
Comment 16 Sven 2007-03-03 06:25:02 UTC
(In reply to comment #15) > -1 for the patch. If the problem exists (I really doubt it), a better solution > will have to be found. Simply one question: Why? setBytes() invalidates everything - except the value within charC is kept. Anyway: there might be one missing recycle call somewhere. I'm already looking for it.
Comment 17 Sven 2007-03-03 06:57:31 UTC
So in org.apache.coyote.Request.recylce(), this.localNameMB.recycle() is not called. If i add that call, the problem is gone too (my first patch has been reverted before testing). The problem is: remoteAddrMB.recycle(), remoteHostMB.recycle(), localAddrMB.recycle() is also not called. And i have no clue why not or is it should actually be called. Anyway: At least localNameMB gets reused without being recycled. So the internal charC contains an old value. setBytes() doesn't clear internal the charC-cache. toChars() reuses the old value. Hence the bug. I admit, that the semantics of the MessageBytes class are a ridle to me. I don't know, why setBytes() shouldn't invalidate the charC cache (even though it invalidates all other cached values). Somebody please explain the semantics of this class to me! And decide, whether setBytes() should clear charC, or not.
Comment 18 Sven 2007-03-03 07:04:53 UTC
Additional information: the value of localNameMB is set in AjpProcessor.prepareRequest() (line 645) by calling "requestHandlerMessage.getBytes(request.localName())". This getBytes method calls setBytes() of localNameMB. Above that line, remoteAddrMB and remoteHostMB are also set with the same method. And they also might not have been recycled. I don't know for sure.
Comment 19 Remy Maucherat 2007-03-03 14:45:08 UTC
(In reply to comment #17) > And decide, whether setBytes() should clear charC, or not. It seems you're eager to contribute and stuff, and I think that's good, but if I said -1 to your patch, this means the answer to this question is "no". This will cause problems, and is also inefficient. I think this is a special case with AJP, where recycle should possibly be called on the localName MessageBytes before reading the localName field (it should be fairly explicit that the localName/Addr are otherwise never recycled in the Request object). This is indeed a special case with the IP vhost feature (which IMO is not a very good feature, but it's another story).
Comment 20 Sven 2007-03-03 15:51:41 UTC
Do you understand, why we need to recycle the localName? (short reason: two ajp-requests with different localName over same AJP connection) So i have the feeling, that you want this fix to be AJP specific. I will try to fix it in that way. I tried to fix AjpProcessor/AjpAprProcessor. But the fix only worked, when i did it in JkCoyoteHandler. I simply changed the code from request.recycle(); to request.recycle(); request.localName().recycle(); I don't feel well leaving the other 3 fields (remoteAddr, remoteHost, localAddr) unrecycled. Somehow i feel, like this issue also applies to the other fields. But i don't have enough knowledge to be sure. So i leave this up to you. Now i will attach a new patch. Maybe you like it :-)
Comment 22 william.barker 2007-05-20 15:35:47 UTC
I've committed a more general fix for the JK Connector (i.e. JkCoyoteHandler). I can port it to the APR and AJP Connectors if/when Remy and Mladen doen't veto it.
Comment 23 Sven 2007-05-20 18:30:57 UTC
> I've committed a more general fix for the JK Connector (i.e. > JkCoyoteHandler). I can port it to the APR and AJP Connectors if/when Remy > and Mladen doen't veto it. Thanks a lot! I took a look at your commits. I think, the JK Connector is fixed now. I will test it soon. (At the moment, i don't have much time).
Comment 24 Sven 2007-05-22 16:56:34 UTC
(In reply to comment #23) > I took a look at your commits. I think, the JK Connector is fixed now. I will > test it soon. (At the moment, i don't have much time). I ran my tests, and yes: it's fixed. Sweet!
Comment 25 Sven 2007-08-06 07:17:45 UTC
So William has made a start. His patch works, and could be ported to the other 2 connectors. But i think he's waiting for some reaction by Remy and Mladen. (see comment #22) Let's get this done, so this can be marked fixed.
Comment 26 Tim Whittington 2011-04-11 16:23:08 UTC
I think this is as fixed as it's ever going to get (and the connectors have moved on a lot since this was raised).