Bug 62667

Summary: RewriteMap not working with %{HTTP_HOST}
Product: Tomcat 9 Reporter: ITASSIST <post>
Component: CatalinaAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED FIXED    
Severity: critical    
Priority: P2    
Version: 9.0.x   
Target Milestone: -----   
Hardware: PC   
OS: Linux   

Description ITASSIST 2018-08-31 08:49:57 UTC
When passing %{HTTP_HOST} as the key to a RewriteMap the lookup function only passes the path of the page «/test/tespage» and not the host «test.example.com»

The URL sent to Tomcat is: http://test.example.com/test/testpage

In our example the %{HTTP_HOST} returns test.example.com which is ok.

But referenced inside RewriteRule with input to map function it fails.

rewrite.conf:
 RewriteMap portals org.apache.catalina.valves.rewrite.EPSMapPage /usr/local/tomcat/easier-conf/portals.txt
 RewriteRule .* ${portals:%{HTTP_HOST}}

The EPSMapPage is a rewrite map class we have made according to tomcat valve documentation and we have checked that this class works correct.

So this portion of the RewriteRule ${portals:%{HTTP_HOST}} should feed in test.example.com (%{HTTP_HOST}) to map portals but actually it feeds in /test/testpage not the hostname.

org.apache.catalina.valves.rewrite.EPSMapPage
 public String lookup(String key) {
  System.out.println("Key: " + key);
  return map.get(key.substring(1));
 }
console output:
 Key: /test/testpage

When using
RewriteRule .* %{HTTP_HOST}
it works at expected And redirects to
test.example.com/test.example.com
Comment 1 Remy Maucherat 2018-08-31 09:20:47 UTC
There's no code to handle %{} in that location. Is it supposed to be supported at all ?

Also, since r1676615, I not sure a simple dumb static key works (probably nobody uses it).
Actually, the only thing supported at the moment is a back reference (which is useful so it's ok I guess).
Comment 2 Rainer Jung 2018-08-31 10:36:06 UTC
IMHO httpd mod_rewrite does full resolution via do_expand() of the key used to look up in the map. So any use of variables, back references etc. seems to be supported in that module. I haven't tried it though.
Comment 3 ITASSIST 2018-08-31 10:50:35 UTC
We have tried it in Apache httpd and it works there. So if we could get this working in tomcat we do not need Apache httpd!
Comment 4 Remy Maucherat 2018-08-31 11:19:25 UTC
Ok, thanks for the feedback. Rainer, does it need to be fully recursive, or is it ok to just handle the simple case (= the vaule is either static, $n or %{variable}) ?
Comment 5 Rainer Jung 2018-09-03 11:07:43 UTC
httpd mod_rewrite code seems to be recursive, but it contains the following a little cryptic comment:

To make rewrite maps useful, the lookup key and
default values must be expanded, so we make
recursive calls to do the work. For security
reasons we must never expand a string that includes
verbatim data from the network. The recursion here
isn't a problem because the result of expansion is
only passed to lookup_map() so it cannot be
re-expanded, only re-looked-up. Another way of
looking at it is that the recursion is entirely
driven by the syntax of the nested curly brackets.

Regards,

Rainer
Comment 6 Remy Maucherat 2018-09-03 12:51:37 UTC
Ok, so adding recursion works. I suppose maps are the only thing that need it, right ?
Comment 7 Remy Maucherat 2018-09-03 13:46:10 UTC
I added the "feature"/bugfix for 9.0.12 and 8.5.34.

I have no idea what the language from the comment "For security reasons we must never expand a string that includes verbatim data from the network." means since well, that's the point and this BZ asks for ${portals:%{HTTP_HOST}} (%{HTTP_HOST} is clearly verbatim network data). Comments ? Maybe it means you shouldn't parse network data (only evaluate), but that's not the case here obviously: the configuration is parsed on valve start and that's it.
Comment 8 Rainer Jung 2018-09-03 14:05:03 UTC
(In reply to Remy Maucherat from comment #6)
> Ok, so adding recursion works. I suppose maps are the only thing that need
> it, right ?

Yes, I think so. I didn't find another way to trigger recursion in mod_rewrite.
Comment 9 Felix Schumacher 2018-09-03 19:35:25 UTC
(In reply to Remy Maucherat from comment #7)
> I added the "feature"/bugfix for 9.0.12 and 8.5.34.
> 
> I have no idea what the language from the comment "For security reasons we
> must never expand a string that includes verbatim data from the network."
> means since well, that's the point and this BZ asks for
> ${portals:%{HTTP_HOST}} (%{HTTP_HOST} is clearly verbatim network data).
> Comments ? Maybe it means you shouldn't parse network data (only evaluate),
> but that's not the case here obviously: the configuration is parsed on valve
> start and that's it.

I would read it so that data sent from client should not be expanded any further.
So for example a user sends "%{HTTP_HOST}" as the value of the host header, this would/could result in a infinite loop. There are probably more malicious examples that could be thought of.
Comment 10 Remy Maucherat 2018-09-03 20:01:21 UTC
Ok, then it's fine as there's no dynamic parsing. If "%{HTTP_HOST}" evaluates to "%{HTTP_HOST}", then it's the final result.
Comment 11 ITASSIST 2018-09-05 23:36:34 UTC
Hi,

seems like you did a patch to fix this and that is really great. Then we can test it when next release is available.

Today we have a .htaccess file with some URL rewriterules used in Apache httpd that we want to convert to use in Tomcat Valve rewrite.config. But we find it very difficult to debug it. Is there any way to enable logging to actually see what rewrite valve is doing when parsing/rendering the rewrite.config file. Since it is many regular expressions it is very hard to find out what is going on.

Thank you!

Best Regards

Frode
Comment 12 Remy Maucherat 2018-09-06 06:14:47 UTC
BZ is for reporting issues and not discussions or support. If you would like additional help, you can move this to the Tomcat user mailing list. If you think about a specific issue, you can file a BZ,