Bug 38895

Summary: Http headers with an underscore "_" change into hyphen "-"
Product: Tomcat Connectors Reporter: Carlos Rutenfrans <carlos>
Component: isapiAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED FIXED    
Severity: normal    
Priority: P2    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows Server 2003   

Description Carlos Rutenfrans 2006-03-08 17:49:37 UTC
I'm using isapi_redirect.dll 1.2.15 on IIS-6 Windows 2003 server and seems 
that isapi_redirect is not respecting the underscores of the header 
lines. 
Is there someone else how is having this problem. And how can i solve this 
problem?
 
Here is a small part of my logging:
 
Logging of the headers on HTTP port 80 IIS6:

	POST /mapxtreme471/mapxtreme HTTP/1.1 
	 Content-Type: text/xml 

	 MI_XMLProtocolRequest: ServerCapabilitiesRequest 
	 MI_XMLProtocolVersion: 
MI_XML_Protocol_ServerCapabilitiesRequest_4_7_1 

	 User-Agent: Mozilla/4.0 (Windows 2003 5.2) Java/1.5.0_06 
	 Host: holds002 
	 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 
	 Connection: keep-alive 
	 Content-Length: 217 

	 <?xml version="1.0" encoding="UTF-8"?> 
	 <!DOCTYPE RequestEnvelope SYSTEM
"MI_XML_Protocol_ServerCapabilitiesRequest_4_7_1.dtd"> 
	 <RequestEnvelope><ServerCapabilitiesRequest
GetXMLEncoding="true" /></RequestEnvelope> 

Logging of the headers on AJP13 port 8009 Tomcat 5.0.30

	.4.s....HTTP/1.1.../mapxtreme471/mapxtreme..
	10.0.0.160..
	10.0.0.160...holds002..P......
	keep-alive.....217.....text/xml....4text/html, image/gif, image/jpeg, 
*; q=.2, */*; q=.2.....holds002....,Mozilla/4.0 (Windows
2003 5.2)
Java/1.5.0_06...mi-xmlprotocolrequest...ServerCapabilitiesRequest...mi-x
mlprotocolversion../MI_XML_Protocol_ServerCapabilitiesRequest_4_7_1.....
......4....<?xml version="1.0" encoding="UTF-8"?>

	<!DOCTYPE RequestEnvelope SYSTEM
"MI_XML_Protocol_ServerCapabilitiesRequest_4_7_1.dtd">

	<RequestEnvelope><ServerCapabilitiesRequest
GetXMLEncoding="true" /></RequestEnvelope> 


You can see the "_" is changed to a "-" 

	MI_XMLProtocolRequest ==> ajp13 ==> mi-xmlprotocolrequest
	MI_XMLProtocolVersion ==> ajp13 ==> mi-xmlprotocolversion
	 
Can someone help me ? Is this an isapi_redirect problem or is it an IIS problem ?
 
Groet
          Carlos Rutenfrans
          Lekdijk west 16
          3961 MC Wijk bij Duurstede
          Holland
 
          tel:        06-14255398
          e-mail:   carlos@holds.nl
Comment 1 Yoav Shapira 2006-04-14 18:40:24 UTC
I'm not an AJP expert, and I've never heard of this bug before, but I'm curious,
does it happen on Tomcat 5.5.16 as well?
Comment 2 Carlos Rutenfrans 2006-04-15 07:40:47 UTC
(In reply to comment #1)
As you can see the logging is between IIS 6  port 80 and AJP port 8009. So it's
before you enter Tomcat. I don't think that the version of Tomcat matters for
this problem. 
If you want I can try it with Tomcat 5.5.16, but I think that this problem is
something in ISAPI.
Comment 3 Tim Whittington 2010-01-07 13:31:47 UTC
This is an explicit behaviour in the ISAPI Redirector controlled by a compile time directive (USE_RAW_HEADERS) that exists in all versions of the redirector.

The redirector can be built in two ways:

1) By default, the headers are queries from IIS using the ALL_HTTP server variable.
This results in all the headers being upper cased, the names being prepended with HTTP_ and (undocumented it seems) all dashes being converted to underscores.
There is code in the redirector to adjust for all this, but unfortunately it means that the case of header names is lost, and all underscores are converted to dashes.

2) With USE_RAW_HEADERS defined, headers are retrieved in their raw state using the ALL_RAW server variable.

I've confirmed that a build with USE_RAW_HEADERS defined fixes your problem, but I'm not sure why the option is there in the first place (and why the default is to not do this) - perhaps someone with a longer memory can comment.

This issue should probably be resolved as by design, but I'd like to see if there's a good reason to not have USE_RAW_HEADERS as the default.
Comment 4 Tim Whittington 2010-03-07 21:28:28 UTC
Changed the default behaviour to use ALL_RAW instead of ALL_HTTP to obtain headers, which prevents the CGI style header name munging.
Old behaviour is available by defining USE_CGI_HEADERS.

Will be part of 1.2.31
Comment 5 Joachim Wagner 2010-09-03 05:02:02 UTC
General comment (without knowing tomcat): HTTP RFC 2616 defines HTTP header field names as case-insensitive. Existing applications will break if suddenly the raw headers are used and the problems may not surface in testing as your test client probably uses the same casing as your server app.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

My best guess why underscores are converted to hyphens by tomcat is that apache httpd converts in the other direction (in order to generate valid environment variable names for CGI programs).
Comment 6 Rainer Jung 2010-09-03 05:15:56 UTC
This is not about Tomcat itself but instead about forwarding requests through a web server reverse proxy. More precisely about using the ISAPI redirector plugin to IIS provided by the Tomcat connectors project.

Traditionally that plugin does some mangling of the request headers before forwarding them to Tomcat (because of using a specific part of the IIS API which behaves similar to the CGI headers convention). After the change, te headers will be forward to Tomcat as is.