Bug 65272 - Problems proccessing HTTP request without CR in last versions
Summary: Problems proccessing HTTP request without CR in last versions
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 9
Classification: Unclassified
Component: Connectors (show other bugs)
Version: 9.0.x
Hardware: PC All
: P2 normal (vote)
Target Milestone: -----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-28 12:31 UTC by Miguel
Modified: 2021-05-05 11:30 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Miguel 2021-04-28 12:31:09 UTC
With last versions of Apache Tomcat 9, we have problems to process http request created by legacy systems that haven't <CR> separating the sections of a HTTP Request, having only a LF as delimeter.

The error returned by Tomcat is:
java.lang.IllegalArgumentException: Invalid character found in the HTTP protocol [HTTP/1.00x0aUser-Agent:]
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:559)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:261)



This request fails in Apache Tomcat 9.0.45, 9.0.41 but works in Apache Tomcat 9.0.26 or in Apache Tomcat 7.


POST /sms/feedback HTTP/1.0[LF]
User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)[LF]
Host: 10.252.12.1[LF]
Proxy-Connection: Keep-Alive[LF]
Content-Type: text/xml; charset=utf-8[LF]
Content-Length: 308[LF]
[LF]
<?xml version="1.0" encoding="UTF-8"?>[LF]
<RESPUESTA>[LF]
<TIPO>2</TIPO>[LF]
<ADC>R</ADC>[LF]
<OADC>666111222</OADC>[LF]
<SCTS>270421133100</SCTS>[LF]
<DSCTS>270421133100</DSCTS>[LF]
<IDMSG>17913195f93000088ee6b37de829cba3</IDMSG>[LF]
<AMSG>El mensaje enviado al numero 666111222 ha sido entregado el 27.04.21 13:31:00</AMSG>[LF]
</RESPUESTA>[LF]



This request works:

POST /sms/feedback HTTP/1.0[CRLF]
User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)[CRLF]
Host: 10.252.12.1[CRLF]
Proxy-Connection: Keep-Alive[CRLF]
Content-Type: text/xml; charset=utf-8[CRLF]
Content-Length: 308[CRLF]
[CRLF]
<?xml version="1.0" encoding="UTF-8"?>[CRLF]
<RESPUESTA>[CRLF]
<TIPO>2</TIPO>[CRLF]
<ADC>R</ADC>[CRLF]
<OADC>666111222</OADC>[CRLF]
<SCTS>270421133100</SCTS>[CRLF]
<DSCTS>270421133100</DSCTS>[CRLF]
<IDMSG>17913195f93000088ee6b37de829cba3</IDMSG>[CRLF]
<AMSG>El mensaje enviado al numero 666111222 ha sido entregado el 27.04.21 13:31:00</AMSG>[CRLF]
</RESPUESTA>[CRLF]




You can test this using this java code (supossing tha Tomcat is running on 8080 port at localhost):
The mensaje2 variable works fine (CRLF is represented by 13,10 codes).
The mensaje2 fails (LF is represented by 10 code)



		int[] mensaje2 = {80,79,83,84,32,47,115,109,115,47,102,101,101,100,98,97,99,107,32,72,84,84,80,47,49,46,48,13,10,85,115,101,114,45,65,103,101,110,116,58,32,77,111,122,105,108,108,97,47,52,46,48,32,40,99,111,109,112,97,116,105,98,108,101,59,32,77,83,73,69,32,
		53,46,48,59,32,87,105,110,100,111,119,115,32,57,56,59,32,68,105,103,69,120,116,41,13,10,72,111,115,116,58,32,49,48,46,50,53,50,46,49,50,46,49,13,10,80,114,111,120,121,45,67,111,110,110,101,99,116,105,111,110,58,32,75,101,101,112,45,65,108,105,
		118,101,13,10,67,111,110,116,101,110,116,45,84,121,112,101,58,32,116,101,120,116,47,120,109,108,59,32,99,104,97,114,115,101,116,61,117,116,102,45,56,13,10,67,111,110,116,101,110,116,45,76,101,110,103,116,104,58,32,51,48,56,13,10,13,10,60,63,120,109,
		108,32,118,101,114,115,105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,110,103,61,34,85,84,70,45,56,34,63,62,13,10,60,82,69,83,80,85,69,83,84,65,62,10,60,84,73,80,79,62,50,60,47,84,73,80,79,62,10,60,65,68,67,62,82,60,47,65,68,67,62,
		10,60,79,65,68,67,62,54,55,55,50,50,50,53,54,50,60,47,79,65,68,67,62,10,60,83,67,84,83,62,50,55,48,52,50,49,49,51,51,49,48,48,60,47,83,67,84,83,62,10,60,68,83,67,84,83,62,50,55,48,52,50,49,49,51,51,49,48,48,60,47,68,83,67,84,83,62,10,60,
		73,68,77,83,71,62,49,55,57,49,51,49,57,53,102,57,51,48,48,48,48,56,56,101,101,54,98,51,55,100,101,56,50,57,99,98,97,51,60,47,73,68,77,83,71,62,10,60,65,77,83,71,62,69,108,32,109,101,110,115,97,106,101,32,101,110,118,105,97,100,111,32,97,
		108,32,110,117,109,101,114,111,32,54,55,55,50,50,50,53,54,50,32,104,97,32,115,105,100,111,32,101,110,116,114,101,103,97,100,111,32,101,108,32,50,55,46,48,52,46,50,49,32,49,51,58,51,49,58,48,48,60,47,65,77,83,71,62,10,60,47,82,69,83,80,85
		,69,83,84,65,62};
		
		int[] mensaje = {80,79,83,84,32,47,115,109,115,47,102,101,101,100,98,97,99,107,32,72,84,84,80,47,49,46,48,10,85,115,101,114,45,65,103,101,110,116,58,32,77,111,122,105,108,108,97,47,52,46,48,32,40,99,111,109,112,97,116,105,98,108,101,59,32,77,83,73,69,32,
				53,46,48,59,32,87,105,110,100,111,119,115,32,57,56,59,32,68,105,103,69,120,116,41,10,72,111,115,116,58,32,49,48,46,50,53,50,46,49,50,46,49,10,80,114,111,120,121,45,67,111,110,110,101,99,116,105,111,110,58,32,75,101,101,112,45,65,108,105,
				118,101,10,67,111,110,116,101,110,116,45,84,121,112,101,58,32,116,101,120,116,47,120,109,108,59,32,99,104,97,114,115,101,116,61,117,116,102,45,56,10,67,111,110,116,101,110,116,45,76,101,110,103,116,104,58,32,51,48,56,10,10,60,63,120,109,
				108,32,118,101,114,115,105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,110,103,61,34,85,84,70,45,56,34,63,62,10,60,82,69,83,80,85,69,83,84,65,62,10,60,84,73,80,79,62,50,60,47,84,73,80,79,62,10,60,65,68,67,62,82,60,47,65,68,67,62,
				10,60,79,65,68,67,62,54,55,55,50,50,50,53,54,50,60,47,79,65,68,67,62,10,60,83,67,84,83,62,50,55,48,52,50,49,49,51,51,49,48,48,60,47,83,67,84,83,62,10,60,68,83,67,84,83,62,50,55,48,52,50,49,49,51,51,49,48,48,60,47,68,83,67,84,83,62,10,60,
				73,68,77,83,71,62,49,55,57,49,51,49,57,53,102,57,51,48,48,48,48,56,56,101,101,54,98,51,55,100,101,56,50,57,99,98,97,51,60,47,73,68,77,83,71,62,10,60,65,77,83,71,62,69,108,32,109,101,110,115,97,106,101,32,101,110,118,105,97,100,111,32,97,
				108,32,110,117,109,101,114,111,32,54,55,55,50,50,50,53,54,50,32,104,97,32,115,105,100,111,32,101,110,116,114,101,103,97,100,111,32,101,108,32,50,55,46,48,52,46,50,49,32,49,51,58,51,49,58,48,48,60,47,65,77,83,71,62,10,60,47,82,69,83,80,85
				,69,83,84,65,62};
		
		try {

			Socket sock = new Socket("127.0.0.1", 8080);
			
			for (int i=0; i<mensaje.length; i++) {
				sock.getOutputStream().write(mensaje[i]);
			}
			
			BufferedReader buffer = new BufferedReader(new InputStreamReader(sock.getInputStream()));
			
			int b;
			while ((b = buffer.read()) >= 0) {
				System.out.print((char) b);
			}
					
		} catch (Exception e) {
			e.printStackTrace();
		}
		
Thank you
Comment 1 Michael Osipov 2021-04-28 13:22:21 UTC
How old are those systems?
Comment 2 Miguel 2021-04-28 14:03:06 UTC
(In reply to Michael Osipov from comment #1)
> How old are those systems?

I haven't the data. But I see that HTTP request are 1.0 version... then is very old...
We have some legacy systems. One of these is a SMS Center that we can't change.

We are afraid because this problem can set our max Tomcat Version to 9.0.26 (I didn't try with all versions between 9.0.26 and 9.0.41).

We know that http standard specify <cr><lf> to separate the components of a HTTP request, but this new behaviour detected in last versions generate problems for us in some use cases.
Comment 3 Mark Thomas 2021-04-28 14:13:18 UTC
This stricter parsing was introduced as part of the fix for CVE-2020-1935.

Because the fix was in response to a security issue, that makes it a lot less likely the current behaviour will be changed. 

I'll note that both RFC 7230 and RFC 2616 state that recipients MAY treat single LR as a line terminator. That makes the behaviour entirely optional and Tomcat is still fully HTTP spec compliant by opting to reject requests that use LF as the line terminator.

I need to look into the details of that vulnerability to see if there are any options to relax the current behaviour without re-introducing a security concern.
Comment 4 Miguel 2021-04-28 15:01:33 UTC
(In reply to Mark Thomas from comment #3)
> This stricter parsing was introduced as part of the fix for CVE-2020-1935.
> 
> Because the fix was in response to a security issue, that makes it a lot
> less likely the current behaviour will be changed. 
> 
> I'll note that both RFC 7230 and RFC 2616 state that recipients MAY treat
> single LR as a line terminator. That makes the behaviour entirely optional
> and Tomcat is still fully HTTP spec compliant by opting to reject requests
> that use LF as the line terminator.
> 
> I need to look into the details of that vulnerability to see if there are
> any options to relax the current behaviour without re-introducing a security
> concern.



Thank you for your work.

Additional information: Now we see that the first version with problems are 9.0.31 (doesn't response) and with 9.0.33 the response is the reported originally.

We wait for news.
Regards
Comment 5 Mark Thomas 2021-04-28 16:20:12 UTC
I've started to look at this. So far I have spotted a couple of minor issues with the current parsing that I need to fix. Commits for those will follow shortly.

I haven't yet found any reason not to allow LF as a line terminator but I am still reviewing the parsing code.
Comment 6 Mark Thomas 2021-04-28 17:28:54 UTC
It currently looks like this is fixable. PR at https://github.com/apache/tomcat/pull/417

Need to allow time for the Tomcat community to review the PR.
Comment 7 Mark Thomas 2021-05-05 11:30:25 UTC
Fixed in:
- 10.0.x for 10.0.6 onwards
- 9.0.x for 9.0.46 onwards
- 8.5.x for 8.5.66 onwards