Bug 56848

Summary:	Tomcat accept-language parsing doesn't properly handle IETF BCP47 language tags
Product:	Tomcat 7	Reporter:	Andy Wang <dopey>
Component:	Catalina	Assignee:	Tomcat Developers Mailing List <dev>
Status:	RESOLVED FIXED
Severity:	normal
Priority:	P2
Version:	7.0.55
Target Milestone:	---
Hardware:	PC
OS:	All

Description Andy Wang 2014-08-13 14:45:23 UTC

Microsoft started using IETF BCP 47 language tags in Windows 8, so for languages such as Chinese Traditional (formerly zh-CN) and Chinese Simplified (formerly zh-TW) the default language tags are now zh-hant-CN and zh-hans-TW respectively.

The tomcat accept-language parsing method Request.parseLocalHeaders doesn't handle these tags properly.

Here's the bcp 47 spec:
http://tools.ietf.org/html/bcp47

And here's a description of some of the changes in Java 7 for this:
http://docs.oracle.com/javase/tutorial/i18n/locale/create.html

Where Java 7 is involved, the Locale.forLanguageTag method
http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html#forLanguageTag(java.lang.String)

works well assuming a fully compliant language tag.  Perhaps Tomcat should try to make use of this via reflection instead of parsing the entry?  The grammar for bcp47 tags doesn't make for easy parsing as various subsections of the tag are only valid if they match to particular ISO codes.
see:
http://tools.ietf.org/html/bcp47#section-2.1

Comment 1 Mark Thomas 2014-08-15 12:47:58 UTC

This has been fixed in 8.0.x for 8.0.11 onwards.

Comment 2 Mark Thomas 2014-09-03 17:48:49 UTC

This has also been fixed in 7.0.x for 7.0.56 onwards.

Comment 3 Mark Thomas 2014-09-03 18:54:40 UTC

The fix is Java 7 specific. Tomcat 7 has to run on Java 6 and there simply isn't the support for BCP47 in the Java 7 API.

I'm leaning towards resolving this as WONTFIX.

Comment 4 Andy Wang 2014-09-04 03:18:03 UTC

Would it make sense to try to use reflection and only support BCP47 if Java 7+ is used?

Comment 5 Mark Thomas 2015-02-28 19:48:14 UTC

I was in two minds whether or nor it was worth doing this. In the end, we had another requirement for something similar so this got included in the general solution. It will be in 7.0.60 onwards.