Bug 58007

Summary: 400 Bad Request with fully qualified domain name over HTTPS
Product: Apache httpd-2 Reporter: Fred Morris <consulting>
Component: mod_sslAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: NEW ---    
Severity: normal CC: jld, leho, szg0000, yfilmps, ylavic.dev
Priority: P2    
Version: 2.4.7   
Target Milestone: ---   
Hardware: All   
OS: Linux   

Description Fred Morris 2015-06-06 19:05:18 UTC
Very simple to reproduce:

1) Visit https://apache.org. or https://apache.org./foo

MIND THE DOT! That makes it really fully qualified. It's a DNS thing.

2) Click through the cert invalid warning (maybe you want to fix that too).

3) See 400 error.
Comment 1 Fred Morris 2015-06-06 19:06:40 UTC
I'll observe that yeap, BZ helpfully rewrote my first URL so that it doesn't work.
Comment 2 Fred Morris 2015-06-06 22:50:14 UTC
Apache rewrite rules to mitigate the bug: https://lists.dns-oarc.net/pipermail/dns-operations/2015-June/013321.html
Comment 3 Fred Morris 2015-06-07 16:35:59 UTC
Actual code in ssl_engine_kernel.c:

            if (strcasecmp(host, servername)) {
                ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server, APLOGNO(02032)
                            "Hostname %s provided via SNI and hostname %s provided"
                            " via HTTP are different", servername, host);
                return HTTP_BAD_REQUEST;

Specifically the hostname provided via SNI has the trailing dot, the hostname provided via HTTP does not.
Comment 4 Fred Morris 2015-06-07 17:37:53 UTC
RFC 6066 says:

   "HostName" contains the fully qualified DNS hostname of the server,
   as understood by the client.  The hostname is represented as a byte
   string using ASCII encoding without a trailing dot.  This allows the
   support of internationalized domain names through the use of A-labels
   defined in [RFC5890].  DNS hostnames are case-insensitive.  The
   algorithm to compare hostnames is described in [RFC5890], Section

However, clients are not doing this. I've tried Firefox (Linux & Mac), Safari. Some versions of curl work, some don't.
Comment 5 Fred Morris 2015-06-07 19:16:29 UTC
I don't think you can fix the invalid cert warning in the browser. ;-) However, I question the logic behind the 400 error.

A cert has been returned, the handshake has completed (or not). The SNI hostname is now compared with the hostname from inside the request, the SNI hostname in question containing an invalid dot at the end. If the invalid dot were removed they would compare equal.
Comment 6 Yann Ylavic 2015-06-07 19:46:01 UTC
I fail to see the bug here...

Apache httpd does not really mind about FQDN or not, its only requirement is that the SNI, the Host header and the ServerName/ServerAlias are consistent.

IMHO, if the browser sends differents SNI and Host header, that's its bug.
Comment 7 Fred Morris 2015-06-07 23:24:53 UTC
(In reply to Yann Ylavic from comment #6)
> [...]
> IMHO, if the browser sends differents SNI and Host header, that's its bug.

Good point. From what I can see, Apache logs the Host: header with the dot if you use %{Host}i.

So Firefox is broken, but it's consistent AFAICT. OTOH Apache is logging the host with the dot in the request log, and complaining that it doesn't have a dot in the error log.
Comment 8 Yann Ylavic 2015-06-08 11:47:39 UTC
The SNI is compared against r->hostname, which is a parsed value of the original Host header (without the port, the trailing dots), supposedly SNI compatible...

So I wonder if:
1) we should fix this, since clients are not supposed to dot FQDNs in SNI,
2) the SNI should be matched against the Host header instead of r->hostname (i.e. the fix belongs in mod_ssl only),
3) server/vhost.c::fix_hostname() should stop stripping the trailing dot.

My personnal preference would be 3) since two hosts w/ and w/o the trailing dot (FQDN or not) are possibly not the same ones (and hence the same VirtualHosts).
httpd could (theorically) be asked to handle them separately (or not), by using differents ServerNames (or one as ServerName, the other as ServerAlias).
There is possibly a compatibility issue though by changing this at the vhost level, and this is beyond the SSL only scope.

So we should probably go for 2) since browsers don't play "game" 1).
I don't think we can be lenient and therefore ignore the trailing dot from SNIs, let's be consistent in a strict way.
Comment 9 Stefan Eissing 2015-06-08 12:30:39 UTC
It cannot be considered a bug when SNI and Host header differ. You cannot expect all requests on the same connection to have the same Host header either.

Browsers will reuse TLS connections, if hostnames resolve to the same IP address and wildcard certs or matching alternate names in the certificate to match.

It is really a client decision to reuse an existing connection for other hostnames.

And if the server wants the client to use another connection for such a request, 421 is the proper status code. See http://httpwg.github.io/specs/rfc7540.html#HttpExtra Section 9.1.2 Additionally, the Alt-Svc http extensions https://httpwg.github.io/http-extensions/alt-svc.html#alt-svc explicitly encourages clients to use a connection for "foreign" URLs.
Comment 10 Yann Ylavic 2015-06-08 13:04:29 UTC
Hmm, so how are we supposed to handle a subsequent request on the same connection but for a different vhost with differents SSL parameters (eg. SSLProtocol, CipherSuite, ...), renegotiate?

Where is the win with SNI here?
How does the client side know about that?
Comment 11 Yann Ylavic 2015-06-08 13:45:10 UTC
Discussion about SNIed connections reuse moved to dev@.
Comment 12 Fred Morris 2015-06-08 18:18:18 UTC
Wow. Well, there's a lot more to this than I reckoned... for the most part this discussion is beyond me. I do have one comment: any final dot should be ignored for comparison purposes.

> 3) server/vhost.c::fix_hostname() should stop stripping the trailing dot.

www.example.com and www.example.com. are the same thing; at least they're supposed to be, at least that's consistent with the principle of least surprise. Actually, the former is supposed to be the latter. Because nobody really wants to go to www.example.com.ru. or www.example.com.example.com. Right? But that's what would potentially happen with a search list. The search list will not be applied to the second one, because it ends in dot.

So: DNS says they're equivalent, but with a footnote regarding resolvers and search lists.

Apache logs them differently (with %{Host}i so I presume this comes from the Host: header): without and with the dot. I don't know if the client should be sending them differently... However I expect them to refer to the same (virtual) host, I would be very surprised if they didn't. Servers which don't treat them the same are as far as I'm concerned broken. (Apache does treat them the same, which is to say if you set up a NameVirtualHost with separate logging, both the no dot (logging 200) and dot (logging 400) get directed to that VirtualHost, which is defined without the dot.)

Now that I'm paying attention, I don't see any ServerName examples with the trailing dot in the Apache manual.

I just looked through RFC 2616. I didn't find anything about the dot (somebody else should check, in case I missed it). The examples all show without the dot though. For the SNI header, the relevant specification says that the hostname should be sent without the dot and it's very specific and clear about that. On that basis we can conclude that Firefox is broken (and it's hardly the only broken client out there); but it is consistent between the SNI hostname and the Host: header.

Now I want to offer a different example: jira and jira. (It's a better example for these purposes than mail and mail., although you can ask the guy who owns mail.com about the impact of search lists if you don't see the importance.)

jira. (with the dot) isn't going to resolve anywhere.. unless I have a private vanity TLD of jira. But if I have example.com in my search list, the resolver library tacks it onto jira and attempts to resolve jira.example.com.

What gets logged is jira without the dot; and apparently that's consistent with SNI because the page gets served.

What do we expect the browser to send in this case? In spite of Firefox's (misguided IMO) attempts to manage its own DNS resolution I don't think we can expect clients to have visibility into what the resolver is doing, I think we just have to expect this; and consequently (and unfortunately) we can't expect the server to magically apply the same search list and come up with jira.example.com. But nonetheless jira and jira. should be the same vhost as far as the server is concerned.
Comment 13 Fred Morris 2015-06-09 01:12:29 UTC
See also: bug 57810
Comment 14 Fred Morris 2015-06-09 01:46:09 UTC
Interesting: bug 56718
Comment 15 Fred Morris 2015-06-09 02:07:57 UTC
It's not like compatibility with broken clients isn't a consideration: bug 56241
Comment 16 Fred Morris 2016-09-22 16:42:34 UTC
When I tried this just now, apache.org served up a certificate for openoffice.org.
Comment 17 Yann Ylavic 2016-09-22 20:47:29 UTC
Is it still reproducible with latest 2.4 (2.4.23)?

The SNI string is not compared directly with the Host header anymore, and the decision to block the request (now with status 421) is solely based on the compatibility (TLS configuration/parameters wise) of the vhost selected from the handshake (based on the SNI) and the one finally selected from the request (based on the Host header).

Both selections compare with the ServerName/ServerAlias(es) declared in the configuration, so it's up to admin to use a FQDN, or not, or both (one as Servername, the other as ServerAlias).
Comment 18 Yann Ylavic 2016-09-22 20:47:42 UTC
*** Bug 57810 has been marked as a duplicate of this bug. ***
Comment 19 Fred Morris 2017-02-09 17:03:51 UTC
(In reply to Yann Ylavic from comment #17)
> Is it still reproducible with latest 2.4 (2.4.23)?

Sorry, did not see this when the comment was posted. It's still reproducible today at apache.org which is running Apache/2.4.7 (Ubuntu)

I'll see if I can find the time to set up a test with latest.