Bug 51223 - 304 HTTP Not Modified strips out CORS headers
Summary: 304 HTTP Not Modified strips out CORS headers
Status: NEW
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Core (show other bugs)
Version: 2.4.10
Hardware: PC All
: P2 normal with 25 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL: https://goldtreemillers.com/product/m...
Keywords: PatchAvailable
Depends on:
Blocks:
 
Reported: 2011-05-18 19:40 UTC by Arthur Kopatsy
Modified: 2023-09-05 07:14 UTC (History)
11 users (show)



Attachments
Patch for the ubuntu package (927 bytes, application/octet-stream)
2011-05-18 19:40 UTC, Arthur Kopatsy
Details
POST (11.19 KB, image/gif)
2015-06-10 23:45 UTC, Mark Nottingham
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arthur Kopatsy 2011-05-18 19:40:42 UTC
Created attachment 27027 [details]
Patch for the ubuntu package

Per the RFC, HTTP Not Modified should not include entity headers "Section 10.3.5: the response SHOULD NOT include other entity-headers"

However, the Cross-Origin-Resource-Sharing spec (http://www.w3.org/TR/cors/) defines a few headers that are not entity headers and should therefore be allowed in the 304 response:
"Access-Control-Allow-Origin",
"Access-Control-Allow-Credentials",
"Access-Control-Allow-Methods",
"Access-Control-Allow-Headers",
"Access-Control-Max-Age"

I understand that CORS is currently only a draft but it currently prevents any web application from properly adopting this new standard. Indeed, a client making a CORS request will not see the response if it is an Http Not Modified 304. The browser will block it due to the missing CORS headers.

Patch attached.
Comment 1 Mark Nottingham 2014-11-19 04:45:16 UTC
CORS doesn't require those headers on a 304, and indeed browsers work without them present on it. This is because many 304s are generated from intermediary caches that can't be updated to know about CORS.

Recommend INVALID.
Comment 2 Michiel de Jong 2014-12-11 17:46:36 UTC
Mark Nottingham:
> browsers work without them present on it

Maybe I misunderstood your point, but I was not able to reproduce that finding.

My steps:

* Run this node script on test.com:

````js
require('http').createServer(function(req, res) {
  res.writeHeader(304);
  res.end();
}).listen(8080);
````

* Visit http://example.com/ and paste this into the console:

````js
xhr = new XMLHttpRequest();
xhr.open('GET', 'http://test.com:8080/', true);
xhr.send();
````

Firefox on Ubuntu reports:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://test.com:8080/. This can be fixed by moving the resource to the same domain or enabling CORS.

Chromium on Ubuntu reports:
XMLHttpRequest cannot load http://test.com:8080/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://example.com' is therefore not allowed access.
Comment 3 Mark Nottingham 2014-12-13 00:10:55 UTC
Full test here:
  https://github.com/w3c/web-platform-tests/pull/1400
Comment 4 Jörn Berkefeld 2015-06-09 14:08:16 UTC
just ran into this issue.
we were upgrading our APIs to have them send less data when we noticed that our off-site installs stopped working. I can fully confirm Michiel de Jong's findings.

While this might present problems with static files and CDNs/proxys as stated on GitHub, it's simply very much helpful if triggered by server-side scripts to tell the browser to carry on using whatever response it received earlier.

in case that wasn't clear enough:

we intentionally send both, the access-control-allow-origin:* header as well as the 304 status code, cutting the transferred data in half.


as of now, we need to check for CORS requests and fall back to code 200.
Comment 5 Mark Nottingham 2015-06-09 23:06:12 UTC
Jörn,

Can you give a page that reproduces it?

Michiel's test isn't realistic; Apache won't send a 304 if the client doesn't send a conditional request.
Comment 6 Jörn Berkefeld 2015-06-10 16:33:07 UTC
go to https://jsfiddle.net/7t84bj05/1/
you will see some video.

in the dev toolset of your choice you will find a call to https://nlv.bittubes.com/api/a/3/
The first time it will return a 200, once you pressed play, more calls will be made that will get a 304 to save on traffic - and causing the error.

this works just fine from the same domain:
https://nlv.bittubes.com/play/?api=dev&uid=FB4HGE


(yes, the same-domain version uses a different api version but that's because I needed to disable our CORS workaround, which uses 200 for CORS requests instead of 304s, for this demo)
Comment 7 Mark Nottingham 2015-06-10 23:45:27 UTC
Created attachment 32804 [details]
POST

That's because you're using POST on it; see attached.
Comment 8 Jörn Berkefeld 2015-06-11 05:42:51 UTC
You know how they say "a picture is worth a thousand words"? - not always the case. Could you please elaborate on that?
Comment 9 Mark Nottingham 2015-06-11 05:55:30 UTC
(In reply to Jörn Berkefeld from comment #8)
> You know how they say "a picture is worth a thousand words"? - not always
> the case. Could you please elaborate on that?

:)

Browsers don't cache POST responses. While technically responses to POST *are* cacheable, it's only so that a subsequent GET can be satisfied by them in very specific conditions (and browsers don't bother to implement that pattern).

I'm not sure why the second POST is getting a 304 response - are you generating that in your code?
Comment 10 Jörn Berkefeld 2015-06-11 09:34:05 UTC
yes exactly - I am generating that manually. It's an API, not a static file. The problem is, that Apache strips the Access-Control-* headers that I also manually set from the response - even if I set that before the 304 header. It's also not about the browser which obviously is fine with being fooled into accepting the 304 even though it has nothing in it's cache to match that. works just fine on same-origin. Fails cross-origin due to the missing Access-... header.

so once again: it's not about 304 being generated by Apache, but via whatever server-side script you use. Apache sends that 304 header but strips the CORS headers.


That's essentially the same situation described by Michiel de Jong, only that I am using Apache+PHP on the server-side and not node.js, which, now that I think about it, actually was a bit of a weird example for an Apache bugtracker. 
Though uncle Google tells me he's probably doing it via mod_proxy.


here is a blog-post coming to the same conclusion:
http://blog.rampinteractive.co.uk/cors-html5-application-cache-manifest-dont-work-together-neither-cors-apache/
Comment 11 adam 2016-04-04 23:02:16 UTC
We have the same issue. CGI Api + Javascript that's called cross origin.
Has this been accepted as a BUG and the fix applied?
Comment 12 Eric Covener 2016-04-05 00:12:26 UTC
(In reply to Arthur Kopatsy from comment #0)
> Created attachment 27027 [details]
> Patch for the ubuntu package
> 
> Per the RFC, HTTP Not Modified should not include entity headers "Section
> 10.3.5: the response SHOULD NOT include other entity-headers"

FWIW Seems to be rephrased significantly in http://tools.ietf.org/html/rfc7232#section-4.1
Comment 13 rdehouss 2016-05-04 12:49:02 UTC
Hello,

I do face the issue as well.

Cross domain, I fetch a resource which is cacheable and when apache respond with 304, the CORS headers are not present and so, the browser (Firefox in this case) alerts with:
ross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://test.com/test.json. (Reason: CORS header 'Access-Control-Allow-Origin' does not match 'http://test.com').

If I do a hard refresh, I get back a status 200 and with it, the CORS headers are well there.

I also do send the Vary: Origin to specify that the cache should take into account the Origin header but it does not help.

Can the patch provided be considered?

Thanks

Cheers,

Raphaël Dehousse
Comment 14 Eric Covener 2016-05-16 14:29:44 UTC
(In reply to rdehouss from comment #13)
> Hello,
> 
> I do face the issue as well.
> 
> Cross domain, I fetch a resource which is cacheable and when apache respond
> with 304, the CORS headers are not present and so, the browser (Firefox in
> this case) alerts with:
> ross-Origin Request Blocked: The Same Origin Policy disallows reading the
> remote resource at http://test.com/test.json. (Reason: CORS header
> 'Access-Control-Allow-Origin' does not match 'http://test.com').
> 
> If I do a hard refresh, I get back a status 200 and with it, the CORS
> headers are well there.
> 
> I also do send the Vary: Origin to specify that the cache should take into
> account the Origin header but it does not help.
> 
> Can the patch provided be considered?

Can you share the request and response headers both on the initial uncached request and the subsequent conditional request?  Based on @mnot's posts in this PR, it seems like there is yet to be a "natural" case of this being an issue documented.
Comment 15 bveilleux 2016-07-11 15:38:10 UTC
I am also experiencing this same problem on Firefox.  Ie seems to work fine, probably because it does not consider accessing a different port to be a cross domain request. Has there been any progress on this?
Comment 16 Edward Rudd 2016-10-19 23:48:24 UTC
I've reported a bug with Mozilla to get more clarification as there is some conflict between the CORS and the original WHAT WG "fetch" spec.

https://bugzilla.mozilla.org/show_bug.cgi?id=1311566
Comment 17 Edward Rudd 2017-02-24 13:52:56 UTC
The latest updates on the bug I reported over at mozilla states that the WhatWG spec on this has changed and 304/307 are now required to have the cors checks.. so Apache needs to be updated to reflect these changes.

https://bugzilla.mozilla.org/show_bug.cgi?id=1311566
Comment 18 Mark Nottingham 2017-02-27 01:14:26 UTC
> 304/307 are now required to have the cors checks

That is not what it says.

A 304 to a conditional request generated by a browser is not required to have CORS headers, because the cache will combine the 304's headers with the stored response. The Fetch spec has been updated to clarify that, and the tests at WPT as well.

If you want to generate the conditional in your JS code -- rather than having the browser create the conditional -- then the processing of the 304 moves back to your code, while CORS checks still happen before your code runs. Therefore, they'll fail CORS checks unless the 304 contains CORS headers, and there appears to be browser interop on that. 

That's what Anne was talking about; it does not translate to a requirement that all 304s have CORS headers.

HTTP doesn't require 304s to carry CORS headers because it's a generic protocol, and we can't update every server / intermediary every time someone adds a new header. Even if you can persuade Apache to implement this, other servers -- including intermediary caches -- won't necessarily do it, so you won't be able to rely upon the 304 always containing CORS headers.

In a situation where you control the client *and* the server *and* the path between them, of course, you can ignore this. However, that's not the Web, and it's not even really HTTP.

Could the folks who are encountering this problem speak a bit more about their use case -- specifically, why it's important to use a custom handler for 304, rather than letting the browser handle it? Have you considered using a different status code?
Comment 19 herr.ernst 2017-02-27 11:01:39 UTC
I've recently stumbled upon this issue. The problem doesn't occur when the CORS response headers are "static" (e.g. always a-c-a-o: *), but are dynamic.
Basically, our setup serving static files is like this:

SetEnvIf Origin "^http(s)?://(.*\.)?example.com(:[0-9]+)?$" has_origin=$0
Header always set Access-Control-Allow-Origin %{has_origin}e env=has_origin
Header always set Vary Origin

There are two problems for us now:
A request can first be a non-CORS request (i.e. no Origin request header), but then in another context of our site with CORS (e.g. image in one place used ordinarily in an <img> element, but later fetched for use in a <canvas> with CORS). For the second request, Apache would answer with 304 *without* CORS headers, although IMHO it should.
And the second problematic case is when the CORS response headers would change because of a changing Origin request header: First request from foo.example.com, response has A-C-A-O: sub1.example.com, then request to same resource from bar.example.com, Apache answers with 304 but again strips our CORS headers.

So, even if CORS response headers are not *required* on 304 if they did not change from the cached response, they should still be allowed for answering with a 304. Or is there another way such a case should handled?

Thanks
Comment 20 andyh 2017-12-19 14:05:56 UTC
#19: Isn't that problem solved by including "vary: origin" on the response, to indicate that the response depends on the value of the "origin" request header?
Comment 21 oberhamsi 2018-08-14 08:07:10 UTC
(In reply to andyh from comment #20)
> #19: Isn't that problem solved by including "vary: origin" on the response,
> to indicate that the response depends on the value of the "origin" request
> header?

Even with vary:origin browser are still allowed to do a conditional request. And the 304 response will fail if it does not include the corrected ACAW-header.

The cached response is 200 and still fresh but due to the different origins it "cannot be selected" https://tools.ietf.org/html/rfc7234#section-4.3 so a conditional request is made. The 304 must then contain the corrected ACAW-header so browsers can update the cached response:

> use other header fields provided in the 304 (Not Modified)
> response to replace all instances of the corresponding header
> fields in the stored response.

https://tools.ietf.org/html/rfc7234#section-4.3.4
Comment 22 Mark Nottingham 2018-08-15 06:02:35 UTC
Yes - if you're trying to update a CORS header (and many others) in a 304, Apache will strip it. See #61820.
Comment 23 Mark Nottingham 2018-08-15 06:03:47 UTC
Arg. See bug 61820.
Comment 24 oberhamsi 2018-08-15 06:23:47 UTC
Thank you for pointing me to that bug.

Our issues is with 304s generated by Apache in response to disk files. The discussion in 61820 seems to differentiate between responses generated by e.g. CGI and responses genereated by apache itself. I don't understand why. I get that some kind of filtering makes sense to adhere strictly to the spec but the CORS headers do not describe the content ("representation metadata") so i assumed they could be let through.

As a workaround to 304 having no CORS information, we attach a query parameter ?origin=document.location.origin to all CORS requests which are potentially served to multiple domains. This forces the browsers to have seperate cache entries for each origin. If we don't do this, the arising bugs are surprising to newcomers and difficult to debug since the behaviour depends on local cache.
Comment 25 Mark Nottingham 2018-08-15 06:29:44 UTC
How are you setting CORS headers? If it's with .htaccess, in mind mind that's covered by 61820 (i.e., putting something in with mod_headers takes responsibility for "generating" it, and Apache shouldn't get in the way).
Comment 26 oberhamsi 2018-08-15 06:32:31 UTC
yes, we add the headers with mod_headers. i will think about 61820
Comment 27 samey 2019-04-25 11:45:37 UTC
i just ran into this issue. we were upgrading our APIs to have them send less data when we noticed that our off-site installs https://www.chiltanpure.com/ stopped working. I can fully confirm Michiel de Jong's findings. so 304 HTTP Not Modified strips out CORS headers
Comment 28 Matt McCutchen 2020-09-07 15:23:31 UTC
Users may wish to consider the workaround I've posted in bug 61820 comment 23.
Comment 29 Christoph Reiter 2021-10-21 07:57:32 UTC
Is this fixed, now that bug 61820 is fixed? And if yes, in which versions?
Comment 30 Ruediger Pluem 2021-10-21 11:45:57 UTC
(In reply to Christoph Reiter from comment #30)
> Is this fixed, now that bug 61820 is fixed? And if yes, in which versions?

Should be. Can you give it a try? If yes we can close this one here as a duplicate.
Comment 31 Christoph Reiter 2022-05-05 09:04:19 UTC
I can confirm that at least "Access-Control-Allow-Origin" is missing for 304 in 2.4.38 in Debian Buster, and that it's there in 2.4.53 in Debian Bullseye. So I think this can be closed and is likely also fixed since 2.4.48.
Comment 32 Goldtree Millers 2023-09-05 07:14:37 UTC
The Gold Tree Millers strongly believe in the strength, purity, and freshness of nature. Our business is a diversified, expansion-focused agribusiness. Our main businesses are selling grains, processing flour, and exporting rice. based in Lahore, Pakistan. We are a group of young experts dedicated to delivering to our clients foods of the highest calibre. We seek to strengthen local communities and broaden people's access to and involvement with real foods. one and only We provide Multigrain Flour with 16 Grains in Pakistan.
https://goldtreemillers.com/
https://goldtreemillers.com/product/multigrain-flour-16-grains/