Bug 53845

I'm a member of the W3C Tracking Protection Working Group. Some added perspective from Do Not Track negotiations may be helpful here. Short version: SVN commit 1371878 is definitely not required by the nascent W3C privacy standards, and it will facilitate running afoul of those standards.

The working group has decided that a mainstream browser is not compliant if it silently enables Do Not Track by default. The beta version of Internet Explorer 10, for example, is noncompliant. The group has *not*, however, decided:

1) An installation/first-run option, like shipping Internet Explorer 10, is noncompliant. The draft text, in fact, notes this is an acceptable implementation: "We do not specify how tracking preference choices are offered to the user or how the preference is enabled: each implementation is responsible for determining the user experience by which a tracking preference is enabled. . . . The user-agent might ask the user for their preference during startup, perhaps on first use or after an update adds the tracking protection feature."

2) A compliant website may ignore a syntactically valid "DNT: 1" signal from a noncompliant browser.

Furthermore, even if the W3C Do Not Track standard were to allow second-guessing "DNT: 1" from particular browsers, it certainly would not require it. Many websites would assuredly want to honor all "DNT: 1" headers or ask a user to confirm his or her preferences.

If httpd is configured in an intermediary role (e.g. mod_proxy), this commit runs into a different compliance issue: intermediaries aren't supposed to tamper with "DNT" headers. Draft text: "An HTTP intermediary must not add, delete, or modify the DNT header field in requests forwarded through that intermediary unless that intermediary has been specifically installed or configured to do so by the user making the requests."

Created attachment 29345 [details]
Proposed Patch

Apache HTTP Server does not yet implement DNT (and makes no claims of compliance) because: (1) DNT impacts first party services differently than third party services and we have no way of knowing which one applies; and, (2) the sections of the specifications regarding server compliance and the tracking status response are still in flux, particularly in regard to access logs. If we do implement DNT, the implementation might impact code throughout the whole server, and the workarounds for broken browsers might then be more subtle than simply dropping the signal. Browsers have chosen to send DNT already, in spite of it not having a proper definition and not actually doing anything for users, because it is easy for them to claim "privacy" while punting the actual work to servers.

Jonathan is incorrect. A dialog box presented to the user with a preselected option of "on" does not qualify as a default of "unset", nor do the Express settings of IE10.0 qualify as a preference for privacy (read them and see). The working group is not a judicial branch -- it will not sit around forever adjudicating whether a given implementation complies or not, and nobody has ever claimed that the standard requires servers to ignore invalid signals.  Apache chose to do so because the signal is meaningless if it is set by default, and it is harmful to deployment of DNT, to the Web, and to the open standards process if we allow such deliberate abuse to be propagated downstream.

That section of the Tracking Preference Expression has been formally reviewed by the WG several times to assure that it represents the consensus on ISSUE-4. It is part of an open standard under development, which means the right way to change it is to go through the working group process and request a change. If the working group changes its opinion regarding the "unset" default or how it might be implemented, then I (or someone faster than me) will submit a patch to Apache that corresponds to the new consensus opinion of the working group.

Apache has no particular interest in what goes in the DNT open standard -- only in that the protocol means what the WG says it means when the extra eight bytes are sent on the wire.  Of course, we'd prefer that the standard specifies something that we can implement, because we are not going to turn off access logs just because a potentially evil client asks, but this block will be removed as soon as the user agent is compliant, whether that is because of IE10 fixing their bug or the WG changing the specification.

>  Apache chose to do so

No, Apache has not chosen to do so.
YOU (or your superior at Adobe) have chosen to do so purely on your PERSONAL POLITICAL AGENDA.

This is a clear abuse of your power and a glaring breach of open source development principles.

The default Apache configuration should show a way to go and present an environment that is useable for most users.
What you did is exactly the opposite: Creating a default environment that is not usable (and so only of political reasons). THIS is just the same as if Apache would redirect every IE user to a page with the text "Get a better browser, noob!".

But please, in case you did not read my initial post, let me reiterate my points:

- This setting is in now way useful to 99,99% of the installed Apache user base and as such DOES NOT QUALIFY as a default setting. It will only cause additional processing costs that are not justifiable.

- Apache has absolutely no profit from handling the DNT setting. It will only be handled at higher levels (PHP, CGI, etc.) where the actual 'tracking' logic would be. Therefore this is an invalid manipulation of the request.

- This setting also excludes people who explicitly WANT TO USE DNT. This is the absolutely WORST CASE that could happen. Not only it is logically insane to assume that people who have a privacy setting enabled did not want it, it is also direct hit in the face of those who DO care.
You can always do what you want to people who don't care, but doing the opposite of that what people who care wish is insulting, damaging Apaches reputation right now in the media and will lead to legal problems if DNT could be enforced by law.

And also:
You might know the quote "If I tomorrow say 'I won't do any political comments anymore' then this is a political comment"?
The same goes for defaults: There is ALWAYS one.
That default is either 0 or 1 - you cannot not have a default as I have said above:

> [let's face it: there will always be a default unless you require the user to set this on first launch of the browser with only the buttons "Enable", "Disable" and "Quit Browser" - which will not happen]

(If there is no setting and the browser does not send the header at all it just means the decision about the default is up to the remote end and common sense tells us already what that means...)

Roy, up to now you have not shown any technical or logical argument that justifies the change in commit 1371878.
So all we can do is assume what right now everyone assumes and some openly says: Adobes largest customer are ad agencies and as such you have a major interest in the DNT standard failing - which you have brought us quite a bit closer to, now that everyone can say "Well, it doesn't matter what you set for DNT, all sites will ignore it either way".

Apart from that the execution of this 'patch' is just purely bad:
- The comment is pretentious and in no way helpful. Just take a look at the comments in the httpd-dav.conf or httpd-ssl.conf file for comparison.
- Your configuration also removes the DNT header if it is set to 0 which can only happen if it is explicitly set that way.

The change was voted on twice, first when it was moved to the release branch and again when the branch was approved for release as 2.4.3.  It is not a bug.  That decision process is publicly archived on the Apache mailing lists.

http://mail-archives.apache.org/mod_mbox/httpd-cvs/201208.mbox/%3C20120814205332.2548223889E1%40eris.apache.org%3E

http://mail-archives.apache.org/mod_mbox/httpd-dev/201208.mbox/%3C88C9F94E-4B5A-4B55-855A-ED7B2FE177C7%40jaguNET.com%3E

Both specifications and ISSUE-4 clearly state that the default for a general purpose UA must be "unset", meaning no signal is sent.  Since the IE dialogs default to "on" and are only presented to the user who installs the operating system (not to each user that opens the browser for the first time), IE10.0 clearly violates the standard.  The effect of that violation would impact the users of non-broken browsers, who would like their properly configured settings honored, if we didn't differentiate based on user agent.  Yes, it sucks, but the alternative is worse. DNT will not be deployed if the browser is allowed to lie about the user's preference setting.

The change has nothing to do with Adobe. I am the editor of one of the W3C specifications and have done more to advance that specification than anyone else in the working group -- it is ridiculous to suggest that Adobe doesn't want the standard to succeed. There are 54 people with voting rights on this project and I am the only one that works for Adobe.

Since you are obviously a troll and just committed libel on a public forum, this discussion is over.  You may open a new bug report when IE is corrected or the WG changes the specifications.

I deeply excuse myself if my wording has offended you - such is an inherit risk in intercultural communication that I do not always pay attention to.
I am neither a troll nor did I commit libel (I can assure you that I would not use my name if I wanted to do this). Take a look at my wording:

> So all we can do is assume what right now everyone assumes and some openly says: 

I just repeated what was the general tone in the thread linked in the first comment.
As you can very well see from the rest of the comment on technical arguments.

Also let me make clear that I have no interest in protecting Microsoft or the like. I never used IE other than for testing in the last 8 years and I have no plans to change that.

Let's split arguments into groups to make it clearer:

1. Execution of the patch
2. Applicability to Apache
3. Impact on DNT/Apache
4. Reasons for this patch

I see that #4 is not part of this discussion, so I will stop mentioning it.

Execution of the patch:

I stand by what I have said above.
- The comment is pretentious* and in no way helpful. Just take a look at the comments in the httpd-dav.conf or httpd-ssl.conf file for comparison.
- Your configuration also removes the DNT header if it is set to 0 which can only happen if it is explicitly set that way.

* (Please note that pretentious purely refers to the what the comment says and was never targeted at you)

These issues need to be addressed in any case because the default configuration is part of the documentation where the goal is to explain WHY something is done and this serves as a bad example.

Applicability to Apache:

Again, I stand by what I have said, twice.
* Apache is the wrong point to add this setting. DNT does only matter for a very small subset of servers (ie those of ad agencies) so this will 1) result in unnecessary processing for most installations and 2) The DNT header will only be processed at a higher level (eg. PHP) where this check could be made. With this setting Apache is 'stealing' information from the actual processing logic that will handle the DNT header.
* Whether DNT should default to 0 or 1 [let's face it: there will always be a default unless you require the user to set this on first launch of the browser with only the buttons "Enable", "Disable" and "Quit Browser" - which will not happen] is something to decide by the specification, not by Apache.

Impact on DNT/Apache:

* I still remember the time when Microsoft ran ad campaigns against OSS (I think the last one was 2008 or so) with slogans like "Do you know who is developing OSS? Everyone can put a virus or spyware in there" or "With thousands of people messing with the code you can never be sure what you get". It took us years to disprove those claims and get the message across that in general OSS is more stable and secure BECAUSE it is open to everyone.
What this patch does (and I'm talking about someone who will only see the patch inside Apache, not the discussion surrounding it) is proving every flagrant claim by MS to be true, because what they see is IE being excluded for some unexplained reason.
This is doing real damage to Apache right now in the media.
I deeply care about this issue as I am a user of Apache and myself active in the open source community. I would like to hear how you think about this.

* Also let me show you another view on the impact of this patch (I'm not advocating this, just saying that it might happen): Until now everything in the media that has been said about the impact of DNT was pretty much hot air and speculation. This patch is the first thing to actually create facts. As I have said before now everone can go "Well, it doesn't matter what you set for DNT, all sites will ignore it either way". As such this might be more damaging to DNT than it would actually help it.

Best regards

Contrary to Roy Fielding's frequent and vocal assertions, the W3C Tracking Protection Working Group has *not* decided that a browser with a preselected first-run option is noncompliant, nor has it decided that a website may ignore a well-formed "DNT" header from a noncompliant user agent.

The Working Group had an in-depth call about browser settings on June 6.
Minutes: http://www.w3.org/2012/06/06-dnt-minutes
Summary: http://lists.w3.org/Archives/Public/public-tracking/2012Jun/0110.html

Quoting directly from Co-Chair Aleecia McDonald's summary:

> (1) Today we reaffirmed the group consensus that a user agent MUST NOT set a default of DNT:1 or DNT:0, unless the act of selecting that user agent is itself a choice that expresses the user's preference for privacy. In all cases, a DNT signal MUST be an expression of a user's preference.
> . . .
> Implication A: Microsoft IE [10 Beta, not Release], as a general purpose user agent, will not be able to claim compliance with DNT once we have a published W3C Recommendation. As a practical matter they can continue their current default settings, since DNT is a voluntary standard in the first place. But if they claim to comply with the W3C Recommendation and do not, that is a matter the FTC (and others) can enforce.
> . . .
> (3) Today we discussed, but did not agree upon, what role the specification does or does not have in dealing with a non-compliant user agent. We still have more to talk about here to make sure we all understand the full decision space.

Apple's David Singer, an Editor, phrased the distinction between a first-run option and a silent default quite well:

> . . . I take 'default' to mean what happens if you don't do anything. If it asks every user on install/first-use, then every user is expressing a preference, and there is no 'default'. The most that there can be is a suggestion, or an initially checked box, suggesting that the user take a certain choice or direction, but it's no longer a 'default' . . . .

Justin Brookman from the Center for Democracy and Technology, another Editor, has helpfully summarized where the draft text stands on the released version of Internet Explorer 10:

> It is inaccurate to say that IE10's implementation is inconsistent with the spec . . . . The Windows flow presents information about DNT along with several other options; as an opt-in flow, you could argue that DNT should be called out more prominently, but I have seen a lot worse.

Jonathan is ignoring the fact that IE's setup is not a first-run option.  It is a configuration dialog for the installer.  The only person who sees it is the installer, and even then it is hidden amongst a set of options that are,
if anything, anti-privacy.  After that, any user added to the operating system is assigned the same settings as the installer, without ever seeing those dialogs.

I am not surprised that David Singer (Apple) has not tested the dialogs himself.

While the opinions of policy wonks are amusing, Microsoft says that this feature defaults to "on", and anyone halfway literate in installation procedures can confirm that on their own.

The Apache community should be guided first by what the W3C Tracking Protection Working Group has decided to standardize. The group has not deemed the final Internet Explorer 10 Do Not Track implementation noncompliant, nor has it sanctioned crude self-help in response to a noncompliant browser. That holds true no matter which terminology is employed to describe the Internet Explorer 10 choice architecture ("preselected," "first-run," "install-time," "multi-user," "system-wide," ...).[1]

This change is procedurally and substantively problematic for myriad reasons.

1) It unilaterally and surreptitiously circumvents a legitimate, longstanding, international, and multi-stakeholder negotiation.

2) It violates good software engineering principles by adding a special-case configuration default that
a) almost always results in unnecessary processing,
b) is assuredly unexpected by most developers, and
c) blurs the layering of web application and web server.

3) It bars some shared-server developers of the ability to honor (or even measure) an Internet Explorer 10 "DNT" header.

4) It renders Apache-based Do Not Track implementations noncompliant by default, in the quite possible (if not likely) event that the Working Group standardizes either
a) the Internet Explorer 10 implementation is compliant, or
b) a website may not ignore syntactically valid "DNT" headers from a noncompliant browser.

5) Under the current Working Group consensus and draft text, Apache itself is now noncompliant by default in a proxy configuration.

6) If pending proposals before the Working Group are accepted, it would make Apache noncompliant by default in a conventional web server configuration.

7) It's bad policy to allow or encourage second-guessing syntactically valid "DNT" headers. Every single user of a noncompliant browser would be stripped of the Do Not Track choice. That includes users who have explicitly enabled Do Not Track. In other words, this change would punish a user for the choices made by his or her browser vendor. Website implementations would likely fragment in which browsers they respect. In consequence, users, browsers, and policymakers would lose the very consistency and trust that Do Not Track is intended to promote. Likely outcomes include more common technical countermeasures by users and browsers (e.g. ad blocking), as well as regulatory enforcement by U.S. and EU authorities. There is, thankfully, a much better alternative: if a website receives a "DNT" header from a noncompliant browser, it is welcome to politely ask the user to confirm his or her preference.

8) One of the primary justifications for this patch, that there should not be a privacy default, is internally inconsistent. This patch is precisely a privacy default—it configures Apache to ignore certain privacy signals unless a developer makes a change. Recall that in the absence of this patch developers simply choose whether to honor Internet Explorer 10 "DNT" headers, either in their application logic or in their configuration files. Without this change, Apache does nothing to sway developers one way or the other.

I strongly urge the Apache community to reverse the change. Until that happens, I expect that the Apache Software Foundation, the Apache community, Adobe, and Roy Fielding will continue to experience intense criticism from researchers, advocates, policymakers, and media. As far as I'm concerned, it couldn't be more warranted.

That's all I have to say on this topic. Let's fix this and focus on successfully concluding the Do Not Track negotiations.

[1] The Working Group has only agreed on one noncompliant user agent design: a silent default in a mainstream browser. The final Internet Explorer 10 implementation is plainly not a silent default. A screenshot is available at http://www.itwriting.com/blog/wp-content/uploads/2012/09/image18.png.

> Microsoft says that this feature defaults to "on", [...]

Now do they? They've said this in relation to the Release Preview, but I have a hard time finding the usage of the word 'default' in reference to the final product in the official statement:

http://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/08/07/do-not-track-in-the-windows-8-set-up-experience.aspx

> The only person who sees it is the installer, and even then it is hidden amongst a set of options that are, if anything, anti-privacy.

I beg to differ. The express settings list things that are mostly related to privacy, actually. It is definitely a concern of privacy which of these options and feedback features you turn on.

For instance, "help protect your PC from unsafe files and websites" refers to Microsoft's SmartScreen technology. For it to work, data has to be sent to Microsoft, and if you'd search for it, you'll find quite some criticizing the technology for the possibility of Microsoft using that data to build up profiles of their users.

Furthermore, "Lets apps give your personalized content [...]" and "Help improve Microsoft software, [...] by sending us info." are also clearly privacy options. I could go on about the others, but anything on there except for network device sharing is a privacy matter because it requires to send some data to either a first or third party.

Then, "hidden amongst a set of options" is your opinion on the subject. I'd be even inclined to agree that you may not notice it right away (can't tell, I always click customize instantly), but the DNT draft says that there are no requirements being made upon the experience of the user choice - so "hidden" is permitted, too.

> After that, any user added to the operating system is assigned the same settings as the installer, without ever seeing those dialogs.

That is semi-correct. I'd like to remind you of something extremely important here: Windows 8 is *NOT* IE10. Microsoft will make IE10 available for Windows 7. On that system, they can't ask you this way in the setup because you install it the update on an installed system. It might be this doesn't even reflect the situation in the Windows 7 version of IE10. The patch however blocks these users, too.

Secondly, this misses the point. On a single user system (which the computer I'm writing this on for instance is - hello from a single user, custom settings-installed, Windows 8 Enterprise with IE 10.0) there is no user's preference not being respected.

Lastly, I'd argue that for additional users such a system is akin to "some controlled network environments": Users I add afterwards are in this situation, basically:

> ... might impose restrictions on the use or configuration of installed user agents, such that a user might only have access to user agents with a predetermined preference enabled, the user is at least able to choose whether to make use of those user agents.

Now, they can make that choice because DNT is a per user setting that can be disabled without administrative consent.
If a "public access terminal" is such a system, then a computer I set up for relatives which then don't have administrative access is, too: Either way there's one administrative user giving a secondary one a login.

Plus, you can't rule out this wasn't a user choice of every user, even in that scenario. For instance I know my mother cares a lot about privacy. Now if I install Windows 8 on my parents' computer, I respect her choice in that I know she wants DNT to be enabled and select it on the install screen. It is still her choice. So now, the only party not respecting her preference would be the web server.


Unrelated to that, the latest editor's draft now says "Implementations of HTTP that are not under control of the user must not generate or modify a tracking preference.". In either case Apache does exactly that. I know you're arguing that removing an 'invalid DNT' is fine, but you can't reason that a single user system is entirely compliant to the DNT spec as-is. The patch removes their valid preference.

-------------------------------------------------------------------------

And again unrelated to all above, some things that personally bother me about the decision of incorporating the patch in the first place:

- If the patch was committed to the trunk on August 11th, it is chronologically impossible that the vote took place with the situation the RTM is in recapitulated. The first public availability (the evaluation, MSDN and TechNet subscriber access) was on August 15th. Maybe some of its members have had access to Windows 8 before using OEM channels, but I strongly doubt that anything close to a majority had such. Judging a product based on a pre-release on that might be subject to change is just not professional.

- It's true that the Windows 8 Release Preview (not the Consumer Preview or the RTM) is clearly in violation with the latest editor's draft. The latest public draft is however the Working Draft - which doesn't explicitly forbid what the Windows 8 Release Preview does.

The configuration lines for Do Not Track in Internet Explorer 10 have been commented out. Some problems remain: the accompanying description is inaccurate and the commented-out lines both are extraneous and encourage potential noncompliance with the W3C standard. That said, since the primary substantive issue has been resolved, it seems appropriate to mark this bug as RESOLVED FIXED.