Subject: Comments on draft-ietf-httpbis-content-disp

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 01:30:26 -0700

== Disclaimers ==

1) I'm aware that there are more implementors of user agents than
browsers.  I'm not interested in being reminded of that fact.  Browser
user agents, however, are one important group of user agents.

2) I'm aware that this working group does not share my perspective on
what constitutes a useful specification for user agent implementors.
I'm not interested in discussing whether the level of precision I'm
requesting is valuable.

3) I'm aware that this document reflects business-as-usual in the
IETF.  My position is that business-as-usual does not meet the needs
of browser user agent implementors, largely because browser user agent
implements have been effectively absent from the IETF process for the
better part of a decade.

== Comments ==

The comments below are relative to
http://tools.ietf.org/html/draft-ietf-httpbis-content-disp-03, which
is the most recent version I could find on the IETF web site.

http://tools.ietf.org/html/draft-ietf-httpbis-content-disp-03#section-3.1

This section defines a grammar for the Content-Disposition header
field.  However, the document does not define how a user agent should
interpret Content-Disposition header fields that do not conform to
this grammar.  To foster interoperability between user agent
implementations, the document should define how user agents are to
process every sequence of bytes they could receive in a
Content-Disposition header field.

=> Parameter names MUST NOT be repeated.

The document should not phrase normative requirements in the passive
voice.  Instead, the document should make clear which protocol
partipants are bound by each requirement.  For example, this
requirement probably should read "servers MUST NOT generate
Content-Disposition header field values with multiple instances of the
same parameter name."

=> a header field value with multiple instances of the same parameter
SHOULD be treated as invalid.

Similarly, this requirement probably should read "user agents SHOULD
treat a header field value with multiple instances of the same
paramater as invalid."  Furthermore, the document should define what
treating a header field value as invalid means.  Presumably the author
intends that user agents ought to ignore such header field values.
I'm skeptical that is the optimum behavior for user agents.  I would
have expected user agents to either use the first or the last instance
of each paramater.

http://tools.ietf.org/html/draft-ietf-httpbis-content-disp-03#section-3.2

This section does not define how user agents ought to process header
field values with multiple disposition types.  According to this test
case <http://greenbytes.de/tech/tc2231/#attandinline2>, user agents
MUST use the first disposition type.

http://tools.ietf.org/html/draft-ietf-httpbis-content-disp-03#section-3.3

This section provides very little guidance about how to extract a file
name from the filename parameter.  For example, it fails to instruct
the user agent about how to handle the following test cases:

http://greenbytes.de/tech/tc2231/#attwithasciifnescapedquote
http://greenbytes.de/tech/tc2231/#attwithasciifilenamenqws
http://greenbytes.de/tech/tc2231/#attwithutf8fnplain
http://greenbytes.de/tech/tc2231/#attwithfnrawpctenca
http://greenbytes.de/tech/tc2231/#attwith2filenames
http://greenbytes.de/tech/tc2231/#attfnbrokentoken
http://greenbytes.de/tech/tc2231/#attbrokenquotedfn

In particular, this document should define an algorithm that takes as
input a sequence of bytes obtained by parsing the Content-Disposition
header field value and returns a sequence of characters which is the
file name requested by the server.  The user agent, of course, can
then treat this requested file name as advisory (e.g., altering the
file name according to platform-specific constraints).

The document defines filename* by referring to RFC5987, but RFC5987
does not define a precise algorithm for computing the file name from a
sequence of input bytes.

http://code.google.com/p/chromium/issues/detail?id=57830

Jungshik Shin writes:

[[
As for RFC 5987, I'm aware that it's a profile of RFC 2231 (it's good
that it's simpler than the full RFC 2231), but I wrote that it's
unnecessarily 'complex' and not many web servers would adopt that
anytime soon. That's why I advocated a much simpler approach of using
(percent-encoded) UTF-8. I'm aware that it has its own share of
issues, but I suspect that it's got a better chance of being adopted
by web servers.
]]

I agree with his assessment.  We should simply use percent-encoded
UTF-8 instead of letting the server specify whatever crazy encoding it
dreams up.  Also, we should remove the language tagging facility
because it is gratuitous.

http://tools.ietf.org/html/draft-ietf-httpbis-content-disp-03#appendix-C.2

As far as I can tell, this is actually the biggest interoperability
problem with the Content-Disposition header field.  Unfortunately,
this document does nothing to resolve this issue.  I recommend that
this document take a position with respect to how to handle
percent-encoded values in the filename parameter.  Specifically, I
recommend that the document instruct user agents to decode percent
encoded values using the user's preferred encoding.  Yes, that's ugly,
but it's the way Content-Disposition works in the real world and the
most likely requirement to actually be implemented by user agents.

At a higher level, this document does not define the
Content-Disposition header field in sufficient detail for user agent
implements to implement the header field in a manner that
interoperates with each other or with existing servers.  Specifically,
the document does not define an algorithm for parsing the
Content-Disposition header field value, nor does it define an
algorithm for computing the requested file name from a parsed header
field value.  The document does not tackle the biggest
interoperability issue with current user agents.

In short, this document does not address the needs of browser user
agent implementors.  This objection can be resolved in two ways:

1) The document can be modified to precisely define the behavior of user agents.

2) The document can be modified to clearly state that it defines a
profile of the Content-Disposition header field for use by servers
(with the definition of the user agent behavior to follow in some
future document).

Adam

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 1 Nov 2010 06:34:13 -0600

> user agents, however, are one important group of user agents.

I'm not interested in having this discussion while ignoring this fact.
Your objection seems to be that the document isn't specifically
tailored to the needs of one and only one class of user-agent.

> I'm not interested in discussing whether the level of precision I'm

But it's perfectly reasonable to discuss whether or not the level of
precision you're after is appropriate to the scope of the document, or
should be included as part of some other document specifying browser
conformance to HTTP.

> implements have been effectively absent from the IETF process for the

Personally, I'm not interested in speculation as to why browser
concerns haven't been considered paramount, the fact is that they
aren't, and I cannot support any changes to the spec which turn HTTP
into a document dictating how components should behave, rather than a
docuemnt which defines messaging semantics between connectors.

> Content-Disposition header field.

No, this would make the document noncompliant with MIME parsing rules,
for the sake of an edge case.  The document needs do no more than it
does, which is define what's standardized, not provide instructions for
handling nonstandardized syntax.  If parsing nonstandardized syntax in
a certain way is a requirement for browser interoperability, then it's
perfectly reasonable to draft some other document for that purpose.

> Content-Disposition header field values with multiple instances of the

C-D headers are generated by *senders* for consumption by *recipients*,
therefore it would be incorrect and confusing to limit the document to
stating "user-agent" or "server" because either may be sender or
recipient.  The existing wording is clear as can be, and does not need
to be made more confusing for the sake of edge-case browser concerns.

> paramater as invalid."

But that makes no sense, if the user-agent is the sender and the server
the recipient.

>
> Furthermore, the document should define what treating a header field
> value as invalid means.  Presumably the author intends that user
> agents ought to ignore such header field values. I'm skeptical that
> is the optimum behavior for user agents.  I would have expected user
> agents to either use the first or the last instance of each paramater.
> 

Once again, this is completely out-of-scope to what HTTP defines.  What
the document says is that the messaging semantics of multiple instances
of the same parameter is undefined between connectors.  What a
component chooses to do with that undefined header, is up to the
component and is not bound by HTTP.  Perhaps a document that's
specifically targeted at user-agent concerns in this regard, is called
for.  Changing how HTTP is drafted, to be that document, is not called
for.

> most likely requirement to actually be implemented by user agents.

No, that's how certain browsers handle an extreme edge-case which was
never within the bounds of any spec, breaks from MIME header syntax,
and enforces multiple parsing models for HTTP headers -- which would
result in a horrific spec if such wording changes were to be adopted.
Again I point out, that the concern here is how components behave and
has nothing to do with the semantics of messaging between connectors,
therefore some other specification tailored to the needs of browsers is
far preferable than making incomprehensible changes to the protocol in
order to dictate user-agent behavior based on the Asian market concerns
of browsers.

> interoperability issue with current user agents.

-1

This has already been discussed, you're simply re-stating your prior
objections without addressing any of the concerns previously stated
regarding your objections.  The only thing it doesn't do is give a
detailed parsing model which magically accounts for all possible
borking of a clearly-readable spec.  But what you suggest, is what I
consider to be going down the path of incomprehensibly-written specs
(like browser vendors have accomplished with HTML) which may only be
understood in the context of what vendors have implemented -- the
results would be *useless* for anyone else's purpose, and wouldn't
likely be followed due to the uncalled-for, incomprehensible complexity.

> (with the definition of the user agent behavior to follow in some

3) browser vendors can create a document for HTTP interoperability
which addresses all these concerns about component behavior, leaving
the HTTP spec to do only what it's always done, which is define
standardized semantics for messaging between connectors *without*
dictating user-agent behavior.

I vote for 3).

-Eric

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 1 Nov 2010 08:06:59 -0600

> I vote for 3).

Of course I don't have a vote, but at least in this WG I have a say,
whereas there appears to be no point in attempting to have any say in
the HTML WG.  I think I'm speaking for a lot of developers, though,
with my opposition to the entire notion of unapproachable spec
authoring.  And in hoping that IETF doesn't follow w3c down this
incomprehensible authoring path.

Aside from a brief sabbatical, I've been a jobbing Web developer since
1994.  When I learned HTML in late 1993, I had a 2400 baud Compuserve
account, which I used to download the Mosaic beta so I could get
started, even without HTTP or even IP.  I also downloaded lots of RFCs
from the Internet to my Compuserve e-mail, this was how I taught myself 
the ISP business, leasing T1s through the Colorado Internet Cooperative
Association as member #30 (anyone who knows Barb from USENIX can check
my bona-fides, there are other ways as well).

Until switching to New York Internet in 2003, I was my own webhost,
having been in that business until 2000.  Since then, I've been pursuing
my own ends at my own speed, but I do still develop and maintain a few
commercial websites, and pick up plenty of consulting through word-of-
mouth.  If you know where to look, you can see that I've interacted with
hundreds of jobbing developers over the years, and that's just what's
public.  I consider myself more business-oriented than technical, so I
also interact with entrepreneurs and business-incubator types, and have
done many quick-fixes to websites in that context as well.

What I've been teaching other developers for many years now, is the
Zeldman and Meyer books, and King's "Speed Up Your Site," all of which
are oriented around following generally well-written, approachable
standards which are comprehensible to the average developer.  My
personal experience with those standards, as well as plenty of other
RFCs, has been to watch the Internet and the Web flourish as a result.
No stronger proof-of-concept is to be found for how RFCs are drafted,
certainly there is no track record of successful protocols defined as
bitwise parsing models with strict component conformance requirements.

I'm not opposed to change in general, but I am opposed to specifically
changing the way specs are written to make them both incomprehensible,
as well as unapproachable to the point where I won't be able to teach
anyone what they say, or convince anyone to follow them instead of just
hacking together something that works.  My opposition to Adam's
proposed changes to HTTP should be considered a plea to the IETF from
down here in the trenches, not to change what's been proven not only to
work, but also to be teachable to the average jobbing developer.

I know of few developers who comprehend how HTML 5 and Web Sockets are
drafted, and even fewer who support drafting specs in such fashion
(which, as I see it, is exactly what's being proposed for HTTP, starting
with C-D).  It took enough effort over the years, evangelizing the
Zeldman and Meyer books, to get folks to even read the damn specs --
but read them, they did.  Had those specs been incomprehensible to the
point of being unapproachable to the average developer, this effort
would've failed, and I doubt we'd have the interoperability we see
between browsers today, as a result of everyday developers filing bug
reports against browsers when they've deviated from those specs.
Browser vendors are a little too quick to pat themselves on the back
for their ACID-3 compliance, without crediting their users' ability to
understand the same specs and pressure them to comply, if you ask me.

Defining custom parsing models on a header-by-header basis for the sake
of "interoperability" is, IMO, a logical fallacy.  Please don't take
HTTP down this path, it must be understood by a far wider audience than
just those who implement browsers, or even servers.  Be clear about how
to conform, instead of authoring volumes about how to deal with non-
conformity.  HTTP is a big-enough spec as it is.  Don't turn it into
something I can barely comprehend, and couldn't possibly teach.  The
Web succeeded because it *didn't* require a degree in Computer Science
to decipher the specs.

-Eric

---

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 11:04:24 -0700

> Web succeeded because it *didn't* require a degree in Computer Science

It's clear from this discussion that you're most familiar with the
concerns of server operators, which I imagine is the case for most
participants in this working group.  I appreciate that server
operators desire specifications that can read by a large number of
people because there are a large number of server operators.  I'm not
requesting that any of that information in the document be removed or
have its presentation altered.  I'm requesting that the document
either clarify that it is intended for that specific audience or that
more information be added to the document so that it is useful to
another audience.

For example, the approach we took in the cookie spec was to have two
descriptions of the protocol, one for server operators and one for
user agent implementors.  The description for server operators is akin
to what you see in this document and in many IETF documents, whereas
the description for user agent implementors contains all the
information you need to actually implement a user agent.  I'd expect
someone such as yourself and those you teach to be interested in the
first description, which I hope is approachable.  The second
description, however, is very useful for the handful people in the
world who need to implement cookies in browser user agents.

We are perfectly capable, of course, of ignoring the IETF and forming
our own standards body to write specifications of HTTP,
Content-Disposition, URLs, etc, that meet our needs, as you suggest.
There are certainly folks who would prefer that we went down that
road.  However, I'd rather we were more engaged in the IETF process,
but that means influencing the IETF to produce documents that are
useful to us, which is the goal of my previous message.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 01 Nov 2010 19:10:04 +0100

>I'm not interested in discussing whether the level of precision I'm

If you are not interested in discussing these points then simply don't
do so, but please do not attempt to discourage others discussing them;
this is an IETF mailing list where we seek to resolve issues through
open discourse and rough consensus.

>implements have been effectively absent from the IETF process for the

As I recall the previous discussion, most other participants in this
Working Group find it more important to focus on real-life problems.
That's http://code.google.com/p/chromium/issues/detail?id=52577#c1
not an uncommon position among actual browser developers either, to
pick the first Chromium bug listed on Julian's page.

>Content-Disposition header field.

There is no basis for interpreting Content-Disposition header values
beyond what is defined in the draft; interpreting headers beyond what
is defined in the draft would be a proprietary extension of the pro-
tocol. Browser developers should not add their proprietary extensions
to standard protocols, that really goes without saying.

I propose browser developers remove their proprietary extensions from
their implementations and publish whatever problems they encounter in
doing so, so the protocol can be revised accordingly if needed. That's
something only the browser developers are properly equipped to do, we
would just risk specifying something that they find they cannot or do
not care to implement if it is not based on their factual findings.

Note in particular that we cannot assume, because something works in a
certain way in some browsers, that it is safe to require that; it may
very well be that the browsers that do something else cannot be up-
dated to match the behavior due to servers doing user agent sniffing,
in which case mandating the particular behavior was pointless and mis-
leading.

>Content-Disposition header field values with multiple instances of the

It does not matter how a Content-Disposition header came to have more
than one parameter using the same name, as such it would be incorrect
to phrase this as a requirement for a particular actor.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 1 Nov 2010 12:37:10 -0600

Adam Barth wrote:
> participants in this working group.
>

No, I have plenty of experience dealing with user agents, and helping
others with their XHR scripting or Xforms in terms of HTTP.  Of probably
a thousand conversations regarding HTTP, it's probably a 60/40 split in
favor of server-side development, but not once in 17 years have I ever
discussed HTTP in terms of developing a browser.  Browsers may be
ubiquitous, but in terms of those who need to understand HTTP, browser
developers are, in reality, an edge case.

>
> I appreciate that server operators desire specifications that can
> read by a large number of people because there are a large number of
> server operators.
>

Again, I've coded or consulted on a variety of non-browser user agents,
as well as client-side development within browsers.  You can't really
dismiss my concerns of approachability of the spec as something which
is only of concern to server developers.  RFCs need to continue to be
accessible to all, because there's no evidence the Internet or the Web
would even exist today if they hadn't been from the start.  Don't fix
what isn't broken, or gamble the future on a wholly unproven style of
authoring RFCs.

>
> I'm not requesting that any of that information in the document be
> removed or have its presentation altered.  I'm requesting that the
> document either clarify that it is intended for that specific
> audience or that more information be added to the document so that it
> is useful to another audience.
> 

The nature of your request is to change the document to break with MIME
syntax re-used by HTML headers, in a way which breaks back-compatibility
for anyone who correctly assumed '%' would be taken literally, and
claiming that it's unusable otherwise, to make it *only* usable to
those coding browsers, while blithely ignoring that C-D may be used by
HTTP-based package management systems having nothing to do with
browsing the Web.  I see nothing wrong with approving C-D as an RFC as
worded, or how it's only relevant to the server side.  As I've stated
before, you haven't made a compelling case to hold up this draft for
being written like any other RFC I was able to teach myself the
Internet by reading.

> useful to us, which is the goal of my previous message.

Please don't twist my words into a strawman.  Actually, the standards
effort I suggest would be right up the w3c's alley, and wouldn't come
anywhere near re-writing the HTTP spec, even C-D.  The problem is, the
HTTP spec can't assume that "user-agent" implies rendering content.  In
fact, many user agents (googlebot) only interpret content.  Many of the
issues you're suggesting HTTP be changed to account for, simply don't
apply to user-agents *unless* they're rendering content.

But that's a separate concern from the semantics of messaging between
connectors, and one worthy of another specification effort catering
specifically to that small minority of those implementers actually
developing browsers or otherwise dealing with the rendering rather than
the interpretation of payloads.  There's really no need to conflate the
term "user-agent" with "browser" when the notion of interpreting vs.
rendering is external to HTTP entirely, because while the result may
please browser vendors it makes the spec incomprehensible except as
understood within that context, which is unapproachable to the vast
majority of those who need to be able to understand the HTTP spec.

-Eric

---

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 12:20:11 -0700

> being written like any other RFC I was able to teach myself the

Do you expect browser user agents to change their behavior w.r.t. the
% character in filename parameter in Content-Disposition header
fields?  I don't see an evidence that they will.  It's disingenuous to
claim this document specifies the Content-Disposition header field
when it ignores how that header field is processed by user agents used
by hundreds of millions of users.

> apply to user-agents *unless* they're rendering content.

I don't see a connection between the ability to compute the requested
file name from a Content-Disposition header field value and rendering
content.  Presumably the requested file name is part of the semantics
of this header field value.  Shouldn't the specification define what
that file name the header field requests?

> majority of those who need to be able to understand the HTTP spec.

It sounds like you're advocating for resolution (2), which I think is fine.

Adam

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 1 Nov 2010 13:55:49 -0600

Adam Barth wrote:
> fields?  I don't see an evidence that they will.
>

No, of course not.  Nor would I expect a package-management system to
start decoding % if the spec is changed based on what browsers get
wrong.

>
> It's disingenuous to claim this document specifies the
> Content-Disposition header field when it ignores how that header
> field is processed by user agents used by hundreds of millions of
> users.
> 

No, what's disingenuous is to tell all the folks who correctly followed
MIME syntax in their existing C-D implementations that they are in
error, based on an edge case where a *minority* of implementations
chose to disregard proper MIME syntax.  Granted browsers have hundreds
of millions of users, but that handwaving doesn't change the fact that
systems based on nonstandard %-handling are a statistical blip.

Just because browsers do this, doesn't mean they should, IOW +1 to
Bjoern's comment:

"Note in particular that we cannot assume, because something works in a
leading."

There's nothing preventing browser vendors from agreeing to %-decode or
even standardizing that *interpretation* of C-D.  The point is, that if
it's in another spec it has absolutely no bearing on how C-D is defined,
avoiding a break with backward compatibility for all user-agents.  That
"other spec" would only affect interpretation of C-D after parsing,
allowing that parsing to be compliant with the current draft, without
requiring the scope of HTTPbis to be changed to define non-MIME-
compliant header parsing.

> content.

There is none.  But there are enough cases where it's true, to justify
a browser-conformance document.  Such a document can also choose to
specify anything else particular to browsers, such as deliberately-
borked C-D parsing which may break if required for HTTP-based package-
management systems, for example.

>
> Presumably the requested file name is part of the semantics of this
> header field value.  Shouldn't the specification define what that
> file name the header field requests?
> 

No, it shouldn't, the user-agent has final say in what filename to use,
based on the filesystem the OS is using, which can't be known by the
sender, so the sender can only suggest a filename.  Which means that as-
written, the spec allows a component to decide to %-decode without it
being mandated that all user-agents ought to do so, because that's how
Microsoft did it...

> It sounds like you're advocating for resolution (2), which I think is

No, I'm most definitely advocating the creation of a document specific
to the concerns of implementing HTTP on those user-agents which render
content, in a way that's interoperable between major browsers, and
keeping HTTP the way it's always been, which doesn't assume user-agent=
browser nor require a degree in computer science to be understood by
the lay developer.

-Eric

---

From: Ian Hickson <[email-masked]>
Date: Mon, 1 Nov 2010 20:13:34 +0000 (UTC)

On Mon, 1 Nov 2010, Eric J. Bowman wrote:
>
> In fact, many user agents (googlebot) only interpret content.  Many of 
> the issues you're suggesting HTTP be changed to account for, simply 
> don't apply to user-agents *unless* they're rendering content.

As someone who works for Google, I assure you that we are interested in 
seeing strict client-side conformance requirements, including those that 
describe required error handling, in the HTTP specification. We want our 
crawling infrastructure to interoperate with Web browsers, and currently 
to do so we have to spend significant resources reverse-engineering those 
browsers. It would be better for us, as well as fostering improved 
competition in our space, if the specification could just define all this 
in detail so we didn't have to reverse-engineer anything.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 1 Nov 2010 15:40:49 -0600

Ian Hickson wrote:
> 
> As someone who works for Google, I assure you that we are interested
> in seeing strict client-side conformance requirements, including
> those that describe required error handling, in the HTTP
> specification. We want our crawling infrastructure to interoperate
> with Web browsers, and currently to do so we have to spend
> significant resources reverse-engineering those browsers. It would be
> better for us, as well as fostering improved competition in our
> space, if the specification could just define all this in detail so
> we didn't have to reverse-engineer anything.
> 

OK, point taken.  But, the argument remains one of the appropriate
scope of HTTP.  There are radically different classes of user-agents
with radically different concerns.  Browsers (and googlebot, fine)
represent only one class of user-agent concerns.  The protocol can't be
tailored to the needs of any class of user-agent, but must remain
agnostic.  In REST, this is known as the layered-system constraint --
messaging between connectors is a separate concern from interpretation
of those messages by components -- and this architectural principle is
reflected in the design of HTTP (and has been, since long before said
constraint was identified, described, or recognized as desirable
through extensive peer review).

HTTP defines the semantics of messaging between connectors.  Where the
handling concerns of an error message vary from one class of user-agent
to another, is the components' interpretation of the status reported by
the connectors.  Thus, implementation details of components are not in-
scope to HTTP -- the protocol design either has to account for handling
by every conceivable class of user-agent, or draw a line at what meaning
the status codes intend to convey.  It is appropriate to draw this line,
rather than convert HTTP into a browser-specific protocol which
sacrifices readability by jobbing developers, while becoming oblivious
to the needs of other classes of user-agent by focusing on what's best
for the browser/search markets.

I'm not dismissive of your concerns, just saying they are not
appropriate to the scope of this protocol, which must also take into
consideration such diverse uses of HTTP as package-management systems
where the error *reporting* requirements are not the same as for
desktop browsers or even crawlers, even though the semantics of the
status codes are invariant across all user-agent classes.  Where there
are security concerns requiring particular handling by all user-agents,
I'm in favor of adopting solutions proven out in existing browsers.
What I'm opposed to, specifically, is basing the protocol on any single
class of component concerns, when its scope should be limited only to
the semantics of messaging between connectors.

There are valid reasons for, and apparently the desire to, create an
interoperability specification based on browsers.  But I see no reason
why such work would or should change how HTTP is parsed, or its
semantics defined.  What a component does with messages is up to the
component, not the protocol.  Which is why I see these requirements as
being out-of-scope to HTTP.  I'm not joking at all when I suggest a w3c
HOM activity devoted to documenting interoperable implementation details
for any class of user-agent for which they're deemed relevant.  I just
can't imagine any such document applying to *all* classes of user-agent,
which is why I'm opposed to expanding the scope of HTTPbis to attempt
just that, which is exactly the slope we start slipping down by speccing
non-MIME-compliant C-D syntax based on, at best, a 1% use-case.

-Eric

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 02 Nov 2010 01:32:01 +0100

>by hundreds of millions of users.

So let's see, rumour has it Firefox, Opera, Safari, and Konqueror have
more than 50% of the browser market here in Germany, and none of them
apparently interpret the percent sign as indicating percent-encoding,
and you see no evidence that any of those browsers will change how they
interpret the character; so you recommend, instead of specifying the
sane behavior implemented by the majority of browsers, and presumably
the overwhelming majority of non-browser applications, the draft should
require the unsound behavior of the minority of browsers where you do
not have any reason to believe that it will be adopted by anyone else.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 2 Nov 2010 11:43:15 +1100

Adam,

Thanks.

At a high level, I'd like to use this discussion to resolve issue #186:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/186
in that once we figure out the depth of error-handling that's appropriate in this spec, we should be able to apply that to the HTTP spec overall. 

Some specific thoughts below. Note that I've opened a number of tickets here; please focus follow-up discussion on specific tickets (e.g., calling out the ticket number in the Subject line).

> implements have been effectively absent from the IETF process for the

As Bjoern has noted, this isn't the best way to start a discussion, but here we are. We will most certainly discuss the implications of any changes on non-browser UAs, and will also discuss the appropriate level of precision to specify. You can choose to participate in those discussions or not, of course.

I'll leave your statements about the IETF as a whole to stand on their own merits.

> is the most recent version I could find on the IETF web site.

Yes; that's the version that is under WGLC.

> Content-Disposition header field values with multiple instances of the

I think this is largely editorial feedback; it probably isn't appropriate to say 'servers..' but something like

Senders MUST NOT generate C-D header field values with multiple instances of the same parameter name.

Ticket: 
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/258

> have expected user agents to either use the first or the last instance

Ticket:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/259

Note that it may be resolved by indicating that 'treat as invalid' is specific to the application at hand. As such, I'd like initial discussion of this in the WG to focus on:
 a) use cases: how different implementations / applications may want to have different notions of 'invalid' (or not), and
 b) security: what the security impact of having different notions of 'invalid' here may be, and
 c) interoperability: likewise, the interop impact.

> MUST use the first disposition type.

Ticket:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/260

> http://greenbytes.de/tech/tc2231/#attbrokenquotedfn

Ticket:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/261

> file name requested by the server.

I'm treating that as editorial advice.

> does not define a precise algorithm for computing the file name from a

If you have specific issues (rather than just a general desire for an algorithm), please raise them (e.g., as in #261, although even more specificity would be appreciated).

> dreams up.

The 'crazy encoding' you refer to isn't dreamed up by the Web server. Regardless, if you'd like to pursue this path, you need to make a proposal and do the legwork to show that implementers will support it, with a reasonable backwards-compatibility story.

> Also, we should remove the language tagging facility

Can you say a bit more here? We can open an issue for this, but your reasoning (beyond "it's gratuitous") isn't clear.

> most likely requirement to actually be implemented by user agents.

Could you expand upon this a bit more? E.g., are you saying that after the 5987 encoding is removed, the resulting string should be percent-decoded? Or that the filename (no *) parameter should be percent-decoded? Both?

Raising as a placeholder:
  http://trac.tools.ietf.org/wg/httpbis/trac/ticket/262

I suspect that this issue is going to be similar to #259; i.e., the answer may be different for different implementers. As such, we should try to figure out if that's the case (and why) first.

> agent implementors.  This objection can be resolved in two ways:

This isn't the W3C, we don't have objections. Make arguments and they'll be judged on technical merit, based upon the rough consensus of implementers and their running code. 

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 17:47:00 -0700

> not have any reason to believe that it will be adopted by anyone else.

My understanding is that the %-encoding behavior is needed to compete
in the Asian market, not the German market.

Adam

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 02 Nov 2010 02:58:16 +0100

>in the Asian market, not the German market.

Yes, and once you can tell us of popular asian web sites where users
download a lot of files that get garbled file names in Firefox et al.
but not in Internet Explorer and Chrome, and that adopting their be-
havior does not cause problems on other popular web sites for them,
then you can be sure that Firefox and Opera and Safari and many other
applications will be updated to match the behavior of Internet Ex-
plorer, and the HTTPbis Working Group will document that behavior in
their deliverables; it'd be demonstratively needed, after all.

Without that, the triage teams for those applications will find more
important bugs to fix ad infinitum, meaning the applications will not
be updated, and any specification contrary to their behavior will de-
scribe a fiction, which is worse than not mandating a particular hand-
ling for percent signs (or the many more, far less important, cases
you want the specification to address aswell). Philosophical debates
about the abstract needs of absent browser developers are nice and all,
but getting application developers to change their code requires good
evidence; that you are not providing.

This is an IETF Working Group; rough consensus is clearly not on your
side for the moment, so show us the running code. Really do, I'd like
nothing more than an IETF specification that tells you all the things
that you really need to know. Don't expect us to act on your hearsay.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 20:09:04 -0700

> in that once we figure out the depth of error-handling that's appropriate in this spec, we should be able to apply that to the HTTP spec overall.

That's a pretty broad scope, but we can try.

> Some specific thoughts below. Note that I've opened a number of tickets here; please focus follow-up discussion on specific tickets (e.g., calling out the ticket number in the Subject line).

Where you've assigned ticket numbers, I've followed up in separate threads.

> The 'crazy encoding' you refer to isn't dreamed up by the Web server. Regardless, if you'd like to pursue this path, you need to make a proposal and do the legwork to show that implementers will su…

Has Julian done the legwork to show that implementers will support his
draft?  All the bugs I've seen that he's filed with Chrome seem to end
up with Jungshik Shin refusing to implement his proposal.

> Can you say a bit more here? We can open an issue for this, but your reasoning (beyond "it's gratuitous") isn't clear.

Put another way: what's the use case for including a language tag?

> This isn't the W3C, we don't have objections. Make arguments and they'll be judged on technical merit, based upon the rough consensus of implementers and their running code.

Feel free to read the word "objection" in the usual english sense and
not in the technical W3C sense.

All the issues I've raised in my email are based on code that's been
running for over a decade.

Adam

---

From: Adam Barth <[email-masked]>
Date: Mon, 1 Nov 2010 20:14:05 -0700

> their deliverables; it'd be demonstratively needed, after all.

This seems backwards to me.  You're saying that once we solve the
interoperability problem, then we can write a specification to
document that fact?  Is not the purpose of a specification to make it
easier for folks to write interoperable implementations?

> evidence; that you are not providing.

If no one is interested in interoperability, then I suggest we turn
out attention to other matters where our standardization efforts can
be productive.

> that you really need to know. Don't expect us to act on your hearsay.

Here's some running code:

http://www.microsoft.com/windows/Internet-explorer/
http://www.google.com/chrome/

Adam

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 02 Nov 2010 13:52:02 +0900

> UTF-8 instead of letting the server specify whatever crazy encoding it

For the record, I also agree with this. The simpler, more general, and 
more straightforward the encoding is, the better. This can be specified 
as an additional layer on top of header processing, if necessary.

> because it is gratuitous.

I also agree with this. There is currently no OS or any related facility 
that 'knows' in any way in what language its filenames are. There is 
also no way for a user to enter that information, in the interfaces I 
know. There is also no practice of language negotiation for filenames 
(there is language negotiation for different language versions of 
content, where as a result the filenames may also be different, but 
that's a different thing).

> most likely requirement to actually be implemented by user agents.

Well, you say 'ugly', but in this case, it's equivalent to "does *NOT* 
work". [Users don't prefer encodings, it's the user's system's preferred 
encoding.] Preferred encodings on the receiving side, and used encodings 
on the server side, differ, and the result of that is garbage. I'm not 
sure recommending to produce garbage is a good idea at all.

As for the wording of Appendix C.2, even if it stays more or less as it 
is, it is confusing. It first says that some user agents accept UTF-8, 
then it says that the first user agent to implement this (i.e. UTF-8) 
used the local encoding (i.e. NOT UTF-8).

Regards,    Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 2 Nov 2010 18:14:00 +1100

> up with Jungshik Shin refusing to implement his proposal.

Yes; he's written a draft, done an interoperability matrix and associated testing, justified his design decisions, and followed up with implementers. 

That's "the legwork."

The bugs that I see Julian and Jungshik interacting over seem to be characterised by Julian answering his questions; I don't see outright refusal at all.

> Put another way: what's the use case for including a language tag?

If for no other reason, it's included now because it's in 5987, where it's included (AIUI) because some headers may have need for it.

Are you suggesting that receivers MAY ignore it, or that senders MUST NOT send it?

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 00:27:34 -0700

> The bugs that I see Julian and Jungshik interacting over seem to be characterised by Julian answering his questions; I don't see outright refusal at all.

It's true that the bugs are getting marked Mstone-X rather than
WONTFIX.  I can follow up with Jungshik in person to see figure out
what's going on.  Certainly implementing filename* seems valuable as
it would give folks a predictable mechanism for using non-ASCII
characters in their requested file names.

> Are you suggesting that receivers MAY ignore it, or that senders MUST NOT send it?

I'm suggesting we should use a syntax that doesn't have a slot for
language.  For example, simply %-encoded UTF8 would seem to address
the real use case here.  If 5987 is squatting on filename*, then we
can call it awesome-filename or whatever.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 09:43:36 +0100

>> case<http://greenbytes.de/tech/tc2231/#attandinline2>, user agents

That's incorrect.

Have a look at <http://greenbytes.de/tech/tc2231/#attandinline>; in this 
case IE picks the second one.

There is no interop here today, and this doesn't seem to be a problem in 
practice.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 09:49:35 +0100

> I'm treating that as editorial advice.

I'm not sure where to start :-)

Like it or not, the draft relies on the ABNF for parameter parsing, and 
delegates the filename* format to RFC 5987.

For valid header field instances this is supposed to be sufficient. If 
it's not, let's focus on details. Would it be helpful if I went through 
the valid header field instances in these test cases and explained how 
they parse?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 09:54:21 +0100

Mark,

thanks for categorizing.

Before I get to the details, I'd like to emphasize that missing 
interoperability for invalid headers is *not* the main problem at all.

The *real* problem we have today is missing interop for *valid* headers, 
in particular when I18N comes into play. Before we focus on edge cases, 
we really need to get *that* improved.

> Yes; that's the version that is under WGLC.

Actually, it was -02 which *was* under WGLC, -03 resolves a few comments 
I received during LC.

> If you have specific issues (rather than just a general desire for an algorithm), please raise them (e.g., as in #261, although even more specificity would be appreciated).

I don't see a problem here.

RFC5987 uses ABNF + prose, but not an algorithm. This is actually more 
detailed than what RFC2231 said, and we have four independant 
implementations getting this right, and none getting this wrong (as far 
as I can tell).

> The 'crazy encoding' you refer to isn't dreamed up by the Web server. Regardless, if you'd like to pursue this path, you need to make a proposal and do the legwork to show that implementers will su…

Absolutely. Document the format. Align with IE (where it depends on the 
UA's locale). Document how servers can detect the right encoding when 
sending the header. Document how that affects caching/vary. And then 
explain how to get this implemented in all UAs.

> ...

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 01:55:28 -0700

On Tue, Nov 2, 2010 at 1:43 AM, Julian Reschke <[email-masked]> wrote:
>> Note that it may be resolved by indicating that 'treat as invalid' is
>> specific to the application at hand. As such, I'd like initial discussion of
>> this in the WG to focus on:
>>  a) use cases: how different implementations / applications may want to
>> have different notions of 'invalid' (or not), and

That's a pretty defeatist attitude.  Put another way, four out of five
browsers agree: we ought to use the first disposition type.

Adam

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 01:56:30 -0700

On Tue, Nov 2, 2010 at 1:49 AM, Julian Reschke <[email-masked]> wrote:
> For valid header field instances this is supposed to be sufficient. If it's
> not, let's focus on details. Would it be helpful if I went through the valid
> header field instances in these test cases and explained how they parse?

I'm more interested in the invalid header field instances.  This
document doesn't explain how to parse them, much less how to process
them.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 10:00:11 +0100

> can call it awesome-filename or whatever.

The language already is optional. It's harmless. There's no problem here.

On the other hand, inventing a new encoding for each new parameter makes 
code-reuse harder.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 10:05:24 +0100

> as an additional layer on top of header processing, if necessary.

Yes. It would be nice if we could fix HTTP/1.1, or existing headers. But 
we can't.

> that's a different thing).

It's optional, and the encoding is generic so it can be used for other 
parameters as well (such as "title" in Link:).

>> most likely requirement to actually be implemented by user agents.

a) As far as I recall, this is not true for Chrome.

b) Even if it was, it's useless for a sender unless it can find out what 
the encoding is.

c) That it's most likely to be implemented is just an opinion, so far 
the evidence is against it.

> sure recommending to produce garbage is a good idea at all.

+1

> used the local encoding (i.e. NOT UTF-8).

That's true: this sections needs to be rephrased. To get this right it 
would be valuable to get feedback from Chrome and IE what they actually do.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 10:07:06 +0100

> browsers agree: we ought to use the first disposition type.

I'm just saying that there's no unique handling today for an edge case, 
and even an invalid one.

As I said before, I'm much more interested in solving the non-edge-case 
problems first.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 12:11:11 +0100

> document doesn't explain how to parse them, much less how to process

No, it doesn't (and that's a separate discussion).

While we are at it, let me explain why 
<http://greenbytes.de/tech/tc2231/> has tests for invalid header fields 
in the first place. After all, if I don't really want to specify how 
they are processed, right?

The reason why there are there is that they serve the purpose of 
observing whether there is any kind of interop in UAs. If there was, it 
would be interesting to see whether that is by accident, or because 
existing content actually requires it.

I imagine that it is controversial how I rate the results. The idea is 
to say "pass" when the header field is ignored, "warn" when something 
happens that could be considered harmless, or "fail" when something 
serious happens (like UA crashing).

Best regards, Julian

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 2 Nov 2010 11:50:03 +0000

Mark,

> in that once we figure out the depth of error-handling that's appropriate in this spec, we should be able to apply that to the HTTP spec overall. 

One thing that occurred to me (and a trawl of the RFC series would probably throw up a few examples like rfc4647[1]) is whether the best way to slice this up is to avoid being exhaustive wrt error-handling for all UA types in the HTTP spec itself and to have a separate BCP that covers the situations that most common(/whatever criteria apply).

Something like "Observed Browser treatment of the C-D header" or "Guidelines for implementing C-D error handling" etc.

Ben

[1] It's just the first BCP I found from a quick search that looked like it might make an example (which means it's probably not a particularly great example).

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 16:24:00 +0100

>    http://trac.tools.ietf.org/wg/httpbis/trac/ticket/258

I made it say:

    Senders MUST NOT generate header field values with multiple instances
    of the same parameter name.  Recipients SHOULD treat these values as
    invalid.

(see <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1073>).

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 09:39:46 -0700

Thanks.

Adam

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 09:46:15 -0700

On Tue, Nov 2, 2010 at 4:11 AM, Julian Reschke <[email-masked]> wrote:
> While we are at it, let me explain why <http://greenbytes.de/tech/tc2231/>
> has tests for invalid header fields in the first place. After all, if I
> don't really want to specify how they are processed, right?
>
> The reason why there are there is that they serve the purpose of observing
> whether there is any kind of interop in UAs. If there was, it would be
> interesting to see whether that is by accident, or because existing content
> actually requires it.

While I appreciate the implied complement that browser vendors are
good at fixing interoperability and compatibility problems, I think
you're giving us too much credit.  For example, my understanding is
that there's a bunch of content in Asia that requires the nutty
%-decoding user-default-charset handling.  Because a number of
browsers have effectively chosen not to compete in this marketplace,
they lack this behavior.  Therefore lack of unanimity among browsers
doesn't necessarily imply that there isn't existing content that
requires the behavior.

Put another way, if all the browsers had an unlimited amount of data
at their disposal, they might make different implementation decisions.
 For example, once we gathered a metric ton of compatibility data
about what sorts of content sniffing were actually required to
compatibility render the web, we were able to align browser behavior.
Some of the common idiosyncrasies turned out to be useful, even though
they weren't unanimous.

> I imagine that it is controversial how I rate the results. The idea is to
> say "pass" when the header field is ignored, "warn" when something happens
> that could be considered harmless, or "fail" when something serious happens
> (like UA crashing).

A UA crashing certainly sounds like failure.  :)

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 17:53:09 +0100

> doesn't necessarily imply that there isn't existing content that

So none of Safari/Opera/Firefox is competing there?

Maybe those sites just gave up on IE and Chrome, and have been sending 
filename* to everybody else for a long time?

It would *really* be useful to get beyond the hearsay and actually get data.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 10:15:13 -0700

> So none of Safari/Opera/Firefox is competing there?

So, you can go to http://gs.statcounter.com/ and play with the region
selector to compare North America to Asian.  Here's what it looks like
to me:

1) Chrome and IE have greater market share in Asia than they do in
North America, where as Firefox and Safari have less.
2) Safari has much less market share in Asian than in North America.
(Opera has small market share in both markets.)
3) Chrome's market share growth in Asia is coming from largely from IE
rather than from Firefox, whereas in North America, Chrome's market
share growth is coming from both Firefox and IE.

Now, I'm sure there are any explanations for these observations, but
these observations lead me to believe that (1) Safari does not compete
in the Asian market, and (2) Chrome is better at acquiring IE
customers in Asia than Firefox is.

As we all know, web compatibility is one of the top 5 reasons users
decide which browser to use.  It seems entirely likely that the
decision to implement IE-like behavior for the Content-Disposition
header has contributed to Chrome's strong performance in the Asian
market.

> It would *really* be useful to get beyond the hearsay and actually get data.

I don't have that data off hand.  We can gather it.  It just takes
time and effort.

Adam

---

From: Karl Dubost <[email-masked]>
Date: Tue, 2 Nov 2010 13:24:38 -0400

> (Opera has small market share in both markets.)

Only on desktop.
http://gs.statcounter.com/#mobile_browser-as-monthly-200910-201010

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations & Tools, Opera Software

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 18:25:06 +0100

> I don't have that data off hand.  We can gather it.  It just takes

Understood. But I think you really need to support that claim with data.

If the data is obtained by through Chrome users we will also have get 
access to the actual site names that do this, checking whether they 
already have separate code paths for different browsers.

Also, related to issue 263, it would be great if you could find out 
whether Chrome always use UTF-8 when percent-unescaping, or tried to 
follow IE.

I know that Asian IE installations *did* not use UTF-8 unless the 
browser was configured for use of UTF-8 when *generating* URIs. It would 
be great if we could confirm what the exact condition is though.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 18:33:38 +0100

> http://gs.statcounter.com/#mobile_browser-as-monthly-200910-201010

In this particular case, we're *only* interested in desktop browsers, as 
mobile browsers do not support content-disposition: attachment anyway 
(at least that's the case for iOS Safari, right?)

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 11:09:16 -0700

> Understood. But I think you really need to support that claim with data.

We can debate who's got the burden of proof, but it doesn't really
matter.  An easier path is probably to just ask jungshik.

> If the data is obtained by through Chrome users we will also have get access
> to the actual site names that do this, checking whether they already have
> separate code paths for different browsers.

Yes.  Another issue is selection bias.

> Also, related to issue 263, it would be great if you could find out whether
> Chrome always use UTF-8 when percent-unescaping, or tried to follow IE.
>
> I know that Asian IE installations *did* not use UTF-8 unless the browser
> was configured for use of UTF-8 when *generating* URIs. It would be great if
> we could confirm what the exact condition is though.

Here's what the code says:

    // Non-ASCII string is passed through and treated as UTF-8 as long as
    // it's valid as UTF-8 and regardless of |referrer_charset|.

    // Non-ASCII/Non-UTF-8 string. Fall back to the referrer charset.

    // Non-ASCII/Non-UTF-8 string. Fall back to the native codepage.
    // TODO(jungshik): We need to set the OS default codepage
    // to a specific value before testing. On Windows, we can use
    // SetThreadLocale().

There are tests to that effect in
<http://src.chromium.org/viewvc/chrome/trunk/src/net/base/net_util_unittest.cc>,
but i haven't looked at the implementation.

If you're willing to not spec what happens to invalid header field
values, it sounds like you could spec that the filename parameter is
first %-decoded and then UTF8 decoded.  The nutty behavior appears to
only rear its ugly head when your %-encoded value isn't valid UTF8
(which you could decide what an "invalid" header field value).

Now, of course, that would still leave us with UA-sniffing code on
servers until everyone implements the spec, but that at least sounds
implementable, unlike the current document, and puts us on a path to a
better future.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 19:23:27 +0100

>      // to a specific value before testing. On Windows, we can use

Thanks for looking this up.

"referrer charset" is the page from where the request comes, right? Of 
course there's no guarantee that this will always be the same.

Also, this seems to fallback to the local codepage only for non-UTF-8 
and missing referrer? That does not match my experience with IE7.

> (which you could decide what an "invalid" header field value).

That may be true for Chrome, but certainly *was* not true for IE when I 
first encountered the problem (trust me, I *was* sending UTF-8).

> implementable, unlike the current document, and puts us on a path to a

Why do you keep saying the current document is not implementable? That's 
not helpful; the RFC 2231 encoding has three independant implementations 
(four if I'm allowed to count iCab).

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 11:36:33 -0700

On Tue, Nov 2, 2010 at 11:23 AM, Julian Reschke <[email-masked]> wrote:
>>> access
>>> whether

Charset calculations are somewhat complicated.  Roughly speaking, the
referrer charset is the charset (if any) used by the resource that
cause this request to be generated.  How that gets computed exactly is
kind of hairy, but that's the idea.

> Also, this seems to fallback to the local codepage only for non-UTF-8 and
> missing referrer? That does not match my experience with IE7.

I suspect jungshik didn't match IE exactly but made an informed
decision about what might be acceptable both w.r.t. compatibility and
w.r.t. sanity.

> first encountered the problem (trust me, I *was* sending UTF-8).

Indeed.  We don't need to slavishly copy IE.  We just need to stay
within some compatibility bound.  Usually we aim for somewhere between
99.99 and 99.999% compatibility.  That usually isn't the same as the
unanimous intersection of all browser behavior.

> Why do you keep saying the current document is not implementable? That's not
> helpful; the RFC 2231 encoding has three independant implementations (four
> if I'm allowed to count iCab).

The document has two pieces:

1) filename
2) filename*

What the document says about (1) isn't implementable by some number of
user agents.  What the document says about (2) is suboptimal, but
might be implementable if we can get jungshik on board (and whoever
the relevant decision makers are on the ie-team).

>From this discussion, it seems you care much more about (2) than about
(1).  One alternative is to remove the parts of the document that
discuss (1) since those appear to be more controversial than the parts
that discuss (2).

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 20:05:25 +0100

> unanimous intersection of all browser behavior.

I like numbers. Can we *please* get numbers about actual use, and names 
of sites that use it?

> the relevant decision makers are on the ie-team).

Ah.

So you say that the spec currently does not allow percent-unescaping 
(btw: the applicable specs never did, for the late comers to this 
discussion).

Björn already pointed out that the filename is advisory only, and that a 
UA can already do what it wants to derive a final filename (mostly due 
to security considerations and local FS conventions). But I would agree 
that this is not optimal.

Another way to address this is to recognize the problem, and suggest 
that servers that actually *want* a verbatim %-sequence in the filename 
will *have* to use the 2231/5987 encoding. If that's what's needed to 
get the missing UAs to implement filename*, I'll be more than happy to 
make that change.

> discuss (1) since those appear to be more controversial than the parts

That's an interesting thought. But I think one of the reasons for the 
missing interoperability was the confusion in the specs (is the token 
form allowed?, is RFC 2047 encoding allowed/required?, is 2231 
allowed/required?) Etc. So I really believe that a single document that 
defines C-D for HTTP is needed, and this WG has decided long ago (issue 
123) that we want to do just that.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 12:19:26 -0700

On Tue, Nov 2, 2010 at 12:05 PM, Julian Reschke <[email-masked]> wrote:
> I like numbers. Can we *please* get numbers about actual use, and names of
> sites that use it?

Feel free to gather that data.  I probably won't have time to gather
it anytime soon because I'm busy fixing color profile support for
images.

>>> not
> So you say that the spec currently does not allow percent-unescaping (btw:
> the applicable specs never did, for the late comers to this discussion).
>
> Björn already pointed out that the filename is advisory only, and that a UA
> can already do what it wants to derive a final filename (mostly due to
> security considerations and local FS conventions). But I would agree that
> this is not optimal.
>
> Another way to address this is to recognize the problem, and suggest that
> servers that actually *want* a verbatim %-sequence in the filename will
> *have* to use the 2231/5987 encoding. If that's what's needed to get the
> missing UAs to implement filename*, I'll be more than happy to make that
> change.

I think it would be fine for this document to define a profile of the
filename parameter (and really the whole Content-Disposition header
field) for use by servers.  We'd say something like "if you want your
Content-Disposition header field to be understood by most user agents,
you'll probably want to generate it according to this grammar."  We
can also say "look, user agents do some nutty things outside of this
grammar, but that's not described by this document."

Maybe we've been miscommunicating this whole time, but that's what I
meant by option (2) at the beginning of this thread.

> missing interoperability was the confusion in the specs (is the token form
> allowed?, is RFC 2047 encoding allowed/required?, is 2231 allowed/required?)
> Etc. So I really believe that a single document that defines C-D for HTTP is
> needed, and this WG has decided long ago (issue 123) that we want to do just
> that.

The problem is that's not the document you've written.  You've written
a document that contains useful information for generating a
Content-Disposition header field.  However, the document doesn't
contain enough information about how to consume the header field.  If
I were to write a new user agent tomorrow, I'd need more information
than what's contained in this document.  Because no one's written that
down for me, I'll have to go reverse engineer some existing
implementations and make some guesses, which is the status quo that's
gotten us into the interoperability mess that we're in now.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 20:25:23 +0100

> that's a different thing).

I made this <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/264>:

Proposal:

1) At the end of 3.1, add:

"Note: the format used for ext-value allows specifying a natural 
language; this is of limited use for filenames and is likely to be 
ignored by recipients."

2) In Section 4, language tagging is used in one example; modify the 
example not to use it.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 02 Nov 2010 20:30:24 +0100

> it anytime soon because I'm busy fixing color profile support for

I think it's up to the people who want to make the "nutty" encoding an 
Internet Standards Track document to provide this data.

> field) for use by servers.  We'd say something like "if you want your

It does define a syntax for valid headers, and how to process them.

Whether we call that a "profile" or not seems to make little difference.

> gotten us into the interoperability mess that we're in now.

This brings us back to the separate issue of whether we want specs to 
define handling of invalid data.

This is already tracked separately (I think), so we should close this 
issue (#261).

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 12:35:09 -0700

On Tue, Nov 2, 2010 at 12:25 PM, Julian Reschke <[email-masked]> wrote:
> "Note: the format used for ext-value allows specifying a natural language;
> this is of limited use for filenames and is likely to be ignored by
> recipients."
>
> 2) In Section 4, language tagging is used in one example; modify the example
> not to use it.

That seems like a local improvement.  We should engage with jungshik
to get his perspective on this issue.

Adam

---

From: Adam Barth <[email-masked]>
Date: Tue, 2 Nov 2010 12:37:18 -0700

On Tue, Nov 2, 2010 at 12:30 PM, Julian Reschke <[email-masked]> wrote:
>>> of

The difference is this document purports to define the semantics of
the header field, but it does not.  It provides information that's
useful for generating the header field but not about how to consume
it.

Adam

> This brings us back to the separate issue of whether we want specs to define
> handling of invalid data.
>
> This is already tracked separately (I think), so we should close this issue
> (#261).
>
> Best regards, Julian
>

---

From: "Eric J. Bowman" <[email-masked]>
Date: Tue, 2 Nov 2010 14:48:31 -0600

> header has contributed to Chrome's strong performance in the Asian

You provide no data in support of this claim.  It seems entirely likely
to me that C-D handling has no bearing whatsoever upon the Asian market
performance of browsers.  We can bandy baseless assertions about 'til
the cows come home, but if you want the document changed on this basis,
I'll have to agree with Julian that it's incumbent upon you to provide
some numbers, or please stop using this to justify your claims that the
draft language is useless as written.

-Eric

---

From: "Eric J. Bowman" <[email-masked]>
Date: Tue, 2 Nov 2010 15:07:35 -0600

Julian Reschke wrote:
> that servers that actually *want* a verbatim %-sequence in the
> filename will *have* to use the 2231/5987 encoding. If that's what's
> needed to get the missing UAs to implement filename*, I'll be more
> than happy to make that change.
> 

But what if that causes more problems than it solves?  There's no
evidence that the expectation of decoding is more common than the
expectation that % be taken literally.  It would be a shame to break
compatibility of existing sites on the basis of a vaguely-defined edge
case, particularly when the %-literal folks have a stronger case for
expecting interoperability than do the %-decode folks, who have never
had any expectation of interoperability.

"If it ain't broke, don't fix it."  I'm all for interoperable browsers,
just against penalizing folks who didn't do anything wrong, to achieve
it.

-Eric

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Nov 2010 11:28:11 +1100

On 03/11/2010, at 6:05 AM, Julian Reschke wrote:

> I like numbers. Can we *please* get numbers about actual use, and names of sites that use it?

I like numbers too, but it seems to me that getting these numbers will be difficult, as many use cases for Content-Disposition are behind firewalls or at least password-protected.

So, it may be worthwhile to look at what server-side code we do have access to; Open Source content management frameworks and other serving containers.

I've done a few below, and AFAICT no-one is internationalising their filename parameters, possibly because of the lack of interop that Julian has noted. It also may be that these products don't have wide deployment in markets that need internationalisation; does anyone know of a popular CRM / CMS in asian markets, for example?

======================

SugarCRM 6.1.0RC2, download.php:120 - 

> 		if(isset($_SERVER['HTTP_USER_AGENT']) && preg_match("/MSIE/", $_SERVER['HTTP_USER_AGENT']))
> 		{	
> 			$name = urlencode($name);
> 			$name = str_replace("+", "_", $name);
> 		}
> 
> 		header("Pragma: public");
> 		header("Cache-Control: maxage=1, post-check=0, pre-check=0");
> 		if(isset($_REQUEST['isTempFile']) && ($_REQUEST['type']=="SugarFieldImage"))
> 			header("Content-type: image");
> 		else {
> 		    header("Content-type: application/force-download");
>             header("Content-disposition: attachment; filename=\"".$name."\";");

Ignoring the abomination that is the caching headers there for a moment, you can see that they're UA-sniffing for IE and url-encoding there, but not specifying a charset (even UTF-8), so it's just encoding URL-special characters.

SugarCRM 6.1.0RC2, class.pdf.php:1922 -

>   $fileName = (isset($options['Content-Disposition'])?$options['Content-Disposition']:'file.pdf');
>   header("Content-Disposition: attachment; filename=".$fileName);

i..e, no encoding (here and in all other circumstances I could find).

I would note that Sugar *is* getting deployment in asian markets.

========================

PHP has two C-D related functions:

* http_send_content_disposition - <http://au2.php.net/manual/en/function.http-send-content-disposition.php>
which appears to boil down to <http://svn.php.net/viewvc/pecl/http/trunk/http_send_api.c?view=markup>, line 324. No attempt to encode.

* HttpResponse::setContentDisposition - <http://au2.php.net/manual/en/function.httpresponse-setcontentdisposition.php>
whose corresponding source is <http://svn.php.net/viewvc/pecl/http/trunk/http_response_object.c?view=markup>, around line 450. There isn't any attempt to encode the parameter in any way there either.

========================

Ruby on Rails' streaming ActionController allows setting Content-Disposition on send_file - <http://api.rubyonrails.org/classes/ActionController/Streaming.html#method-i-send_file>. 

The corresponding code that sets headers <http://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L110> doesn't appear to do any encoding (although I'm not familiar with the subtleties of some Ruby syntax).

========================

Django doesn't appear to treat C-D in any special way:
  http://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment

... nor does Plone:
  http://plone.org/documentation/manual/plone-community-developer-documentation/serving/http-request-and-response#content-disposition

---

From: Julian Reschke <[email-masked]>
Date: Wed, 03 Nov 2010 12:12:48 +0100

> 2) In Section 4, language tagging is used in one example; modify the

-> <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1074>:

    Furthermore note that the format used for ext-value allows specifying
    a natural language; this is of limited use for filenames and is
    likely to be ignored by recipients.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 04 Nov 2010 09:23:44 +0100

> would be valuable to get feedback from Chrome and IE what they actually do.

I made it say in 
<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1075>:

-- snip --
C.2.  Percent Encoding

    Some user agents accept percent encoded ([RFC3986], Section 2.1)
    sequences of characters.  The character encoding being used for
    decoding depends on various factors, including the encoding of the
    referring page, the user agent's locale, it's configuration, and also
    the actual value of the parameter.

    In practice, this is hard to use because those user agents that do
    not support it will display the escaped character sequence to the
    user.  For those user agents that do implement this it is difficult
    to predict what character encoding they actually expect.
-- snip --

That should be more correct, but I welcome corrections from those people 
who may be able to describe more precisely what the two UAs actually do.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 9 Nov 2010 13:13:13 +1100

I think I agree this can be closed -- we seem to have established that we don't want mandatory error handling. 

On 02/11/2010, at 7:49 PM, Julian Reschke wrote:

> Like it or not, the draft relies on the ABNF for parameter parsing, and delegates the filename* format to RFC 5987.
> 
> For valid header field instances this is supposed to be sufficient. If it's not, let's focus on details. Would it be helpful if I went through the valid header field instances in these test cases and explained how they parse?
> 
> Best regards, Julian
>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 9 Nov 2010 13:13:15 +1100

As with others, I think we have consensus that error handling isn't mandatory, thus WONTFIX. 

On 02/11/2010, at 8:07 PM, Julian Reschke wrote:

> I'm just saying that there's no unique handling today for an edge case, and even an invalid one.
> 
> As I said before, I'm much more interested in solving the non-edge-case problems first.
> 
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 09 Nov 2010 10:29:48 +0100

> As with others, I think we have consensus that error handling isn't mandatory, thus WONTFIX.

Done.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 09 Nov 2010 10:31:16 +0100

> I think I agree this can be closed -- we seem to have established that we don't want mandatory error handling.

Closed as "wontfix" (although maybe "invalid" would be more correct here).

Best regards, Julian