Subject: #305 Header ordering

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 11:12:41 -0800

Hervé made a few comments on github
(https://github.com/http2/http2-spec/issues/305) that I think needed
to be made here:

Hervé:
There are at least to ways of providing ordering between headers:

 * Using null-separated list of values, and mandating that the
ordering of the values in these lists must be preserved.

 * Relying on the emission order. The only difficulty here is that the
ordering of the headers in the reference set can not be chosen by the
sending application. However tricks (like double indexed
representation) can be used by the encoder to enforce an order.

If we are only targeting the ordering of cookies, then using
null-separated list of values is sufficient.

 * It stays in the main HTTP/2.0 spec, therefore is not dependent of
the header compression layer.

 * It allows removing from HPACK the emission ordering constraints.
<<<

On the first, this contradicts a previous decision.  Cookies need to
be decomposed into pieces to get compression efficiency
(https://github.com/http2/http2-spec/issues/292).

The actual ordering requirements are very narrow:
http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-25#section-3.2.2

I see three options:

1. A null-delimiter and collapsing all header field instances for the
same name into the same value.

2. A requirement on the compression to preserve order (for fields with
the same name).  The best part about this is that it isn't that
difficult to achieve, because the only non-deterministic part of the
decoder is the reference set emission.  Make that deterministic (emit
in same order as last time; emit from highest table index to lowest)
and we avoid the need for null-delimited sequences altogether.

An encoder then follows an algorithm where it forces emission of
header fields as they appear.  Items can be left in the reference set
if they are in the same order as last time (which requires a little
bit of accounting to implement, or you can double-emit the index and
avoid the accounting entirely).

3. Avoid the problem altogether and recommend the use of commas for
preserving order.  The only cases where this doesn't work is for
Cookie and Set-Cookie.  For those, I know it might sound a little
risky for some, but losing ordering might not be a bad thing there,
despite what 6265 says.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 11:49:20 -0800

#1 works today (and is my preference).
If we simply specify that  a null in a value is to be interpreted as
multiple key-values (with the same key), then we allow everything #3 does.

#2 makes it very difficult to use HTTP2's framing layer for non-HTTP
transport. That sucks. It also still requires either #1, #3, or machinery
in the HTTP<->Encoder interface to force things to be emitted in the order
desired by referencing items multiple times.

#3 seems more finicky than #1, with no real advantages.

#1 is my strong preference.
-=R

---

From: Patrick McManus <[email-masked]>
Date: Thu, 21 Nov 2013 15:03:03 -0500

based on running code, #1 is good

On Thu, Nov 21, 2013 at 2:49 PM, Roberto Peon <[email-masked]> wrote:

> On Thu, Nov 21, 2013 at 11:12 AM, Martin Thomson <[email-masked]
> > wrote:
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 12:34:36 -0800

And to be clear, I mean for #1 to happen only when the semantic-layer cares
about ordering.
If it doesn't (e.g. potentially cookie), then it doesn't collapse the
values into a null-delimited-list-value

-=R

On Thu, Nov 21, 2013 at 12:03 PM, Patrick McManus <[email-masked]>wrote:

>> On Thu, Nov 21, 2013 at 11:12 AM, Martin Thomson <
>> [email-masked]> wrote:
>>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 12:49:22 -0800

On 21 November 2013 12:34, Roberto Peon <[email-masked]> wrote:
> If it doesn't (e.g. potentially cookie), then it doesn't collapse the values
> into a null-delimited-list-value

This is where it gets interesting.  That exact same argument can be
made for #3, and that doesn't require new semantics.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 12:54:40 -0800

#3 doesn't work for cookie/setcookie, whereas #1 works for everything.
#3 also requires parsing quoted-strings, which is painfully annoying,
whereas #1 requires no such machinery-- a simple Split() will be sufficient
in most languages.
#1 also more faithfully recreates the input, if that ever matters.
-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 12:58:33 -0800

> #3 doesn't work for cookie/setcookie, whereas #1 works for everything.

You already require special code for these two:
http://http2.github.io/http2-spec/#CompressCookie
https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/25/p1-messaging.html#field.order

> #3 also requires parsing quoted-strings, which is painfully annoying,

It does not, it only requires concatenation at the encoder.  Once
concatenated, the field is semantically equivalent and therefore
doesn't need to be re-split.

---

From: Patrick McManus <[email-masked]>
Date: Thu, 21 Nov 2013 16:05:18 -0500

> made for #3, and that doesn't require new semantics.

I think the new semantics are pretty simple.. and given that we've seen it
br reliable I'm not strongly feeling like there is motivation to go another
direction. But maybe I'm just overcome by pragmatism today :)

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 13:23:33 -0800

> http://http2.github.io/http2-spec/#CompressCookie

That is an optimization for Cookie which can safely be ignored in
implementations.

> https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/25/p1-messaging.html#field.order

#1 solves the problem of concatenation for set-cookie since '\0' is always
disallowed in header values-- with #1, set-cookie is no longer special.

> doesn't need to be re-split.

True so long as the values are well formed, but at the decoder:
With #1, to tokenize (but not necessarily interpret), I simply split on
NULL.
With #3, to tokenize (but not necessarily interpret), I must parse the
value for quoted strings and split on ','s which are outside those strings.
There are good reasons for one to wish to tokenize and not interpret
(logging, computing differences, etc.).

>From a security perspective, with #1 it is more difficult to get a
malformed quoted-string to interact with the rest of the header fields.
-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 13:45:32 -0800

> That is an optimization for Cookie which can safely be ignored in

I hope not.  A decoder will need to pay attention.

Regardless, there is still the prohibition regarding multiple Cookie
headers absent this optimization.

> disallowed in header values-- with #1, set-cookie is no longer special.

Only if ordering is significant for Set-Cookie.  It's not.  Hence the
"no special desire to invent a new mechanism".

> With #1, to tokenize (but not necessarily interpret), I simply split on

Again not so, because you still have to look for commas in the values
that weren't split across multiple fields.

> From a security perspective, with #1 it is more difficult to get a malformed
> quoted-string to interact with the rest of the header fields.

Malformed header fields generating unexpected results sounds like an
interesting theoretical problem.  It means that you might be able to
sneak something past an intermediary.  It means that you can trigger
something.  I don't find that especially interesting as an attack,
because it exploits an intermediary that isn't doing its job
particularly well.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 14:08:53 -0800

> I hope not.  A decoder will need to pay attention.

The cookie optimization can be safely ignored by an encoder, but never a
decoder, true.

> headers absent this optimization.

The application should see a single cookie, I think you mean (we don't
otherwise care how it transmitted).

> "no special desire to invent a new mechanism".

We need a mechanism one way or another for fields where order might matter.

I'm saying that the mechanism in #1 can be used to transmit the multiple
Set-Cookie headers in the response instead of potentially sending a bunch
of opcodes one at a time. This can end up being more efficient than
sticking it into the compression context (e.g. when not using an indexed
mode).
The fact that we don't care about the order for a particular field means
that it can't hurt if we do specify an order for that field. Any such order
can simply be ignored after detokenizing for fields that don't care about
ordering.

> that weren't split across multiple fields.

I don't need to do that for SetCookie today-- each one is its own header.

> malformed

It would/could affect any recipient where the sender allowed a value to be
stuck into a header without doing adequate checking for correctness.
I agree it isn't something to be overly concerned about.

We've already implemented '\0' delimiting! It hasn't yet caused any issue
that I know of.

-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 14:20:13 -0800

I think you're getting the proposal mixed up a little here.

1. We're creating special rules for Cookie, but those rules are
separable.  One value emitted only (multiple carried).  No nulls
needed there.

2. Set-Cookie is already special, but since ordering doesn't matter,
allow the compressor to emit these in any order, no concatenation.  It
remains as a perpetual gotcha, but no proposal was going to change
that characteristic.

3. Everything else already has rules for handling multiple header
field name-value pairs.  Leverage those existing rules and require
concatenate on encode to preserve order (unless you know that order
doesn't matter, which might let you get some compression efficiency,
c.f. cache-control).

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 14:32:45 -0800

> separable.  One value emitted only (multiple carried).  No nulls

Unless/until we want to specify an order for Cookie, but even then ','
could be used.
I think we're on the same page on this.

> remains as a perpetual gotcha, but no proposal was going to change

I was saying that with null as the concatenation-separator one can safely
specify an order for Set-Cookie since we don't care about the ordering.
This is potentially useful in reducing the on-the-wire size of things.

> doesn't matter, which might let you get some compression efficiency,

Yup.

#1 and #3 are very similar. #1 arguably has some small marginal benefits
over #3 (as I've been attempting to point out, if poorly :) ), the most
important of which is that it is already implemented.
-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 14:39:15 -0800

> important of which is that it is already implemented.

I see other advantages for #3.  The fact is that you need to implement
it anyway (since comma-concatenation is still valid), and that it
doesn't require additional code in the decoder seem like actual
advantages, despite the risks.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 14:59:30 -0800

I suspect we'll both remain biased towards each of #1 and #3.
Thankfully, in either case the marginal difference is small and both #1 and
#3 are substantially better than #2.
-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 21 Nov 2013 15:04:19 -0800

> #3 are substantially better than #2.

Yep.  Time for RFC 3929 procedures.  No sense in restricting all the
fun in RTCWEB to just that group.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 15:05:42 -0800

Muahahahaha!
no.
:)
-=R

---

From: Patrick McManus <[email-masked]>
Date: Thu, 21 Nov 2013 20:23:17 -0500

:)

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 12:25:46 +1100

So, *any* header that uses the list production *could* be sensitive to ordering. 

From a quick look at the registry, besides cookies the following define a meaningful semantic for ordering:

A-IM
IM
Accept-Language (maybe; see our note in p2)
Content-Encoding
Forwarded
Via

I can imagine a case where Content-Encoding is applied by an intermediary, but having more than one encoding isn't that common (which might lead to worse bugs, since intermediaries might not be written to check for an existing C-E and fold the headers).

Via is interesting, because intermediaries are required to append to it as a message goes through it, and ordering is important for debugging (e.g., loop detection). 

Presumably X-Forwarded-For and Forwarded suffer from this as well.

Another interesting case is one where a header field only allows one value, and an implementation picks the first one (for example) -- e.g., Host. If ordering after compression isn't deterministic, it may be an attack vector.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: James M Snell <[email-masked]>
Date: Thu, 21 Nov 2013 17:43:04 -0800

I would note also that implementations can vary on how they handle multiple
header instances.  For instance, I've seen some impls that only pay
attention to the first link header in a request while others only see the
last one.  Nondeterministic ordering could cause bad things to occur.
On Nov 21, 2013 5:29 PM, "Mark Nottingham" <[email-masked]> wrote:

> I can imagine a case where Content-Encoding is applied by an intermediary,
> but having more than one encoding isn't that common (which might lead to
> worse bugs, since intermediaries might not be written to check for an
> existing C-E and fold the headers).
>
> Via is interesting, because intermediaries are required to append to it as
> a message goes through it, and ordering is important for debugging (e.g.,
> loop detection).
>
> Another interesting case is one where a header field only allows one
> value, and an implementation picks the first one (for example) -- e.g.,
> Host. If ordering after compression isn't deterministic, it may be an
> attack vector.
>
> Cheers,
>
>
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 17:57:39 -0800

All of the proposals handle any potential nondeterminism amongst header
fields with the same name, so that isn't a problem...

-=R

On Thu, Nov 21, 2013 at 5:43 PM, James M Snell <[email-masked]> wrote:

> I would note also that implementations can vary on how they handle
> multiple header instances.  For instance, I've seen some impls that only
> pay attention to the first link header in a request while others only see
> the last one.  Nondeterministic ordering could cause bad things to occur.
>> I can imagine a case where Content-Encoding is applied by an
>> intermediary, but having more than one encoding isn't that common (which
>> might lead to worse bugs, since intermediaries might not be written to
>> check for an existing C-E and fold the headers).
>>
>> Via is interesting, because intermediaries are required to append to it
>> as a message goes through it, and ordering is important for debugging
>> (e.g., loop detection).
>>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 13:13:29 +1100

Well, if by "handle" you mean "anyone who generates or modifies this header has to understand the special handling", yes...

On 22/11/2013, at 12:57 PM, Roberto Peon <[email-masked]> wrote:

> All of the proposals handle any potential nondeterminism amongst header fields with the same name, so that isn't a problem...
> 
> -=R
> 
> 
> I would note also that implementations can vary on how they handle multiple header instances.  For instance, I've seen some impls that only pay attention to the first link header in a request while others only see the last one.  Nondeterministic ordering could cause bad things to occur. 
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 18:29:17 -0800

As I understand it for either #1 or #3:
ordering between header values with the same key MUST use
value-concatenation unless the header is known to not care about ordering
(setcookie, etc.).

thus, if we saw the following headers:
  foo: bar
  boo: hiss
  foo: baz

then we'd expect to see foo: bar<delim>baz, and boo: hiss (or boo: hiss and
foo: bar<delim>baz)

If we had:
  set-cookie: a
  set-cookie: b
  foo: bar
  foo: baz
then we could expect:
  set-cookie: b
  set-cookie: a
  foo: bar<delim>baz

In other words, ordering is always maintained except when we know it is
safe to ignore.
-=R

On Thu, Nov 21, 2013 at 6:13 PM, Mark Nottingham <[email-masked]> wrote:

> Well, if by "handle" you mean "anyone who generates or modifies this
> header has to understand the special handling", yes...
>
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 13:39:24 +1100

I'm OK with that as long as we have the MUST... unless clause.

On 22/11/2013, at 1:29 PM, Roberto Peon <[email-masked]> wrote:

> ordering between header values with the same key MUST use value-concatenation unless the header is known to not care about ordering (setcookie, etc.).
> 
> then we'd expect to see foo: bar<delim>baz, and boo: hiss (or boo: hiss and foo: bar<delim>baz)
> 
> If we had:
>   set-cookie: a
>   set-cookie: b
>   foo: bar
>   foo: baz
> then we could expect:
>   set-cookie: b
>   set-cookie: a
>   foo: bar<delim>baz
> 
> In other words, ordering is always maintained except when we know it is safe to ignore.
> -=R
> 
>

---

From: James M Snell <[email-masked]>
Date: Thu, 21 Nov 2013 19:34:36 -0800

Do we (yet) have a list of all the registered header fields for which order
is insignificant?
On Nov 21, 2013 6:39 PM, "Mark Nottingham" <[email-masked]> wrote:

> > > All of the proposals handle any potential nondeterminism amongst
> header fields with the same name, so that isn't a problem...
> > >
> > > -=R
> > >
> > >
> > > >From a quick look at the registry, besides cookies the following
> define a meaningful semantic for ordering:
> > >
> > > A-IM
> > > IM
> > > Via is interesting, because intermediaries are required to append to
> it as a message goes through it, and ordering is important for debugging
> (e.g., loop detection).
> > >

---

From: Roberto Peon <[email-masked]>
Date: Thu, 21 Nov 2013 19:36:37 -0800

So far, the ones I know about are cookie and setcookie, but mainly because
those are the ones that I care about.
-=R

On Thu, Nov 21, 2013 at 7:34 PM, James M Snell <[email-masked]> wrote:

> Do we (yet) have a list of all the registered header fields for which
> order is insignificant?

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 14:37:20 +1100

To be clear - what makes me somewhat comfortable with this approach is that the default is that order is preserved; only if you know that ordering *is* insignificant are you allowed to break it up.

Cheers,

On 22/11/2013, at 2:34 PM, James M Snell <[email-masked]> wrote:

> Do we (yet) have a list of all the registered header fields for which order is insignificant?
>

---

From: James M Snell <[email-masked]>
Date: Thu, 21 Nov 2013 20:00:36 -0800

Order is supposed to be insignificant for Link and Prefer. There are others
which could, in theory appear more than once but typically don't.
If-None-Match, If-Match, If-Modified-Since, and If-Unmodified-Since for
example.

The various Auth headers are theoretically order agnostic also... But it
may be safer to preserve order on those.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 19:11:41 +1100

IIRC in Seattle we talked about cases (esp. back ends) where preserving the difference between comma delimitation and a new header line was important when going 1.1<->2.0. IIRC Jeff felt strongly about this… (?)

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 23 Nov 2013 03:16:01 +1300

> I can imagine a case where Content-Encoding is applied by an intermediary,

The Squid ecap-gzip adaptor does something along this line. Updating the
request to negotiate gzip compression with the server and decompressing
the response. Although I'm not sure about it enough to know if there are
multiple C-E values present as a result (I hope not).

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 23 Nov 2013 03:18:55 +1300

> fields with the same name, so that isn't a problem...

Also keep in mind that 1.1->2.0 and 2.0->2.0 software is all going to be
fresh code. If the spec is clear enough on where multiple values are to
be found and how to handle there is no excuse for this mess to continue
outside of legacy 1.1 connections.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 23 Nov 2013 03:20:24 +1300

> To be clear - what makes me somewhat comfortable with this approach is that the default is that order is preserved; only if you know that ordering *is* insignificant are you allowed to break it up.

+1.

Amos

---

From: Mike Bishop <[email-masked]>
Date: Fri, 22 Nov 2013 16:51:27 +0000

Our RFC2616bis says this:

The order in which header fields with the same field name are received is therefore significant to the interpretation of the combined field value; a proxy MUST NOT change the order of these field values when forwarding a message.

If 1.1 says that order is always significant and we're not supposed to be changing 1.1 semantics....

Sent from Windows Mail

From: Amos Jeffries<mailto:[email-masked]>
Sent: ?Friday?, ?November? ?22?, ?2013 ?6?:?22? ?AM
To: [email-masked]<mailto:[email-masked]>

+1.

Amos

---

From: RUELLAN Herve <[email-masked]>
Date: Fri, 22 Nov 2013 16:57:58 +0000

+1

I like both #1 and #3 proposals, with a preference for #3 as it doesn't introduce any new mechanism.
Not using #2 means that we will be able to drop some emission ordering stuff from HPACK.

Hervé.

> -----Original Message-----
> From: Amos Jeffries [mailto:[email-masked]]
> Sent: vendredi 22 novembre 2013 15:20
> To: [email-masked]
> Subject: Re: #305 Header ordering
> 
> > To be clear - what makes me somewhat comfortable with this approach is
> that the default is that order is preserved; only if you know that ordering *is*
> insignificant are you allowed to break it up.
> >
> 
> +1.
> 
> Amos
> 
> 
>

---

From: James M Snell <[email-masked]>
Date: Fri, 22 Nov 2013 09:01:45 -0800

That's why going with the default option of requiring strict ordering is
the right approach. Only the header fields we absolutely know do not care
about ordering would be impacted. Most impls, I suspect, will simply choose
not to reorder anything.
On Nov 22, 2013 8:55 AM, "Mike Bishop" <[email-masked]> wrote:

> If 1.1 says that order is always significant and we’re not supposed to be
> changing 1.1 semantics….
>
>  Sent from Windows Mail
>
>   *From:* Amos Jeffries <[email-masked]>
> *Sent:* ‎Friday‎, ‎November‎ ‎22‎, ‎2013 ‎6‎:‎22‎ ‎AM
> *To:* [email-masked]
>
> that the default is that order is preserved; only if you know that ordering
> *is* insignificant are you allowed to break it up.
> >
>
> +1.
>
> Amos
>
>
>
>
>

---

From: Martin Thomson <[email-masked]>
Date: Fri, 22 Nov 2013 09:58:27 -0800

> those are the ones that I care about.

More importantly, those are the ones that are *special*.  Neither uses
the standard list construction form and thus require special handling.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 22 Nov 2013 09:59:53 -0800

> To be clear - what makes me somewhat comfortable with this approach is that the default is that order is preserved; only if you know that ordering *is* insignificant are you allowed to break it up.

That is my sense too.  I think that Roberto and I are in complete
agreement about this.  The point of disagreement is the delimiter
character.  Not quite a bike shed, but that's where I'd like the
attention focused :)

---

From: Martin Thomson <[email-masked]>
Date: Fri, 22 Nov 2013 10:02:02 -0800

On 22 November 2013 08:51, Mike Bishop <[email-masked]> wrote:
> The order in which header fields with the same field name are received is
> therefore significant to the interpretation of the combined field value; a
> proxy MUST NOT change the order of these field values when forwarding a
> message.
>

That's generic guidance.  I am personally comfortable with a
semantics-aware encoder reordering the header fields, which is what
you effectively do if you don't use the ordering-guaranteed mechanism
we define.

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Sat, 23 Nov 2013 15:17:51 +0900

> Not using #2 means that we will be able to drop some emission ordering
I also prefer #3 to #1 with the same reason.

Best,
Tatsuhiro Tsujikawa

---

From: Roberto Peon <[email-masked]>
Date: Fri, 22 Nov 2013 22:38:11 -0800

How about replacing all ','s with '\0's?
'\0' is a disallowed character everywhere, unlike ',' which requires
special parsing because of quoted strings.
I'd rather that the lower layers of the protocol stack not have to do
quoted string parsing.
That belongs to the part of the stack that is parsing the individual
fields, not the code that has to determine the field boundaries.

-=R

On Fri, Nov 22, 2013 at 10:17 PM, Tatsuhiro Tsujikawa <[email-masked]

>
>
>
> On Sat, Nov 23, 2013 at 1:57 AM, RUELLAN Herve <[email-masked]
> > wrote:
>
>> +1
>>

---

From: Martin Thomson <[email-masked]>
Date: Sat, 23 Nov 2013 13:11:56 -0800

On 22 November 2013 22:38, Roberto Peon <[email-masked]> wrote:
> '\0' is a disallowed character everywhere, unlike ',' which requires special
> parsing because of quoted strings.
> I'd rather that the lower layers of the protocol stack not have to do quoted
> string parsing.

As proposed, you don't need to parse unless you are interpreting the value.

What you propose here would force lower layers of the stack to parse
quoted strings to determine where to replace ',' with '\0'.  That
sounds worse to me.

---

From: Roberto Peon <[email-masked]>
Date: Sat, 23 Nov 2013 13:33:05 -0800

Wouldn't it be the higher-layers only that need to parse quoted-strings
today?  Many loadbalancers need-not parse quoted-strings today, for
instance: it is unlikely that any field they care about examining allows
quoted-strings...

At least in my implementation, the data within the header fields (which
might include quoted-strings) are provided by the application which
manipluates a headers object. The headers object or lower never needs to
parse the field in the outbound path.
On the reverse side, the IO library+framer feeds the keys/values into a
header object. It doesn't tokenize-- that is left to the application. The
framer/headers object never needs to parse the fields in the inbound path.

Setting that point aside, how would I deal with the following response
headers in the #3 proposal?
  Set-Cookie: name1=value1; Expires=Wed, 09 Jun 2021 10:18:14
  Set-Cookie: name2=value2; Expires=Thu, 10 Jun 2021 10:18:14
  Set-Cookie: name3=value3; Expires=Fri, 11 Jun 2021 10:18:14

With proposal #1, an encoder can simply do:
  Set-Cookie: name1=value1; Expires=Wed, 09 Jun 2021
10:18:14\0name2=value2; Expires=Thu, 10 Jun 2021 10:18:14\0name3=value3;
Expires=Fri, 11 Jun 2021 10:18:14

With #3, attempting to use the #3 mechanism requires a context-aware parser
(ick):
  Set-Cookie: name1=value1; Expires=Wed, 09 Jun 2021 10:18:14,name2=value2;
Expires=Thu, 10 Jun 2021 10:18:14,name3=value3; Expires=Fri, 11 Jun 2021
10:18:14
or it requires the encoder to send these as separate header fields.

Much of the time, there isn't a good reason to want to send these as
separate header fields.

-=R

On Sat, Nov 23, 2013 at 1:11 PM, Martin Thomson <[email-masked]>wrote:

> quoted

---

From: Martin Thomson <[email-masked]>
Date: Sat, 23 Nov 2013 13:47:15 -0800

> framer/headers object never needs to parse the fields in the inbound path.

That's all true, and exactly the point.

>   Set-Cookie: name3=value3; Expires=Fri, 11 Jun 2021 10:18:14

I don't know why we keep coming back to the special cases.  But
anyway, as proposed, we understand order to not be significant for
Set-Cookie, so this would manifest as multiple name-value pairs, in a
non-deterministic order.  Cookie is special too, but we actually need
new rules for Cookie.

Set-Cookie breaks the general grammar rules for header fields and
needs to be treated specially.  Neither proposal changes that
characteristic.

> Much of the time, there isn't a good reason to want to send these as

For Set-Cookie, sure, but in terms of cost, it's a question of a few
bytes difference at most.  I'm not going to get cut up about that.

---

From: Jeff Pinner <[email-masked]>
Date: Sun, 24 Nov 2013 22:39:41 -0800

My $.02,

We've seen '\0' delineation work well and I'm not sure what we gain by
changing it to ','

If an intermediary is going to fold multiple headers because it isn't sure
that order is important (especially for headers that do not allow commas)
then I would prefer to be able to reconstruct the original as faithfully as
possible. For example, let's suppose I get a request with multiple host or
content-length headers (yes this does occur). If an intermediary
concatenates them, I would prefer it be in a way that lets me re-construct
the original input.

- Jeff

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Nov 2013 13:18:39 +1100

Given that this is holding up the implementation draft and we have running code for #1, let's stick with it for now. If people feel strongly, we can revisit in the next round.

Make sense?

Cheers,

On 25/11/2013, at 5:39 PM, Jeff Pinner <[email-masked]> wrote:

> My $.02,
> 
> We've seen '\0' delineation work well and I'm not sure what we gain by changing it to ','
> 
> If an intermediary is going to fold multiple headers because it isn't sure that order is important (especially for headers that do not allow commas) then I would prefer to be able to reconstruct the original as faithfully as possible. For example, let's suppose I get a request with multiple host or content-length headers (yes this does occur). If an intermediary concatenates them, I would prefer it be in a way that lets me re-construct the original input.
> 
> - Jeff
> 
> 
> 
> 
> On Sat, Nov 23, 2013 at 1:47 PM, Martin Thomson <[email-masked]> wrote:

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Tue, 26 Nov 2013 10:45:37 +0100

Le Lun 25 novembre 2013 07:39, Jeff Pinner a écrit :
> possible.

Actually, it would be nice to reserve an external separator in http/2, so
bits like user-agents can be logged without playing quoting games. Surely
utf-8 space is big enough to reserve a bloody separator?

(it's not for nefarious monitoring, it's to detect the broken clients that
break the service for everyone, and logging http headers that can contain
spaces when the default apache log format is space separated is killing
tools as soon as you try to query millions of lines that need unquoting)

Regards,

-- 
Nicolas Mailhot

---

From: RUELLAN Herve <[email-masked]>
Date: Tue, 26 Nov 2013 16:40:39 +0000

I'm OK with this.

For HPACK, this means that we need to remove the ordering requirements for emission. Add we should probably add that header values should not be split at the compaction layer.

Hervé.

> From: Mark Nottingham [mailto:[email-masked]]
> Sent: mardi 26 novembre 2013 03:19
> To: Jeff Pinner
> Cc: Martin Thomson; Roberto Peon; Tatsuhiro Tsujikawa; RUELLAN Herve;
> Amos Jeffries; [email-masked]
> Given that this is holding up the implementation draft and we have running
> code for #1, let's stick with it for now. If people feel strongly, we can revisit in
> the next round.
> 
> Make sense?
> 
> Cheers,
> 
> 
> > If an intermediary is going to fold multiple headers because it isn't sure that
> order is important (especially for headers that do not allow commas) then I
> would prefer to be able to reconstruct the original as faithfully as possible. For
> example, let's suppose I get a request with multiple host or content-length
> headers (yes this does occur). If an intermediary concatenates them, I would
> prefer it be in a way that lets me re-construct the original input.
> >
> > - Jeff
> >
> >
> >
> >
> > > Wouldn't it be the higher-layers only that need to parse
> > > quoted-strings today?  Many loadbalancers need-not parse
> > > quoted-strings today, for
> > > instance: it is unlikely that any field they care about examining
> > > allows quoted-strings...
> > >
> > > At least in my implementation, the data within the header fields
> > > (which might include quoted-strings) are provided by the application
> > > which manipluates a headers object. The headers object or lower
> > > never needs to parse the field in the outbound path.
> > > On the reverse side, the IO library+framer feeds the keys/values
> > > into a header object. It doesn't tokenize-- that is left to the
> > > application. The framer/headers object never needs to parse the fields in
> the inbound path.
> >
> > > Setting that point aside, how would I deal with the following
> > > response headers in the #3 proposal?

---

From: Martin Thomson <[email-masked]>
Date: Tue, 26 Nov 2013 09:36:18 -0800

> For HPACK, this means that we need to remove the ordering requirements for emission. Add we should probably add that header values should not be split at the compaction layer.

I didn't realize that you did have ordering requirements on emission.
I wouldn't have thought that you would change any of the text around
emission, though you might say - as a more general statement - that
headers can be emitted in any order.

BTW, I think that Section 3.1.5 should be moved under Section 3.2.
Section 3.1 is about the encoder, but 3.1.5 is about the decoder.
http://http2.github.io/http2-spec/compression.html#header.emission