Subject: Call for Proposals re: #314 HTTP2 and http:// URIs on the "open" internet

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 12:02:36 +1100

<https://github.com/http2/http2-spec/issues/314>

While discussion over the last week certainly has been interesting, we can't debate the big questions forever; there's too much other work for us. 

Issue #314 pertains to what -- if anything -- we say about the deployment and support of HTTP/2.0 for http:// URIs on the "open internet".

So far, we don't have any text for this issue, so I'm asking for proposals to be made now. 

If we can't get consensus (or if one isn't made), the default is to leave the specification as-is; that is, we'll continue to define how to use HTTP/2.0 for both http:// and https://, and implementations will choose which scheme(s) they support for the new protocol. You're welcome to explicitly propose the status quo, of course. 

To reiterate -- some browser folks have stated that they will not be implementing HTTP/2.0 for HTTP without TLS in their products, so unless they become convinced otherwise, there will still be a *market* requirement to implement TLS if you want to get the benefit of HTTP/2 with the broadest selection of clients. What we're talking about here is what normative requirements we want to put in the spec itself.

When making a proposal, please keep in mind that it needs to meet the bar of rough consensus and running code. Also, your proposal should define its scope of applicability (the "open internet" definition; generally, we've been talking about browsers interacting with sites on the public Web).

Also, note that we have a lot of related issues to discuss. Proposals might include assumptions about these; e.g., you can suggest that we require opportunistic TLS to be implemented for HTTP URIs. Please don't include too much detail about upgrade processes, mechanics of opportunistic encryption, etc., since these are separable decisions.

To help us focus here, please refrain from the *general* discussion of the pros and cons of requiring TLS, so that we can concentrate on the proposals (if any emerge). I will be reminding people of this nicely (at first).

Thanks,

P.S. If you intend to make a proposal but aren't ready to do so yet, please contact me privately. I'm willing to defer discussion of this issue for a reasonable amount of time if it'll help us get to consensus.

--
Mark Nottingham   http://www.mnot.net/

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 20 Nov 2013 11:53:05 +1000

On 20 November 2013 11:02, Mark Nottingham <[email-masked]> wrote:

> If we can't get consensus (or if one isn't made), the default is to leave
> the specification as-is; that is, we'll continue to define how to use
> HTTP/2.0 for both http:// and https://, and implementations will choose
> which scheme(s) they support for the new protocol. You're welcome to
> explicitly propose the status quo, of course.
>

I explicitly propose the status quo: leave any recommendation (normative or
otherwise) out of the base HTTP/2.0 spec.  Instead I suggest we/you/someone
create a separate (informational?) document describing these issues.

At the least, I would imagine the base spec to be (hopefully) current and
useful for decades, whereas what various browsers do or don't support, or
even what browsers exist or matter in the market, might be a bit more
transient; thus any documentation that depends on them should be equally
versatile.  Updating or obsoleting an informational RFC would be a lot
easier than keeping a "that's *so* 2013" social reference in the HTTP/2.0
spec, and possibly pointing out to people that it, in fact, no longer
applies, or whatever.

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Grahame Grieve <[email-masked]>
Date: Wed, 20 Nov 2013 13:20:42 +1100

+1.

Unless you propose to simply ban non-TLS HTTP/2 altogether under any
circumstance, this becomes a policy issue, and belongs elsewhere (and,
in fact, what about that is specific to http/2? - probably negotiated
upgrade, which is not possible in http/1)

Grahame

-- 
-----
http://www.healthintersections.com.au /
[email-masked] / +61 411 867 065

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 20 Nov 2013 03:50:23 +0100

* Mark Nottingham wrote:
>To reiterate -- some browser folks have stated that they will not be 
>implementing HTTP/2.0 for HTTP without TLS in their products, so unless 
>they become convinced otherwise, there will still be a *market* 
>requirement to implement TLS if you want to get the benefit of HTTP/2 
>with the broadest selection of clients.

It seems very likely to me that whatever has been stated was meant to be
something far more nuanced. For instance, they may have meant that they
will not personally write the code, but patches from others might still
be accepted e.g. because the decision isn't up to them. Importantly, I'd
think they respect and participate in the Standards process and evaluate
any outcome carefully, which very much includes considering to implement
HTTP/2.0 without TLS if the Working Group decides to mandate support for
that.

If any major browser vendor has issued an official statement that they
will use their market position to force certain design and deployment
decisions in subversion of the Standards process, that would be highly
inappropriate. I would expect those browser folks to take offense at a
suggestion that "we" have to convince "them" any more than "they" have
to convince "us", whatever the issue may be.

There are only about four browser vendors with sufficient market share
to force deployment decisions. They should be identified by name, and
not as an anonymous collective, if they have stated anything for us to
consider, with references to their statements.

>P.S. If you intend to make a proposal but aren't ready to do so yet, 
>please contact me privately. I'm willing to defer discussion of this 
>issue for a reasonable amount of time if it'll help us get to consensus.

I sort-of intend to make related proposals and am not ready yet, but I
see no reason to defer any discussion (as opposed to decision) because
of that. It is not entirely clear to me what the scope of your call for
proposals is, but for the record, discussion of issues like whether to
mandate support for HTTP/2.0 without TLS, or making web browsers non-
conforming if they support HTTP/2.0 without TLS, has barely begun and I
need quite a bit of additional information before I could form an in-
formed opinion.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 14:03:50 +1100

> HTTP/2.0 without TLS if the Working Group decides to mandate support for

No one has yet proposed that we mandate implementing HTTP/2.0 *without* TLS yet -- we'll cross that bridge if we come to it. Talking about "subverting the standards process" is thus WAY too premature.

> need quite a bit of additional information before I could form an in-

Please read that as deferring a decision. I'm happy for people to make proposals and discuss them. What I'm not willing to do is let a general discussion without focus continue for too long and threaten the rest of our work. It's quite apparent to me that people are digging in on their positions and we can't reach consensus that way.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 14:07:59 +1100

Hi Matthew,

On 20/11/2013, at 12:53 PM, Matthew Kerwin <[email-masked]> wrote:

> I explicitly propose the status quo: leave any recommendation (normative or otherwise) out of the base HTTP/2.0 spec.  Instead I suggest we/you/someone create a separate (informational?) document describing these issues.
> 
> At the least, I would imagine the base spec to be (hopefully) current and useful for decades, whereas what various browsers do or don't support, or even what browsers exist or matter in the market, might be a bit more transient; thus any documentation that depends on them should be equally versatile.  Updating or obsoleting an informational RFC would be a lot easier than keeping a "that's *so* 2013" social reference in the HTTP/2.0 spec, and possibly pointing out to people that it, in fact, no longer applies, or whatever.

So, to be clear, you'd be OK with a separate Informational RFC called something like "HTTP/2.0 for Web Browsers" or "HTTP/2.0 for the Open Web" or similar, even if that spec constrained (or eliminated) using HTTP/2.0 for http:// URLs?

Thanks,

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 20 Nov 2013 13:27:30 +1000

On 20 November 2013 13:07, Mark Nottingham <[email-masked]> wrote:

> Hi Matthew,
>
> So, to be clear, you'd be OK with a separate Informational RFC called
> something like "HTTP/2.0 for Web Browsers" or "HTTP/2.0 for the Open Web"
> or similar, even if that spec constrained (or eliminated) using HTTP/2.0
> for http:// URLs?
>

Yes.  I want to see HTTP/2.0 progressed, I don't want it to get bogged down
in debates about deployment policy recommendations/mandates.  If that
separate discussion eventually results in consensus, then so be it,
whatever it may say; and if it changes over time, even better.

> Thanks,

No problem.

---

From: James M Snell <[email-masked]>
Date: Tue, 19 Nov 2013 19:43:23 -0800

On Tue, Nov 19, 2013 at 7:03 PM, Mark Nottingham <[email-masked]> wrote:
>[snip]

Honestly, I'm close to this, but *only* over a new dedicated port. To
be clear, as an application developer building on top of HTTP/2, I
want to be able, should I so choose, to rely on the ability to use
plain text http/2 and do not want a handful of user-agent developers
to make that decision for me. That said, however, I recognize the
challenges with making plaintext HTTP/2 over port 80 a mandatory to
implement thing, therefore, mandatory to implement over a new
dedicated port would appear to be a reasonable compromise option.

- James

---

From: Roberto Peon <[email-masked]>
Date: Tue, 19 Nov 2013 19:57:34 -0800

How about:
HTTPS schemed URLs MUST be sent on an authenticated TLS channel.
HTTP schemed URLs MAY be sent as unencrypted HTTP2 plaintext, or may be
sent over a TLS channel.

If a server does not wish to handle HTTP schemed URLs over a TLS channel,
it MUST reject these requests with a RST_STREAM or GOAWAY with an error
code that indicates that the server does not support HTTP schemed URLs on
port 443.
-=R

On Tue, Nov 19, 2013 at 7:43 PM, James M Snell <[email-masked]> wrote:

> > No one has yet proposed that we mandate implementing HTTP/2.0 *without*
> TLS yet -- we'll cross that bridge if we come to it. Talking about
> "subverting the standards process" is thus WAY too premature.
> >
>

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 15:00:18 +1100

> dedicated port would appear to be a reasonable compromise option.

I think that, if proposed, it would be even more difficult to get consensus on this than on prohibiting HTTP/2 for http:// URIs. Not only are some implementers against it, but on its own, this would be a step backwards in security -- right now, HTTP/1.1 doesn't require implementation without encryption. 

Much experience has shown us that MUSTs and SHOULDs are ignored when they're disconnected from implementation needs -- even if those requirements are intended for the greater good.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 15:02:26 +1100

So I'm interpreting this as a two-part proto-proposal --

a) don't constrain the URI scheme for HTTP/2
b) develop opportunistic encryption of some sort (issue #315).

Is that accurate?

Cheers,

On 20/11/2013, at 2:57 PM, Roberto Peon <[email-masked]> wrote:

> HTTP schemed URLs MAY be sent as unencrypted HTTP2 plaintext, or may be sent over a TLS channel.
> 
> If a server does not wish to handle HTTP schemed URLs over a TLS channel, it MUST reject these requests with a RST_STREAM or GOAWAY with an error code that indicates that the server does not support HTTP schemed URLs on port 443.
> -=R
> 
> 
>

---

From: Roberto Peon <[email-masked]>
Date: Tue, 19 Nov 2013 20:04:08 -0800

Yup.

-=R
On Nov 19, 2013 8:02 PM, "Mark Nottingham" <[email-masked]> wrote:

> > If a server does not wish to handle HTTP schemed URLs over a TLS
> channel, it MUST reject these requests with a RST_STREAM or GOAWAY with an
> error code that indicates that the server does not support HTTP schemed
> URLs on port 443.
> > -=R
> >
> >
> >
> > > No one has yet proposed that we mandate implementing HTTP/2.0
> *without* TLS yet -- we'll cross that bridge if we come to it. Talking
> about "subverting the standards process" is thus WAY too premature.
> > >
> >

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 20 Nov 2013 04:08:49 +0000

my 2c is that http/2.0 (TLS or not) is enough of a departure from http, 
that trying to put plaintext http/2.0 over port 80 will just be an 
impossible nightmare.

We could consider that http/2.0 is an entirely new protocol. It just has 
the same purpose as and contains (as a subset) the semantics of http/1.1 
and we intend it to replace 1.1.

So if what is being proposed here is that http/2.0 uses another port and 
https/2.0 uses port 443 enabled by NPN/ALPN etc then I'm happy with 
that.

Adrien

------ Original Message ------
From: "James M Snell" <[email-masked]>
To: "Mark Nottingham" <[email-masked]>
Cc: "Bjoern Hoehrmann" <[email-masked]>; "HTTP Working Group" 
<[email-masked]>
Sent: 20/11/2013 4:43:23 p.m.
Subject: Re: Call for Proposals re: #314 HTTP2 and http:// URIs on the 
"open" internet

---

From: James M Snell <[email-masked]>
Date: Tue, 19 Nov 2013 20:24:20 -0800

On Tue, Nov 19, 2013 at 8:00 PM, Mark Nottingham <[email-masked]> wrote:
[snip]
> I think that, if proposed, it would be even more difficult to get consensus on this than on prohibiting HTTP/2 for http:// URIs. Not only are some implementers against it, but on its own, this woul…

Nor does HTTP/1.1 require implementation *with* encryption. Note that
by making plaintext on a dedicated port required to implement
obviously doesn't make it required to use.

That said, if there's no chance of getting consensus around this, then
maintaining the current status quo is my second best choice.

- James

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 20 Nov 2013 06:27:52 +0100

>"subverting the standards process" is thus WAY too premature.

To make this short, permit me to try an analogy. What if Chairs said:

  To reiterate -- some browser folks have stated that they will not be 
  implementing XYZ for WebRTC in their products, so unless they become
  convinced otherwise, there will still be a *market* requirement to
  implement ABC if you want to get the benefit of WebRTC with the
  broadest selection of clients.

Who will argue to make XYZ mandatory-to-implement for WebRTC browsers?

If somebody does, and they convince all but those browser folks that XYZ
should be mandatory-to-implement, but 80% of deployed web browsers end
up not implementing XYZ, then there is no Standards process, just the
browser folks doing whatever they want.

If nobody does because it's obviously pointless to argue, then there is
no Standards process either.

And if the browser folks are not convinced, but submit to the Standards
process, then there would be no "market requirement" for anything else.

Over in the Real-Time Communication in WEB-browsers Working Group the
"browser folks" have, as far as I have been able to follow, comitted to
the Standards process. I expect no less of them, or others, here.

>Please read that as deferring a decision. I'm happy for people to make
>proposals and discuss them. What I'm not willing to do is let a general
>discussion without focus continue for too long and threaten the rest of
>our work. It's quite apparent to me that people are digging in on their
>positions and we can't reach consensus that way.

I appreciate that. I am very interested in the threat model, knowing
what we should protect. I am extremely concerned that users might be-
come unable to audit what their "user agents" send and receive over the
wire if we mandate encryption of HTTP traffic without safeguards. My
suggestion is to work on problem statements before we decide on
solutions.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 20 Nov 2013 17:05:39 +1100

Bjoern,

> the Standards process. I expect no less of them, or others, here.

Your analogy doesn't hold, because we haven't agreed to make anything MTI, encryption or not, and this thread is literally the start of discussion for the latter.   You're also spinning a pretty big hypothetical about a possible future outcome in a different WG with a very different set of constraints, and one that arguably would not represent consensus either. 

The "standards process" is not a means of forcing one's will upon others; that doesn't work, as is proven by the litter of MUSTs and SHOULDs ignored in specs over the years. That's why I said "...unless they become convinced otherwise." Failure to interoperate is one way to convince people, IME; it just takes some time.

Regards,

---

From: Eliot Lear <[email-masked]>
Date: Wed, 20 Nov 2013 07:06:59 +0100

Roberto,

> be sent over a TLS channel.

s/TLS channel/via TLS atop the same port/

?
> channel, it MUST reject these requests with a RST_STREAM or GOAWAY
> with an error code that indicates that the server does not support
> HTTP schemed URLs on port 443.
> -=R
I'm a little concerned about backward compatibility with this approach. 
There do exist many web sites that offer different content today on the
two ports.  Are you suggesting that they would become non-compliant?

Eliot

---

From: Roberto Peon <[email-masked]>
Date: Tue, 19 Nov 2013 22:19:38 -0800

I'm suggesting that IF a site supports HTTP/2 over TLS (generally over
:443, but that isn't required as we know, e.g. https://foo.com:1234/bar):
It MUST accept HTTPS schemed URLS.
It MAY accept HTTP schemed URLS

And:
If a site does accept HTTP schemed URLS, it MUST NOT reject them with a
status code indicating that the client shouldn't retry HTTP schemed URLs
over TLS.
If a site does NOT accept HTTP schemed URLS, it MUST reject them with a
status code indicating that the client shouldn't retry HTTP schemed URLs
over TLS.

This requires nothing new, makes nothing uncompliant.
-=R

On Tue, Nov 19, 2013 at 10:06 PM, Eliot Lear <[email-masked]> wrote:

> There do exist many web sites that offer different content today on the two
> ports.  Are you suggesting that they would become non-compliant?
>
> Eliot
>

---

From: Ilari Liusvaara <[email-masked]>
Date: Wed, 20 Nov 2013 08:21:08 +0200

> two ports.  Are you suggesting that they would become non-compliant?

There is HTTP vs. HTTPS indication in the bytestream itself (:scheme, IIRC).

Also, in some situations, the application protocols might not appreciate if
client and server can't agree on HTTP vs. HTTPS...

There's another problem tho. Many present websites don't like requests for
HTTP URLs over TLS (I tested what one does[1]: It returns a default page with
200 status, Ouch).

[1] That website is using PHP (FastCGI) on top of Apache (mod_ssl).

-Ilari

---

From: Roberto Peon <[email-masked]>
Date: Tue, 19 Nov 2013 22:25:24 -0800

Yup-- as Ilari says, every request is tagged with the scheme.

-=R

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 20 Nov 2013 08:28:26 +0100

> that trying to put plaintext http/2.0 over port 80 will just be an 

Changing the port will require to change the scheme as well otherwise
it will end up being even worse. For example, *right now* over the net
and even much more in corporate networks, you have many applications
running on non-80 ports. So when the browser will have to connect to
"http://foo.bar.tld:8080/", what version will it use ? We could decide
that it will only use HTTP/1 but then we'll have a hard time migrating
everything to 2.0, especially considering that developers are the
primary users of random ports which allow them to put a lot of stuff
and versions on the same development server. So anyway even if we use
another port, we'll need the protocols to be distinguishable from 1.x,
going back to the HTTP Upgrade again. So then a new port will not bring
any benefit (well, just more transparency for now) and will cause other
issues (blocked ports).

Willy

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 20 Nov 2013 17:30:00 +1000

> Yup.

3.4 says "servers supporting HTTP/2.0 are required to support protocol
negotiation in TLS for "https" URIs", so I'm not sure if Roberto's first
sentence is required.  I guess there's a *minor* ambiguity there.

If the second and third sentences are part of an opportunistic encryption
mechanism, I'd suggest that means we can resolve #314 by either keeping the
document as-is or possibly adding the clarifying statement above, and move
the rest to #315.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 08:08:35 +0000

> From: [email-masked]
> To: [email-masked]
> CC: [email-masked]
> Date: Wed, 20 Nov 2013 03:50:23 +0100
> Subject: Re: Call for Proposals re: #314 HTTP2 and http:// URIs on the "open" internet
> 
> * Mark Nottingham wrote:

Hi Bjoern
I cannot know what they meant, but I do know what they stated. It was pretty much, "XXX has no interest in implementing HTTP/2 without TLS", where you can substitute "the Chromium project" and "Mozilla" for "XXX".  There were additional comments about using HTTP/2 as the carrot to get people to deploy TLS universally.
These comments may be what caused the conversation in the last few days to be so heated. It sounded like, "No TLS? Then no HTTP/2 for you!"
Yoav

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 08:12:02 +0000

> From: [email-masked]
> Date: Wed, 20 Nov 2013 14:07:59 +1100
> CC: [email-masked]
> To: [email-masked]
Of course, I would argue "why only for version 2?" if somebody wrote that draft.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 20 Nov 2013 08:13:42 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>Much experience has shown us that MUSTs and SHOULDs are ignored when =
>they're disconnected from implementation needs -- even if those =
>requirements are intended for the greater good.

Until we know how good or bad performance HTTP/2.0 has relative to
HTTP/1.1, it is very hard to gauge how marketable HTTP/2.0 will be
in what segments, and consequently what the implementation needs
will be.

I feel it is premature to try to decide 314, until we know the
protocol we're talking about much better.

That being said, my position is always that attempting to push a
political agenda with technical means is fools errand, and the
end-to-end principle of systems design certainly argues against
attempting to do it with one of the bits in the middle.

So I think we should give people maximum freedom:

1. HTTP/2 on separate port, unencrypted or with per trasaction
   encryption, for maximum performance.

2. HTTP/2 as upgrade on TLS.  People tell us that there is a
   roboust negotiation mechanisem and sufficient transparency on 443.

3. Leave port 80 as HTTP/1 only because there is too much "pidgin
   HTTP" kit out there, and the RTT delay spent on upgrading would
   prevent pure HTTP/2 implementations and delay when HTTP/2 performance
   benefits kicks in.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 08:15:09 +0000

And it seems that I'm in agreement with Roberto, which is... jarring.
Anyway, since this came up several times on the related threads, I think we should have a new issue about HTTP/2 in cleartext: Upgrade dance vs new dedicated port.
There are good arguments for each of these, but I think that defining both adds unnecessary complexity.
Yoav

> From: [email-masked]
> Date: Wed, 20 Nov 2013 15:02:26 +1100
> CC: [email-masked]; [email-masked]; [email-masked]
> To: [email-masked]

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 08:15:41 +0000

+1

> From: [email-masked]
> To: [email-masked]; [email-masked]
> CC: [email-masked]; [email-masked]
> Date: Wed, 20 Nov 2013 04:08:49 +0000

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 20 Nov 2013 08:17:06 +0000

In message <[email-masked]>, Bjoer
>Who will argue to make XYZ mandatory-to-implement for WebRTC browsers?

If XYZ for BLA is desirable for a sufficient large market segment,
the browser vendors will fold.  (See also my email from yesterday.)

I don't think we should try to lay down the law on what people SHALL
implement, only how they MUST implement it, if they choose to.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 20 Nov 2013 08:23:38 +0000

In message <[email-masked]>, Willy Tarreau writes:
>"http://foo.bar.tld:8080/", what version will it use ?

Long time ago I argued that we should look into "no-RTT upgrade", ie
a scheme where the first byte sent from client to server on HTTP/2
would allow the server to decide which protocol it was.

Together with an "Also:" or alt-svc header which announces which
protocols are supported by the server, this could work seamlessly:

	if port == 100
		http/2
	elif we've seen Also: header announcing http/2
		http/2
	else
		http/1

This idea may be worth revisiting.

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 08:23:31 +0000

> Date: Wed, 20 Nov 2013 08:28:26 +0100
> From: [email-masked]
> To: [email-masked]
> CC: [email-masked]; [email-masked]; [email-masked]; [email-masked]

A new scheme means changing stuff at other layers (HTML, links in emails, lots of other places). We could do something like this:
* http://www.example.com  - soon: happy eyeballs.  later: attempts HTTP/2 and fallback if that fails.* http://www.example.com:80 - HTTP/1* http://www.example.com:100 - attempt HTTP/2 and fallback if that fails.* http://www.example.com:xxxxx - soon: begin with HTTP/1, upgrade if you get an alt-svc for the same port.  later on: attempt HTTP/2 and fallback if that fails.
Not pretty, but getting people to migrate to giving out http2:// urls is something I don't think is going to happen.
Yoav

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 20 Nov 2013 10:03:59 +0100

> This idea may be worth revisiting.

Your "Also" header is in fact the Upgrade header :-)

Note that I'm regularly reading here that Upgrade always wastes a
round trip, which is wrong. First, when the browser knows that the
destination supports Upgrade, it can send the payload immediately
after the headers. Second, 101 is an interim response meaning that
the complete response is supposed to follow. So if you send something
like "OPTIONS * HTTP/1.1\r\nUpgrade: HTTP/2.0\r\n..." it will need
a round trip because we won't get any data with 101. But if you send
"GET / HTTP/1.1\r\nUpgrade: 2.0\r\n", the server must reply to this
request and provide the contents. The response can very well look
exactly like a server push in practice, in that from a 2.0 point
of view, the server sends first.

Willy

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Wed, 20 Nov 2013 10:32:11 +0100

Le Mer 20 novembre 2013 04:07, Mark Nottingham a écrit :

Hi Mark

> or similar, even if that spec constrained (or eliminated) using HTTP/2.0

I wouldn't. That would only serve to provide an external varnish of
support for browsers using TLS as a magic pill without actually looking at
problems.

Regards,

-- 
Nicolas Mailhot

---

From: Stefan Eissing <[email-masked]>
Date: Wed, 20 Nov 2013 10:55:14 +0100

Am 20.11.2013 um 09:23 schrieb Yoav Nir <[email-masked]>:
> * http://www.example.com  - soon: happy eyeballs.  later: attempts HTTP/2 and fallback if that fails.
> * http://www.example.com:80 - HTTP/1
> * http://www.example.com:100 - attempt HTTP/2 and fallback if that fails.
> * http://www.example.com:xxxxx - soon: begin with HTTP/1, upgrade if you get an alt-svc for the same port.  later on: attempt HTTP/2 and fallback if that fails.
> 

I think, sooner or later, all "http*://hostname/path" need to point to the same resource. "http" becomes then an umbrella scheme for http1/https/http2/futurehttp. (and "https" the umbrella for all secure variants of http)

It is awkward that services like google need to redirect URIs all the time, just to encrypt the traffic. And they will need to continue doing so when the next, even better, protocol variation comes. It believe it's naive to assume that port 443 will solve all future protocol upgrade scenarios.

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 20 Nov 2013 12:13:25 +0000

>Your "Also" header is in fact the Upgrade header :-)

No, because in addition to protocols, it can also tell you where
to find them (different servers).  Mark has a "alt-svc" draft
which does more or less this.

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 07:59:15 -0500

Grahame,

> upgrade, which is not possible in http/1)

HTTP/1.1 supports negotiated upgrade via the Upgrade and Connection headers. RFC 2817 describes how to use it to upgrade the current hop to TLS.

>> spec, and possibly pointing out to people that it, in fact, no longer

_______________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 08:05:15 -0500

James,

On Nov 19, 2013, at 10:43 PM, James M Snell <[email-masked]> wrote:
> …

My concern with a new port is that how do you specify that http://example.com:8100/somefile.html should use HTTP/2.0?

I’m not convinced that HTTP/2.0 over port 80 is dead just yet - there are some upgrade paths that haven’t been tested with proxies (and boy I would love to see a list of ‘broken’ proxies that I can test with) that might map a path forward.

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 09:46:58 -0500

Poul-Henning,

On Nov 20, 2013, at 3:13 AM, Poul-Henning Kamp <[email-masked]> wrote:
> ...

Again I call for names of proxies that break with HTTP/2.0 (or TLS) upgrade from HTTP/1.1. We (or at least *I*) need to test again them to a) confirm that the reliability issues are as bad as reported and b) determine whether there are alternate strategies that improve things, particularly delayed upgrade, with the client sending an OPTIONS request to do the upgrade after getting a response indicating that the other end supports HTTP/2.0:

    Client                               Server

    GET / HTTP/1.1
    Host: server.example.com
    Upgrade: HTTP/2.0
    . . .

                                         HTTP/1.1 200 OK
                                         Content-Length: 1234
                                         Content-Type: text/html
                                         Upgrade: HTTP/2.0
                                         . . .
                                         <<content>>

    OPTIONS * HTTP/1.1
    Host: server.example.com
    Connection: upgrade
    Upgrade: HTTP/2.0
    HTTP2-Settings: <base64url encoding of HTTP/2.0 SETTINGS payload>

                                         HTTP/1.1 101 Switching Protocols
                                         Connection: upgrade
                                         Upgrade: HTTP/2.0

                                         [ HTTP/2.0 connection ...

This strategy would basically mean that single request clients would only ever use HTTP/1.1, at least for http:// URIs.

Also, the RTT delay for an OPTIONS request from the client to upgrade to HTTP/2.0 will be less than the RTT delay for dozens of TCP connections starting up to grab page resources separately. And doing so is much more server and network friendly, don’t you think?

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 09:55:01 -0500

Poul-Henning,

> would allow the server to decide which protocol it was.

Technically the connection header (section 3.5 of the current draft) looks enough like a HTTP/1.1 request (minus the protocol version) that it *should* work with existing HTTP servers, if only to elicit a 400 response and a closed connection.

I think this is definitely worth investigating, *however* a failed HTTP/2.0 connection will require opening a new TCP connection to retry with HTTP/1.1 with this method.  That could get expensive compared to an HTTP Upgrade approach (see my previous email).

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 09:58:56 -0500

Willy,

> of view, the server sends first.

It is my understanding from prior comments that the GET-with-upgrade method does not work with some/many proxies (I’m still waiting to hear which ones…), so doing a GET that advertises the capability (without the Connection: upgrade header) and following up with an OPTIONS request when the server’s response also includes Upgrade: HTTP/2.0 might be the best we can reliably do (but still a lot better than opening dozens of parallel connections to get the page’s linked resources…)

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 20 Nov 2013 16:11:57 +0100

On Wed, Nov 20, 2013 at 09:58:56AM -0500, Michael Sweet wrote:
> It is my understanding from prior comments that the GET-with-upgrade method
> does not work with some/many proxies (I?m still waiting to hear which ones?),

All those who didn't understand from 2616/2817 how it was supposed to be
used. Haproxy in versions prior to 1.3 was one of those. I've seen a number
of nginx, apache and squid versions not support it either. Packet-based
load balancers are even more affected (eg: Alteon used to be, never tried
versions after 22.x). That said, many of them tend to behave the same way
and simply ignore the Upgrade header without removing it. Those which
rebuild a new request cause no trouble, they just don't support the upgrade
(apache/nginx/squid were in this category and did not hang). Old haproxy
versions would simply let the 101 pass and would either let the rest pass
through or close the connection after the 101 message. Packet based products
are really the worst ones.

> so doing a GET that advertises the capability (without the Connection:
> upgrade header) and following up with an OPTIONS request when the server?s
> response also includes Upgrade: HTTP/2.0 might be the best we can reliably do
> (but still a lot better than opening dozens of parallel connections to get
> the page?s linked resources?)

In my opinion, advertising the capability will not provide anything since
the problematic devices are the non-compliant ones which will ignore your
advertisement. Upgrade easily allows both ends to agree on what they both
support, they just don't know that there's some crap in the middle.

Willy

---

From: Roberto Peon <[email-masked]>
Date: Wed, 20 Nov 2013 08:23:30 -0800

Part of the problem is that it is extremely difficult to find out which
proxies are doing what given that they are implicit(ly somewhere in the
path). There is, however, evidence of this pain amongst folks who attempted
to use WebSockets over port 80.

-=R
On Nov 20, 2013 6:50 AM, "Michael Sweet" <[email-masked]> wrote:

> Again I call for names of proxies that break with HTTP/2.0 (or TLS)
> upgrade from HTTP/1.1. We (or at least *I*) need to test again them to a)
> confirm that the reliability issues are as bad as reported and b) determine
> whether there are alternate strategies that improve things, particularly
> delayed upgrade, with the client sending an OPTIONS request to do the
> upgrade after getting a response indicating that the other end supports
> HTTP/2.0:
>
>     Client                               Server
>
>     GET / HTTP/1.1
>     Host: server.example.com
>     Upgrade: HTTP/2.0
>     . . .
>
>                                          HTTP/1.1 200 OK
>                                          Content-Length: 1234
>                                          Content-Type: text/html
>                                          Upgrade: HTTP/2.0
>                                          . . .
>                                          <<content>>
>
>     OPTIONS * HTTP/1.1
>     Host: server.example.com
>     Connection: upgrade
>     Upgrade: HTTP/2.0
> This strategy would basically mean that single request clients would only
> ever use HTTP/1.1, at least for http:// URIs.
>
> Also, the RTT delay for an OPTIONS request from the client to upgrade to
> HTTP/2.0 will be less than the RTT delay for dozens of TCP connections
> starting up to grab page resources separately. And doing so is much more
> server and network friendly, don’t you think?
>

---

From: Piotr Galecki <[email-masked]>
Date: Wed, 20 Nov 2013 16:36:01 +0000

I've been following this very interesting discussion as an observer.
We all agree that since HTTP/2.0 message formats are not backwards compatible with HTTP/1.x
the proxies processing HTTP/2.0 requests will cause problems and break end-to-end HTTP/2.0.
In most networks this is a temporary situation though.
Eventually the proxies in the HTTP request processing path will get software updates and they will get support for HTTP/2.0.
Once that happens end2end HTTP/2.0 in plaintext would be possible.

I wonder if some detection mechanism could be added to HTTP/2.0 protocol to detect HTTP/1.x proxies between client and server.
If HTTP/1.x proxy is detected HTTP communication would not be eligible for upgrade to HTTP/2.0.
If no proxies or only HTTP/2.0 proxies are detected the connection can be upgraded to HTTP/2.0.
HTTP/1 detection should always be done first before switching to HTTP/2.

Alternatively a client would attempt HTTP/2 connection first and if it fails
it would roll back to HTTP/1.1 connection.

Also some HTTP/2.0 over HTTP/1.x proxy traversal mechanism could be designed.
Possibly using HTTP/1.x CONNECT.
This would work like this:
* run 1.x proxy detection
* if 1.1 proxy detected issue HTTP/1.1 CONNECT
* run HTTP/2.0 over HTTP/1.1 CONNECT session

Thanks,
Piotr

-----Original Message-----
From: Willy Tarreau [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2013 10:12 AM
To: Michael Sweet
Cc: Poul-Henning Kamp; Adrien de Croy; James M Snell; Mark Nottingham; Bjoern Hoehrmann; HTTP Working Group
Subject: Re: Call for Proposals re: #314 HTTP2 and http:// URIs on the "open" internet

On Wed, Nov 20, 2013 at 09:58:56AM -0500, Michael Sweet wrote:
> > the complete response is supposed to follow. So if you send 
> > something like "OPTIONS * HTTP/1.1\r\nUpgrade: HTTP/2.0\r\n..." it 
> > will need a round trip because we won't get any data with 101. But 
> > if you send "GET / HTTP/1.1\r\nUpgrade: 2.0\r\n", the server must 
> > reply to this request and provide the contents. The response can 
> > very well look exactly like a server push in practice, in that from 
> > a 2.0 point of view, the server sends first.
> 
> It is my understanding from prior comments that the GET-with-upgrade 
> method does not work with some/many proxies (I?m still waiting to hear 
> which ones?),

All those who didn't understand from 2616/2817 how it was supposed to be used. Haproxy in versions prior to 1.3 was one of those. I've seen a number of nginx, apache and squid versions not support it either. Packet-based load balancers are even more affected (eg: Alteon used to be, never tried versions after 22.x). That said, many of them tend to behave the same way and simply ignore the Upgrade header without removing it. Those which rebuild a new request cause no trouble, they just don't support the upgrade (apache/nginx/squid were in this category and did not hang). Old haproxy versions would simply let the 101 pass and would either let the rest pass through or close the connection after the 101 message. Packet based products are really the worst ones.

> upgrade header) and following up with an OPTIONS request when the 
> server?s response also includes Upgrade: HTTP/2.0 might be the best we 
> can reliably do (but still a lot better than opening dozens of 
> parallel connections to get the page?s linked resources?)

In my opinion, advertising the capability will not provide anything since the problematic devices are the non-compliant ones which will ignore your advertisement. Upgrade easily allows both ends to agree on what they both support, they just don't know that there's some crap in the middle.

Willy

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 12:38:23 -0500

Piotr,

> the proxies processing HTTP/2.0 requests will cause problems and break end-to-end HTTP/2.0.

I think “we all agree” is a prevailing assumption, but given my experience with HTTP Upgrade to TLS for CUPS I don’t think all hope is lost.

Instead of assuming that it won’t work because WebSockets failed, I am now trying to collect real information and do some testing to see what exactly doesn’t work and why.  If we can find a path that allows clients to upgrade successfully or detect when it won’t work then we will have both the reliability needed to deploy it AND the information needed to document the failure modes for the spec.

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 14:17:32 -0500

Willy,

> support, they just don't know that there's some crap in the middle.

Well, if the server’s Upgrade header is filtered, and/or if the server only includes the Upgrade header if the client (or intermediate proxy) supplies it, then we don’t have an interoperability/compatibility problem - with my proposed delay-upgrade strategy the client only attempts to upgrade when the response from the server indicates that it supports HTTP/2.0.

I’ll see what I can find out with the proxies you mentioned to determine what level of functionality is available with current software - broken, HTTP/1.1 only, or compatible w/HTTP/2.0.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 20 Nov 2013 20:29:04 +0100

On Wed, Nov 20, 2013 at 02:17:32PM -0500, Michael Sweet wrote:
> Well, if the server?s Upgrade header is filtered, and/or if the server only
> includes the Upgrade header if the client (or intermediate proxy) supplies
> it, then we don?t have an interoperability/compatibility problem -

I know, that's the easiest case, which happens with 1.1-compliant products.

> with my
> proposed delay-upgrade strategy the client only attempts to upgrade when the
> response from the server indicates that it supports HTTP/2.0.

Ah OK I didn't catch what you meant. Sure this is what is needed at least for
the first connection attempt. If the browser keeps a cache of successful
destinations, it can later decide not to wait for the response and upgrade
right after the headers. I believe some browsers (Opera?) did this for
pipelining, though I may be wrong.

> I?ll see what I can find out with the proxies you mentiogned to determine
> what level of functionality is available with current software - broken,
> HTTP/1.1 only, or compatible w/HTTP/2.0.

At the moment, to the best of my knowledge, none of them is broken. We tried
hard to get some failure cases during the work on WebSocket, and we ended up
with this upgrade precisely because it failed cleanly with the software we
had access to.

I remember one issue was reported with an anti-virus software running on the
client, f-secure something I believe but I could be wrong. It did the dirty
thing, ignore Upgrade, pass it silently and expect data after the 101 without
letting anything else pass through.

We should keep in mind that when we talk about websocket failures, we indeed
talk about failures to Upgrade to WS whatever the cause. We'll have the same
failure rate, except that in our case, processing the 1.1 request normally
is the expected fallback method (it was not acceptable for WS).

Regards,
Willy

---

From: Yoav Nir <[email-masked]>
Date: Wed, 20 Nov 2013 21:50:26 +0200

> Instead of assuming that it won’t work because WebSockets failed, I am now trying to collect real information and do some testing to see what exactly doesn’t work and why.  If we can find a path th…
Hi Michael

That info will be interesting. I worry, though, that it's a huge 
undertaking to get a complete picture because of the long tail.

A little over four years ago Marsh Ray and Martin Rex independently 
found a security problem in TLS (all versions). Marsh, acting 
responsibly, talked to the TLS WG chairs, and together they convened a 
secret meeting of the implementers of the major SSL libraries, because 
really, how many SSL libraries can there be out there? So they had 
Microsoft, and OpenSSL, and NSS, and Opera, maybe a few others (we never 
got the full details on those meetings).

When the story broke, it turned out that there were dozens of SSL 
libraries. We have one, SAP has one. There's a few open source ones. 
Fixing the problem required fixing all of them, so it could not be done 
in secret, even if it hadn't broke before they planned it. SSL libraries 
are obscure things. If you look at HTTP clients, there's also dozens of 
libraries.

Proxies are probably the worst of the bunch. If we define proxy as any 
middlebox that does anything to the HTTP protocol other than just pass 
packets along (I like to think of them as broken routers), I'm sure 
there's over a hundred products and versions out there. Some are 
obsolete. Some haven't been patched in years. Some are discontinued 
products that are out of service. Some are by companies that have gone 
out of business. Any one of them might fail in an unpredictable way, and 
you won't be able to test them all.

Any one of them might respond might forward the Upgrade header, and then 
when seeing something unexpected (like HTTP/2), lose state for the 
connection without sending any RST or FIN. The user experience from this 
is a blank window waiting several seconds for a timeout. This won't 
happen all the time. It probably won't happen 95% of the time. But the 
browser vendors are worried for a reason.

Yoav

---

From: Michael Sweet <[email-masked]>
Date: Wed, 20 Nov 2013 15:12:34 -0500

Yoav,

On Nov 20, 2013, at 2:50 PM, Yoav Nir <[email-masked]> wrote:
> ...
> That info will be interesting. I worry, though, that it's a huge undertaking to get a complete picture because of the long tail.

I’m less concerned about getting a 100% complete picture and more wanting to get a general picture based on proxies that are popular and/or known to have issues.

I am under no illusion (delusion? :) that we will be able to do HTTP/2.0 upgrade over current proxies.

Rather, I want to know whether a HTTP/2.0 client can successfully continue to function with HTTP/1.1 proxies - can we reliably either a) know that we can attempt an upgrade or b) recover from a failed upgrade?

Answering those questions will determine whether it is feasible to support plain text HTTP/2.0 on port 80 and/or whether we would really need a new URI scheme to differentiate between HTTP/1.x and HTTP/2.0.  I’m hopeful that the answer is actually *yes* since the “failure” mode for HTTP/2.0 upgrades is just using HTTP/1.1, vs. WebSockets and other similar extensions that rely on upgrade to work at all.

In short (and I apologize for paraphrasing): Failure IS an option.  We just need to know *how* HTTP/2.0 upgrade will fail to determine if it would prevent implementors from supporting plain text HTTP/2.0 over the Internet.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 21 Nov 2013 10:12:11 +1100

<https://github.com/http2/http2-spec/issues/1>

To make sure everybody is in sync -- We've been discussing HTTP/1 (without TLS) negotiation for some time.

Currently, the draft documents one method of doing this, using Upgrade:
  <http://http2.github.io/http2-spec/#discover-http>

Recently, we've also been discussing using Alt-Svc for this (as distinct from the opportunistic encryption case):
  <http://tools.ietf.org/html/draft-nottingham-httpbis-alt-svc-00>
  
Both of these were discussed in the Vancouver meeting:
  <http://tools.ietf.org/wg/httpbis/minutes?item=minutes-88-httpbis.html>
(search for "#### Issue 1: Upgrade Mechanism")

There, we stated an intent to leave the Upgrade mechanism in and get implementer experience, and also reference Alt-Svc and get experience with that as well. Neither is required.

We've also talked in the past about DNS-based negotiation, but we haven't yet landed on an approach that we've seen implementer interest in (although Patrick has said that he wants to do some experimentation and bring the results back to us).

I would characterise the overall approach as documenting the fewest number of negotiation mechanisms we think are necessary, acknowledging that there are a variety of use cases with HTTP with potentially different requirements. 

So, just a suggestion -- it would probably be most productive if you have a look at those and make proposals for changes / questions, etc. based upon those, since they reflect the current state of discussion. If you don't think they're the right approaches, you might want to consider writing an I-D proposing a different one.

Cheers,

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 20 Nov 2013 23:46:08 +0000

there are a lot of WinGate versions out there that will ignore (pass not 
strip) Upgrade on outbound, and wait for a message after the 101 back. 
They will in general I think pass back the 101, but probably prevent 
data flowing properly in both directions after that.

We need the ability in writing up RFCs to be able to put 
sections/text/warnings in bold and red so that I would have noticed 
this.  I guess I'm a lazy RFC-reader.  In fact in 1995, when I did the 
initial proxy, I didn't read any spec at all.  Maybe there was none.

How many people that impacts I have no idea sorry.  there was a free 
version for many years.  I'd be inclined to suggest the old ones are 
infrequently used now, but they still pop up at a rate higher than I 
would expect.

I'm also wondering if we may be facing a similar issue with port 443 and 
http/1.1 over TLS.

As more and more MITMs roll out, we're going to have issues with things 
intercepting port 443, and expecting to see http/1.1 not anything else.  
I'd expect this to break NPN/ALPN so maybe that's not an issue (fail 
safe).

I do wonder though, with negotiated crypto (e.g. "opportunistic") 
whether we couldn't use port 100 for both http:// and https://

E.g. the base protocol supports negotiated (optional) crypto, and just 
have different requirements for whether crypto is required based on the 
scheme in the URI.  I think the STARTTLS / STLS etc model in 
SMTP/POP3/IMAP works pretty well.  But others may not agree with this.  
Since we're talking about some initial settings negotiation, crypto for 
an entire channel or for streams becomes advertiseable and selectable.

Adrien

------ Original Message ------
From: "Michael Sweet" <[email-masked]>
To: "Yoav Nir" <[email-masked]>
Cc: "HTTP Working Group" <[email-masked]>
Sent: 21/11/2013 9:12:34 a.m.
Subject: Re: Call for Proposals re: #314 HTTP2 and http:// URIs on the 
"open" internet
>I’m less concerned about getting a 100% complete picture and more 
>wanting to get a general picture based on proxies that are popular 
>and/or known to have issues.
>
>Rather, I want to know whether a HTTP/2.0 client can successfully 
>continue to function with HTTP/1.1 proxies - can we reliably either a) 
>know that we can attempt an upgrade or b) recover from a failed 
>upgrade?
>
>Answering those questions will determine whether it is feasible to 
>support plain text HTTP/2.0 on port 80 and/or whether we would really 
>need a new URI scheme to differentiate between HTTP/1.x and HTTP/2.0. 
>I’m hopeful that the answer is actually *yes* since the “failure” mode 
>for HTTP/2.0 upgrades is just using HTTP/1.1, vs. WebSockets and other 
>similar extensions that rely on upgrade to work at all.
>
>In short (and I apologize for paraphrasing): Failure IS an option. We 
>just need to know *how* HTTP/2.0 upgrade will fail to determine if it 
>would prevent implementors from supporting plain text HTTP/2.0 over the 
>Internet.
>

---

From: Ilari Liusvaara <[email-masked]>
Date: Thu, 21 Nov 2013 09:52:10 +0200

> pipelining, though I may be wrong.

Actually, isn't the "global" upgrade case pretty much all-or-nothing (w.r.t
severs that support HTTP/2.0)? That is, middleware either screws pretty much
any upgrade or it screws up none ("local" destinations may not go through it)?

> with this upgrade precisely because it failed cleanly with the software we

The failure modes that spring to mind (not that I know any products that fail
in these ways):

- Upgrade results 2xx response (HTTP/1.1).
- Upgrade results 3xx response (HTTP/1.1).
- Upgrade results 4xx error (HTTP/1.1).
- Upgrade results 5xx error (HTTP/1.1).
- FIN (EOF) after upgrade attempt.
- RST (ECONNRESET) after upgrade attempt.
- The connection hangs after upgrade attempt.
- 101 response is given but the connection then hangs.
- 101 response followed by server response is given, half-link to server is
  hung.
- 101 response is given, followed by HTTP/1.1 4xx/5xx error.

Of course, there might be other failure cases too...

The "hung" cases are the nastiest here. Pretty all other responses should
come very quickly (on order of few tens of milliseconds, except the
ones involving 101, which might be on order of few hundred milliseconds).

-Ilari

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 21 Nov 2013 09:20:16 +0100

Hi Ilari,

> any upgrade or it screws up none ("local" destinations may not go through it)?

I don't understand what you mean. A given middlebox will generally screw up all
similarly-behaving connections the same way. However not all middleboxes will
screw up the same way. For example if a middlebox prevents queued data from the
client to flow to the server, it will only cause trouble to the clients which
don't do a round trip first.

> - Upgrade results 5xx error (HTTP/1.1).

The way you describe it is ambiguous here, because these results
may either be the valid ones from the origin server or may be emitted
by the middlebox itself. I expect most deployed products to result in
this above, where the Upgrade header field is removed and the response
comes in 1.1 from the origin server.

> - FIN (EOF) after upgrade attempt.

Yes, generally just after the 101 response headers. I don't expect
to see many of them over the internet though, because this would
come from those which do not support 101 nor keep-alive, and the
latter is not realistic over the internet as an intercepting proxy,
since it degrades performance.

> - RST (ECONNRESET) after upgrade attempt.

Can also come from those with a non-compliant IPS installed in front.

> - The connection hangs after upgrade attempt.

That can happen if content-length is advertised by the client. This
is a good reason for avoiding to start the upgrade with a POST/PUT.
I believe this is noted somewhere in the draft/wiki or was discussed
here.

> - 101 response is given but the connection then hangs.

This one can happen when 101 is not understood as an interim response.
The middlebox considers it final, sees no content-length and expects
to read more until the server closes.

> - 101 response followed by server response is given, half-link to server is

For me it is the same as above.

> - 101 response is given, followed by HTTP/1.1 4xx/5xx error.

Yes when the client pushes data to the server, believing the channel
is clean. Generally a "400 bad request" or a "408 request timeout"
can happen if the middlebox waits for a second valid request. This
is the reason why in WS I preferred that we put "connection:close"
in the exchanges (to incite middleboxes to close when non-compliant),
but since I failed to make up that specific case again, I could not
defend it anymore :-)

> ones involving 101, which might be on order of few hundred milliseconds).

Yes absolutely.

Regards,
Willy

---

From: Ilari Liusvaara <[email-masked]>
Date: Thu, 21 Nov 2013 11:34:01 +0200

> don't do a round trip first.

That's what I meant. Different middleboxes screw things up in different way, but
generally the same middlebox screws everything passing through it the same way.
 
> but since I failed to make up that specific case again, I could not

The process I considered to cause this was more like:

The middlebox passes the upgrade but doesn't process it (also passing
the 101). Thinks that the connection is still HTTP/1.1. Then the client
sends connnection magic, which of course causes things blows up...

Yeah, really broken proxy...

-Ilari

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 21 Nov 2013 10:46:25 +0100

> sends connnection magic, which of course causes things blows up...

We're in sync. But if the 101 response contained "connection: close",
the middlebox would most often propagate the close to the client and
refrain from reading anything else. It's not rocket science though.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Nov 2013 14:14:05 +1100

How about changing the Subject: on this thread?

Thanks,

---

From: Martin Thomson <[email-masked]>
Date: Fri, 22 Nov 2013 10:05:28 -0800

>> refrain from reading anything else. It's not rocket science though.

Wasn't there some guidance that came out of the websockets work to
this effect already?  I believe that Connection:close was something
that was found to be useful in both directions.  Sadly, the 101 can
sometimes be interpreted as being informational by the intermediary,
whereupon it has trouble identifying the final response.  Adding a
server prologue that starts with 5xx GTFO\r\nConnection:close\r\n\r\n
might help, but is that overkill?

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 22 Nov 2013 22:11:13 +0100

On Fri, Nov 22, 2013 at 10:05:28AM -0800, Martin Thomson wrote:
> that was found to be useful in both directions.

Yes I remember having discussed about this to great extents but I don't
remember the outcome.

> Sadly, the 101 can
> whereupon it has trouble identifying the final response.

Yes if the intermediary considers it as equivalent to 100 since the spec
suggests so (fallback to the x00 code of the same class). I'm not aware
of any doing that but surely they do exist. Well... just checked and
haproxy used to do this between versions 1.3.21 and 1.3.23 inclusive,
4 years ago. So it definitely does exist.

> Adding a

I'm not totally convinced in fact. On the one hand, returning something
not acceptable by a parser might help it reject the message and close. On
the other hand, some parsers might consider this is an HTTP/0.9 response
and forward until the connection closes (which might be good or bad,
depending whether the request channel is still forwarding or not). Still,
I suspect that an unparsable response might be enough without being
overkill (eg: "HTTP2!\r\n").

Another possibility is to rely on "HTTP/" 1*DIGIT "." 1*DIGIT SP 3DIGIT SP
as suggested in RFC1945 to detect 0.9 vs later versions, and fail just
after with an error code and some non-parsable contents :

     HTTP/2.0 500 \x00\r\n\x00\r\n\r\n

The most permissive ones would probably close on "2.0", "500" or the lack
of CRLF before \x00 or lack of CRLF. The most compliant ones will probably
abort on \x00 in the header name.

I thought that a valid HTTP response such as "HTTP/2.0" with an error code
and an invalid content-length might also have chances of being caught, but
looking at the old haproxy versions above, it will pass unmolested !

Regards,
Willy