Subject: workability (or otherwise) of HTTP upgrade

From: Greg Wilkins <[email-masked]>
Date: Fri, 26 Nov 2010 16:06:41 +1100

I'm cross posting this message to both the hybi (websockets) and
http-bis mailing list as I think this is an issue that is very
relevant to both groups.

The hybi/websocket protocol, as currently proposed, has a handshake
mechanism that is roughly based on the HTTP upgrade mechanism.
Progress in the hybi/websocket mailing has ground to a halt because we
appear unable to get a clear consensus between the two advocated ways
forward:

 1) Make the "roughly based" HTTP upgrade mechanism a fully compliant
HTTP upgrade.
 2) Move away from HTTP upgrade and use another mechanism (potentially
CONNECT or SSL extensions)

To paraphrase the arguments against 1), there are some that are
concerned that an Upgrade based handshake will never be able to be
adequately secured against cross protocol attacks from ws-browsers to
non ws-servers. Also there is some concern that intermediaries might
not well support Upgrade and that other mechanisms might have a higher
success rate.

So I thought that it would be good to move that discussion from hybi
to HTTP-bis so we can stall progress here.... no I mean so that we can
get a wider view on the capabilities and vulnerabilities that might
apply to Upgrade.

My understanding is that Upgrade was included in HTTP for precisely
the reason that Websocket wishes to use it - ie to take an existing
HTTP connection and to upgrade the protocol run over the connection to
something with different capabilities than HTTP.
I would also expect that if there are problems with lack of Upgrade
support in intermediaries, then it would be easier to get
intermediaries to be updated to a standard generic HTTP mechanism
rather than some special purpose usage of CONNECT.

If websocket is unable to securely use Upgrade, then there must be
some fundamental flaw in Upgrade that should either be fixed in
httpbis (if allowed by the charter).   If Upgrade cannot be fixed,
then perhaps httpbis needs to somehow deprecate the mechanism and we
can look together for alternatives?

So I'd ask the httpbis readers, are there any reasons you know of that
would prevent Upgrade being used for websocket?
And I'd ask the hybi upgrade sceptics if they could perhaps voice
their concerns about Upgrade in more detail than my paraphrasing
above.

regards

---

From: Adam Barth <[email-masked]>
Date: Thu, 25 Nov 2010 22:54:07 -0800

Using Upgrade for WebSockets is insecure in practice.  My colleagues
and I have run an experiment using live traffic on the Internet and
have successfully exploited a number of users using an Upgrade-based
handshake (our exploit is harmless by proves the concept).  I'm still
getting clearance from some affected vendors before releasing a report
on the experiment publicly.  It's looking like I'll be able to release
the report within a week.

Upgrade might well be useful for other purposes, and I think
definition in HTTP is fine.  The problem is that not everyone
implements Upgrade properly, which leaves room for an attack to
leverage an Upgrade-based WebSockets handshake in attacks.

Kind regards,
Adam

On Thu, Nov 25, 2010 at 9:06 PM, Greg Wilkins <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Eric J. Bowman" <[email-masked]>
Date: Fri, 26 Nov 2010 00:03:52 -0700

> CONNECT or SSL extensions)

What about:  3) Use another port?  I've seen that opinion voiced, so I
haven't signed up to the hybi list to voice it myself.  No criticism of
Web Sockets is intended or implied here, it's just a concrete example of
something which increasingly concerns me regarding Upgrade.

> something with different capabilities than HTTP.

This is my understanding, also.  The problems hybi has run into make me
question whether this is at all wise -- much easier to secure port 80 if
Internet Media Types are required to be exposed in some sort of header,
using the request/response paradigm.  Having some notion of a processing
model expressed in a header externally from the data dates back to
Gopher; using media types enabled the proliferation of intermediaries
based on the resource/representation paradigm.

Shouldn't a line be drawn between what's appropriate for port 80, and
what constitutes an entirely different protocol?  Or is the intent of
Upgrade to deprecate registering new URI schemes in favor of a registry
of HTTP tokens?

> can look together for alternatives?

It's perhaps fundamentally flawed for HTTP to be the handshake mechanism
for protocols that are not even remotely similar to HTTP.  In this case,
we have a half-duplex, request/response, resource/representation
protocol establishing a full-duplex, asynchronous, opaque-bits-on-the-
wire connection that's fundamentally incompatible with the deployed
architecture of port 80 (caches, security gateways); and which nobody
seems happy with.

My suggested fixes to 9.8, based on the notion that HTTP 1.0, 1.1
and 2.0, and PTTH 1.0, are not "incompatible" protocols:

"The 'Upgrade' general-header field allows the client to specify which
version of the communication protocol it would like to use, if the
server chooses to switch versions.  Additionally, the server MUST use
the Upgrade header field within a 101 (Switching Protocols) response to
indicate the version(s) being switched to.

The Upgrade header field is intended to provide a simple mechanism for
transitioning from HTTP/1.1 to some newer, compatible protocol...  This
eases the difficult transition between versions of compatible protocols
by allowing the client to initiate a request..."

This opens a whole can of worms about what constitutes a compatible
protocol, instead of the current wording:  "The capabilities and nature
of the application-layer communication after the protocol change is
entirely dependent upon the new protocol chosen," which seems like the
sort of free-for-all that leads to unintended security consequences.

My issue is whether it's overreaching to extend Upgrade from being a
versioning mechanism for HTTP, into a general-purpose protocol tunneling
mechanism.  Or securable.  Does a registry need to be defined here, and
if so, shouldn't there be some fidelity with the notions of the media
type and request/response architectural paradigms?

> would prevent Upgrade being used for websocket?

Yes -- Web Sockets is fundamentally different from anything I'd expect
to see on port 80.  The fact that this is exactly what Upgrade allows,
even encourages with the proposed registry, gives me the heebie-jeebies.

-Eric

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 26 Nov 2010 23:55:58 +1100

> something which increasingly concerns me regarding Upgrade.

The problem with another port, is that the success rate of  opening an
arbitrary port through firewalls is not that high.     Thus if
websocket was allocated it's own sockets, then there would still be
need for a websocket over 80 protocol (eg like there is BOSH for
XMPP).

I would like to see a dedicated port for websocket so that over time
it could be opened as needed, but I think there will be a desire to
function over 80 for some time to come.     This could be done with
some enhanced HTTP (eg SPDY), but I'm not sure if such a protocol
would match all the features of websocket (eg improved data density),
nor fit with your HTTP-like restrictions described below.

> sort of free-for-all that leads to unintended security consequences.

Indeed.  Is HTTP over SSL a compatible protocol? semantically yes, but
it is fundamentally different on the wire.

Would this restriction also apply to protocols like SPDY that are
enhanced HTTP-like protocols, but with additional features.

> type and request/response architectural paradigms?

RFC2817 would appear to have gone beyond that already.

> even encourages with the proposed registry, gives me the heebie-jeebies.

And do you get similar feeling to think about using the CONNECT method
to establish tunnels for arbitrary protocols?

cheers

---

From: Julian Reschke <[email-masked]>
Date: Fri, 26 Nov 2010 14:33:23 +0100

On 26.11.2010 08:03, Eric J. Bowman wrote:
> ...
> ...

That would be material for "Upgrade Token Registry" section...

> ...
> ...

Not convinced.

If the protocol is compatible, you don't need "Upgrade" in the first place.

So, HTTP/2.0 *will* need upgrade, if you start the conversation using 
HTTP 1.*. See 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-12.html#rfc.section.2.5>:

"HTTP uses a "<major>.<minor>" numbering scheme to indicate versions of 
the protocol. The protocol versioning policy is intended to allow the 
sender to indicate the format of a message and its capacity for 
understanding further HTTP communication, rather than the features 
obtained via that communication. No change is made to the version number 
for the addition of message components which do not affect communication 
behavior or which only add to extensible field values. The <minor> 
number is incremented when the changes made to the protocol add features 
which do not change the general message parsing algorithm, but which 
might add to the message semantics and imply additional capabilities of 
the sender. The <major> number is incremented when the format of a 
message within the protocol is changed."

> ...
> ...

Could you elaborate in these security consequences?

 > ...

Best regards, Julian

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 29 Nov 2010 17:53:57 -0700

Greg Wilkins wrote:
> 
> > What about:  3) Use another port?  I've seen that opinion voiced,
> > so I haven't signed up to the hybi list to voice it myself.  No
> > criticism of Web Sockets is intended or implied here, it's just a
> > concrete example of something which increasingly concerns me
> > regarding Upgrade.
> 
> arbitrary port through firewalls is not that high.
>

Uptake is always a problem with new protocols.  I don't see how this
justifies breaking the security architecture of port 80 (by eliminating
media-type filtering).

>
> Thus if websocket was allocated it's own sockets, then there would
> still be need for a websocket over 80 protocol (eg like there is BOSH
> for XMPP).
> 

Perhaps.  XMPP over BOSH looks like HTTP to me, though.

> function over 80 for some time to come.
>

Just because there's a desire for new protocols to work with existing
URLs, doesn't mean it's a good idea.  There's also a desire to keep
port 80 securable, which gets harder each time another dissimilar
protocol is factored in.  IMO, the latter desire outweighs the former.

>
> This could be done with some enhanced HTTP (eg SPDY), but I'm not
> sure if such a protocol would match all the features of websocket (eg
> improved data density), nor fit with your HTTP-like restrictions
> described below.
> 

Most of the problems Web Sockets addresses, seem solveable to me by
adding a new method(s) to HTTP (say, IDLE).  I'm not convinced that
improved data density, at the expense of cacheability, results in a net
performance gain -- the way to improve performance is to reduce the
instances of communication over the network (i.e. caching).  Reducing
header overhead via reduced verbosity makes sense; eliminating headers,
thus forcing tunneled communications, does not.

> 
> > It's perhaps fundamentally flawed for HTTP to be the handshake
> > mechanism for protocols that are not even remotely similar to HTTP.
> >  In this case, we have a half-duplex, request/response,
> > resource/representation protocol establishing a full-duplex,
> > asynchronous, opaque-bits-on-the- wire connection that's
> > fundamentally incompatible with the deployed architecture of port
> > 80 (caches, security gateways); and which nobody seems happy with.
> >
> > "The 'Upgrade' general-header field allows the client to specify
> > which version of the communication protocol it would like to use,
> > if the server chooses to switch versions.  Additionally, the server
> > MUST use the Upgrade header field within a 101 (Switching
> > Protocols) response to indicate the version(s) being switched to.
> >
> > The Upgrade header field is intended to provide a simple mechanism
> > for transitioning from HTTP/1.1 to some newer, compatible
> > protocol...  This eases the difficult transition between versions
> > of compatible protocols by allowing the client to initiate a
> > request..."
> >
> > protocol, instead of the current wording:  "The capabilities and
> > nature of the application-layer communication after the protocol
> > change is entirely dependent upon the new protocol chosen," which
> > seems like the sort of free-for-all that leads to unintended
> > security consequences.
> 

It looks like HTTP to participants in the communication, gobbledygook
to others.  As opposed to another protocol which, when decrypted, isn't
remotely similar to HTTP.  So I meant semantically, as opposed to on-
the-wire; IOW, I don't see TLS as being germane to my point.

> enhanced HTTP-like protocols, but with additional features.

Solving the multiplexing problem with HTTP over SCTP seems like better
engineering to me.  The Web's continuing success has to do with the
shared-intermediary architecture that's been deployed for port 80.  I'm
opposed to any protocol meant for port 80 which precludes this option.
HTTP+TLS is *optional* tunneling.  Web Sockets and SPDY are *mandatory*
tunneling.  Caching scaled the Web, that success should be built upon,
not abandoned as obsolete (particularly absent any evidence indicating
this to be so).

> > versioning mechanism for HTTP, into a general-purpose protocol
> > tunneling mechanism.  Or securable.  Does a registry need to be
> > defined here, and if so, shouldn't there be some fidelity with the
> > notions of the media type and request/response architectural
> > paradigms?
> 

How so?  It's still request/response, and a user-agent may still be
configured to, for example, not render application/pdf even if it's
tunneled through a security firewall which ordinarily blocks PDF on
port 80.  Serving JSON as text/html is a known security problem, i.e.
context escaping.  If an HTML page sets this context, and port 80 is
used to fetch some JSON, how do I change the context if there's no
media type?  HTTPS, HTTP+TLS, XMPP over BOSH and so forth, don't have
this problem; whereas I don't see how Web Sockets avoids it without
being on its own port.

>
> >> So I'd ask the httpbis readers, are there any reasons you know of
> >> that would prevent Upgrade being used for websocket?
> >>
> >
> > Yes -- Web Sockets is fundamentally different from anything I'd
> > expect to see on port 80.  The fact that this is exactly what
> > Upgrade allows, even encourages with the proposed registry, gives
> > me the heebie-jeebies.
> 

Yes.

-Eric

---

From: "Eric J. Bowman" <[email-masked]>
Date: Mon, 29 Nov 2010 18:29:41 -0700

> If the protocol is compatible, you don't need "Upgrade" in the first

Two issues here.  One, is whether Upgrade should be a general-purpose
protocol tunneling mechanism vs. an HTTP-specific versioning mechanism.
Two, is how to change the wording.  There are probably specific terms
which describe exactly what I'm getting at, in languages other than
English (you Germans probably have an absurdly-long compound word that
captures exactly what I'm trying to say :-).

> HTTP 1.*.

Of course.  I'm not arguing against Upgrade as a versioning mechanism
for HTTP and HTTP-ish protocols.  I can't imagine that HTTP 2.0 would
abandon the request/response or resource/representation paradigms,
which I'm suggesting be a requirement for protocols bound to port 80.

> > security consequences. ...
> 

I mentioned context escaping in my last post to this thread, but I'm
really making more of a common-sense argument here.  How does one craft
potential exploits for protocols nobody's thought of yet?  Conversely,
how do we know it won't be possible?  It seems rational to me, that if
the security of the deployed architecture for port 80 is based on media
types, secure interoperability would be unlikely with protocols which
don't use media types.

-Eric

---

From: Julian Reschke <[email-masked]>
Date: Tue, 30 Nov 2010 12:03:47 +0100

> captures exactly what I'm trying to say :-).

:-)

> which I'm suggesting be a requirement for protocols bound to port 80.

It's not clear to me where you want to draw the line.

- Uprade to TLS is already defined (although not really used), and 
changes the message framing

- Things like SPDY and/or Waka will have binary headers, so will have no 
on-the wire resemblance to HTTP messages

- I can easily imagine that HTTP/2.0 could add server-initiated messages.

If these things qualify as HTTP/2.0 features, why not allow other 
protocols to use same mechanism?

Best regard, Julian

---

From: "Eric J. Bowman" <[email-masked]>
Date: Tue, 30 Nov 2010 06:24:25 -0700

Julian Reschke wrote:
>
> >
> > Of course.  I'm not arguing against Upgrade as a versioning
> > mechanism for HTTP and HTTP-ish protocols.  I can't imagine that
> > HTTP 2.0 would abandon the request/response or
> > resource/representation paradigms, which I'm suggesting be a
> > requirement for protocols bound to port 80. ...
> 

It isn't clear to me whether a line should be drawn, merely that it's
worth taking a moment to ponder the implications of not drawing one.

> changes the message framing

I was wondering about that; I've never come across it in the wild, is
there any data?

>
> - Things like SPDY and/or Waka will have binary headers, so will have
> no on-the wire resemblance to HTTP messages
> 

While adhering to the same architectural principles, primarily resource/
representation instantiation via media types.

> - I can easily imagine that HTTP/2.0 could add server-initiated

To which I can easily imagine a client responding.

> protocols to use same mechanism?

These things are all vaporspecs, right?  So it isn't too late to think
about some minimal guidelines for protocols desiring to use Upgrade, to
keep port 80 manageable.  Whatever family of protocols interoperate
around the http: scheme should have *something* in common, i.e. using
media types, as a basis for that interoperability being securable.

-Eric