Subject: [hybi] Do we have a sense of when browsers will be supporting draft-ietf-hybi-thewebsocketprotocol-03 ?

From: Matthew Butler <[email-masked]>
Date: Wed, 1 Dec 2010 09:00:37 -0800

We were doing work based on Draft-76 but have recently seen the changes going on in the spec, and we're wondering how/what the migration path from the current state of the browsers to one supporting something along the lines of the current draft will be.

Thoughts ?

Thanks
Matt Butler
Principal Scientist, Adobe.

---

From: Martyn Loughran <[email-masked]>
Date: Thu, 2 Dec 2010 18:23:51 +0000

I suggest taking a look at this thread
http://www.ietf.org/mail-archive/web/hybi/current/msg04772.html

The browser consensus is to wait for the connect/upgrade discussion to
be concluded and not to implement 03 in the interim, even though it's
an improvement on 76.

The migration from 76 to 03 would be quite easy, since a server can
easily support both protocols (em-websocket for example already does
so). I worry that it will be harder to implement adam's proposal and
76 simultanously. Has anyone attempted this?

Martyn

On 1 December 2010 17:00, Matthew Butler <[email-masked]> wrote:
> We were doing work based on Draft-76 but have recently seen the changes
> going on in the spec, and we're wondering how/what the migration path from
> the current state of the browsers to one supporting something along the
> lines of the current draft will be.
> Thoughts ?
> Thanks
> Matt Butler
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 2 Dec 2010 13:37:57 -0500

> 76 simultanously. Has anyone attempted this?

Yes, and working code is in the websocket-draft-eval project.
Basically, any HTTP-compatible handshake that has some distinguishing
characteristic in either the request line or the headers can be
supported transparently with the others.  Adam's proposal is easily
distinguished by using the CONNECT method instead of GET.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 02 Dec 2010 11:15:17 -0800

> Yes, and working code is in the websocket-draft-eval project.

No, John. The issue is integration with existing web servers and proxies 
(including management, configuration, servlet API, etc.).

Since writing a dedicated websocket server is easy for any of the 
proposed protocols (parsing is not hard), your example really doesn't 
prove what you seem to think it proves.

I upgraded Resin's websocket to Greg's latest proposal without much 
change needed other than updating the protocol parsing at the servlet 
level, not affecting the HTTP stack at all. The CONNECT with the fake 
"websocket.invalid" would need more work, including fake 
protocol-specific virtual hosts. But the old CONNECT that you 
implemented would blow up the entire virtual host dispatching and screw 
up the HTTP stack.

> distinguished by using the CONNECT method instead of GET.

CONNECT does not dispatch to websocket uniquely. In fact, as Roy pointed 
out, CONNECT is not supposed to dispatch at all and using it to dispatch 
goes against the purpose of the HTTP CONNECT.

Typically servers dispatch based on host/uri and let the application 
handle the HTTP method. Changing that dispatch changes the entire server 
model.

The reason the updated CONNECT could be make to work for servers is 
because the fake "websocket.invalid" can be dispatched as a 
protocol-specific virtual host which could fit into the existing 
host/uri server dispatch (I think), but it's a hack.

With the CONNECT model, I no longer understand the purpose of the HTTP 
integration.

If the only purpose of the websockets HTTP handshake is to tunnel 
through port 80 to avoid firewalls (and increase connectivity from 60% 
to 70%), that's just abusing HTTP, and I don't see the justification.

-- Scott

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 2 Dec 2010 11:32:56 -0800

On Thu, Dec 2, 2010 at 11:15 AM, Scott Ferguson <[email-masked]> wrote:

> Since writing a dedicated websocket server is easy for any of the proposed
> protocols (parsing is not hard), your example really doesn't prove what you
> seem to think it proves.
>
> I upgraded Resin's websocket to Greg's latest proposal without much change
> needed other than updating the protocol parsing at the servlet level, not
> affecting the HTTP stack at all. The CONNECT with the fake
> "websocket.invalid" would need more work, including fake protocol-specific
> virtual hosts. But the old CONNECT that you implemented would blow up the
> entire virtual host dispatching and screw up the HTTP stack.
>
>

I believe John was talking about distinguishing between a -00/-76 handshake
and a -04(?) handshake using CONNECT.

As for the "purpose of the HTTP CONNECT" we can argue all day about
semantics but at the end of the day it's the one thing that appears to work.

>
> Typically servers dispatch based on host/uri and let the application handle
> the HTTP method. Changing that dispatch changes the entire server model.
>
> The reason the updated CONNECT could be make to work for servers is because
> the fake "websocket.invalid" can be dispatched as a protocol-specific
> virtual host which could fit into the existing host/uri server dispatch (I
> think), but it's a hack.
>
> If the only purpose of the websockets HTTP handshake is to tunnel through
> port 80 to avoid firewalls (and increase connectivity from 60% to 70%),
> that's just abusing HTTP, and I don't see the justification.
>
> -- Scott
>
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 2 Dec 2010 14:40:51 -0500

> protocols (parsing is not hard), your example really doesn't prove what you

It proves that you can support multiple versions simultaneously, which
was the question asked.  The example given, em-websocket, appears to
also be a dedicated server.

Regarding existing servers, I think many of them are going to have
significant work required to support long-lived connections and
provide an appropriate API for WebSockets anyway, so minimizing the
change to the servers doesn't seem that important.

> goes against the purpose of the HTTP CONNECT.

Obviously, you check the other aspects of the connection before
assuming it is a WebSocket request, but if the server supports
WebSockets and isn't a proxy, then any CONNECT request it receives is
either a WebSocket request or an error.  If it is also a proxy, then
it would have to look for WebSocket-specific values to distinguish the
two.

> the HTTP method. Changing that dispatch changes the entire server model.

Two issues with this statement:  first, I don't think choosing the
application based on the path has to change at all.  Second, you don't
want the application to have to handle the details of WebSockets, but
instead have a message-oriented API for that.  So, the API for
receiving WebSocket connections is necessarily going to be different
than for receiving HTTP requests.

> that's just abusing HTTP, and I don't see the justification.

As I understand it, the primary purpose of HTTP compatibility is in
fact so it is deployable with some effectiveness at getting through
existing proxies/firewalls in the HTTP infrastructure.  Is comet or
BOSH any less an abuse of HTTP?

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 13:52:19 -0600

> semantics but at the end of the day it's the one thing that appears to work.

This has become a popular myth, without evidence. We've been told that
Adam's experiments showed that CONNECT works better than GET+Upgrade.
Now the data is presented to us, I don't think it supports that
argument.

Security: it appears that framed WS data after handshake is extremely
unlikely to be mistaken as HTTP. 99.9% parsers are busted.

Connectivity: after HTTP compliant handshake, whether WS frames can be
transmitted on the wire without obstacle and corruption. What's the
success rate of that for the two methods? I can't find any direct
answer in the paper. If CONNECT indeed has better luck in this
perspective, we need to see the number to judge whether the difference
is significant.

- Zhong Yu

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 02 Dec 2010 11:56:02 -0800

John Tamplin wrote:
> ... if the server supports WebSockets and isn't a proxy, then any CONNECT request it receives is
> either a WebSocket request or an error.

Do I need to explain why that's a horrible specification for the HTTP 
CONNECT?

-- Scott

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 15:57:07 -0600

2010/12/2 Gabriel Montenegro <[email-masked]>:
> Exactly, Adam's paper and measurements are good and interesting, but are not relevant to the improved Upgrade+ handshake proposed by Greg. They don't even use the websocket framing.

I don't know about that. The more I read the paper, the more I am
confused. It's missing key technical details for us to understand what
exactly was measured.

> Given that there is nothing proven against the Upgrade+ proposal, and that is what is closest to what the WG started with (even before -76), that should be the way to go forward.

GET+Upgrade is confusing though. Is it a GET? Or is it an Upgrade?
Most current implementations treat it as a normal GET. I doubt you can
fault the implementations by the letter of the spec.

Just because Upgrade is defined in the spec, doesn't mean it's useful.
If nobody has ever implemented it, we may as well treat it as
non-existent. Resurrect the dead "Upgrade" is no less disruptive than
introducing a new "WEBSOCKET" method.

- Zhong

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 2 Dec 2010 14:41:49 -0800

2010/12/2 Gabriel Montenegro <[email-masked]>

> Exactly, Adam's paper and measurements are good and interesting, but are
> not relevant to the improved Upgrade+ handshake proposed by Greg. They don't
> even use the websocket framing.
>
>
The whole point is that the handshake is supposed to guard against cross
protocol attacks and ensure that we are talking to a websocket server. That
the framing may prevent some attacks is good, but is not something upon
which I would care to rely as a browser vendor.

I also have no intention of shipping something that introduces an extra RTT.

> So their paper is irrelevant to the issue of comparing Adam's CONNECT+
> proposal and Greg's Upgrade+ proposal.
>
> The paper shows that HTTP's Upgrade (unrelated to websockets) has some
> issues. Those exist independently of websockets. I'd even argue that if
> websockets were to use Upgrade as part of the Upgrade+ handshake, increased
> visibility of Upgrade would eventually help its use within HTTP. At any
> rate, it wouldn't make that pre-existing vulnerability in HTTP any more
> vulnerable.
>
> On the other hand, the point raised by Roy and others about CONNECT not
> leading anywhere due to its redefining something that already exists in HTTP
> is not to be taken lightly. This is the kind of thing that can raise DISCUSS
> positions in the IESG until the WG fixes it.
>
> Given that there is nothing proven against the Upgrade+ proposal, and that
> is what is closest to what the WG started with (even before -76), that
> should be the way to go forward.
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Zhong Yu
> Sent: 02 December, 2010 11:52 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [hybi] Do we have a sense of when browsers will be supporting
> draft-ietf-hybi-thewebsocketprotocol-03 ?
>

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 17:25:02 -0600

> protocol attacks and ensure that we are talking to a websocket server. That

I don't think that's the point of Adam's paper, at all. He's trying to
show that, when both ends are WebSocket compliant, there could be
hidden intermediaries that do non transparent things with the data
flow. These intermediaries could be attacked by placing chosen bytes
on the wire. Using Java Socket, Adam detected that this is a potential
problem in almost 4% cases. If we allow arbitrary bytes on a WebSocket
connection, the result will be disastrous.

There are 3 ways to avoid such attacks.

1. Use CONNECT method. Hopefully, most intermediaries will give up
interpreting remainder bytes once they see a CONNECT request. On page
11, "we observed two proxies which appear not to understand CONNECT
but simply to treat the request as an ordinary request and then
separately route subsequent requests"

2A with simple framing bytes. Although only a few bytes, it's possible
that it's enough to defeat most intermediary parsers, so that WS
frames will not be interpreted as HTTP. I believe, but not sure, that
this is responsible for eliminating 99.9% of firewall circumvention
attacks in the Upgrade experiment.

2B Encrypt each frame with a different key. There is no way attacker
can control any bits.

#1 is independent of #2A/B.

If #2B is adopted, #1 becomes redundant and irrelevant.

If #2A is deemed good enough, #1's utility is negligible.

---

From: Gabriel Montenegro <[email-masked]>
Date: Thu, 2 Dec 2010 21:24:31 +0000

So their paper is irrelevant to the issue of comparing Adam's CONNECT+ proposal and Greg's Upgrade+ proposal.

The paper shows that HTTP's Upgrade (unrelated to websockets) has some issues. Those exist independently of websockets. I'd even argue that if websockets were to use Upgrade as part of the Upgrade+ handshake, increased visibility of Upgrade would eventually help its use within HTTP. At any rate, it wouldn't make that pre-existing vulnerability in HTTP any more vulnerable.

On the other hand, the point raised by Roy and others about CONNECT not leading anywhere due to its redefining something that already exists in HTTP is not to be taken lightly. This is the kind of thing that can raise DISCUSS positions in the IESG until the WG fixes it. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Zhong Yu
Sent: 02 December, 2010 11:52 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [hybi] Do we have a sense of when browsers will be supporting draft-ietf-hybi-thewebsocketprotocol-03 ?

- Zhong Yu

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 2 Dec 2010 17:16:00 -0800

2010/12/2 Zhong Yu <[email-masked]>

> show that, when both ends are WebSocket compliant

Adam / the paper make no assumptions about the other end being WebSocket
compliant. Indeed, the whole fear of cross protocol attacks is an attack
against an endpoint or intermediary not able to distinguish / handle
WebSockets handshakes or traffic.

> , there could be

As a browser vendor, I find papers like adam's much more convincing than "I
believe, but not sure" :) No offense, but I would much rather go with
something where we have a better understanding of the security properties
and why it's secure, than just a hodgepodge of defenses that we think will
avoid problems.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 2 Dec 2010 17:52:07 -0800

> The paper shows that HTTP's Upgrade (unrelated to websockets) has some issues. Those exist independently of websockets. I'd even argue that if websockets were to use Upgrade as part of the Upgrade+…

Upgrade doesn't have any issues.  The attack described in

   http://www.adambarth.com/experimental/websocket.pdf

uses the Host misdirection technique, a known vulnerability in
L7 routers and intercepts that reroute requests based on packet
inspection.  The issues therefore apply to any protocol operating
on port 80 within such a network, including HTTP itself, and the
only cure is to replace that software.

  http://www.kb.cert.org/vuls/id/435052
  http://www.securityfocus.com/bid/33858
  http://www.cgisecurity.com/2009/07/more-products-identified-using-vulnerable-transparent-proxy-architecture.html

What I hadn't seen published before is the variation wherein an
attacker site uses a similar technique to poison a shared cache
within an intercept proxy based on the cache key (which some
intercepts obtain from the Host header because, even though it
isn't operating as a true proxy, most of the implementations
share the same code base and cache as a true proxy).  That bug
can be fixed by adding the IP address to the cache key.

Someone should post a new CVE for this (and update the old CVE).

....Roy

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 20:10:41 -0600

> compliant. Indeed, the whole fear of cross protocol attacks is an attack

The experiments do simulate compliant WebSocket(of some version) end
points. The focus is on using valid WebSocket traffic to attack
intermediaries.

> believe, but not sure" :) No offense, but I would much rather go with

What I meant is, that's the only conclusion I can think of that
explains the data in the paper. I believe that based on the data in
the paper, not by thinking abstractly. The "99.9%" number is not a
made up figure, it's calculated based on the data in the paper.

However the paper is confusing enough that I cannot be certain. I have
asked Adam if he has any alternative explanation, why the Upgrade
experiment eliminated 99.9% of firewall circumvention attacks, if not
because the framing bytes busted 99.9% HTTP parsers.

> and why it's secure, than just a hodgepodge of defenses that we think will

I am trying very hard to understand the numbers in the paper.

- Zhong Yu

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 08:07:56 +0100

> semantics but at the end of the day it's the one thing that appears to work.

Ian,

that is an overstatement of the findings of Adam's paper.

The full WS handshake was not tested, and certainly not with several
proposals that have been made to improve GET+Upgrade.

GET+Upgrade does work and is protected from the attack by framing.
Hello packets and changes to framing can offer further protection.
Some feel that is weak protection and wish to consider CONNECT as an
alternative because there is some evidence that vulnerable
intermediaries treat CONNECT differently.

But it is wrong to say that CONNECT is the only thing that works,
specially as it brings with it a new set of issues and objections that
have yet to be widely tested.  It also seams to me that relying on
intermediaries to treat CONNECT differently is no more/less
strong/weak than relying on intermediaries to not scan within frames.
 Thus the conversation about encrypting, XORing on escaping content is
ongoing, because we are still concerned about this threat regardless
of the handshake.

I'm beginning to think we are spooking ourselves too much and that
regardless of what handshake we use, we should simply have a framing
mechanism that is designed to not look like HTTP and that we can
empirically test is sufficient to prevent intermediaries seeing HTTP
requests within the body of the request.   We need to do this
regardless of CONNECT or GET+Upgrade.

The use of CONNECT instead of GET+Upgrade needs to be proposed as a
change in it own right and not mixed up with several other issues.  If
a CONNECT proposal can overcome the objections to it, then great!
But it is not the only option we have.

cheers

>> protocol-specific virtual host which could fit into the existing host/uri
>> server dispatch (I think), but it's a hack.
>>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 08:15:48 +0100

Ian,

as a browser vender, framing is exactly what you are relying on to
prevent this attack in normal HTTP.

It is the HTTP framing around request bodies that prevents vulnerable
intermediaries seeing phantom HTTP requests within the payloads.
This framing defence is apparently good enough for all of the HTTP
today, so I don't see why a suitably robust framing defence is not
sufficient for websocket?

> I also have no intention of shipping something that introduces an extra RTT.

The Hello proposal does not require an extra round trip.   It just
ensures that the first frame in each direction is something that is
entirely not HTTP.

The Hello frames are also carrying nonces and hashes, and it could be
argued that the server should not send data until it sees the Hello
from the client on that basis - but that is for reasons other than
framing and apply equally to any handshake.  If we want the server to
wait until the client has verified a nonce+hash from the server, then
there is an extra RTT no matter what handshake is used.
I don't think that verification is necessary.

cheers

---

From: Maciej Stachowiak <[email-masked]>
Date: Fri, 03 Dec 2010 00:18:51 -0800

> But it is not the only option we have.

You supported a CONNECT proposal yesterday:

http://www.ietf.org/mail-archive/web/hybi/current/msg04860.html
http://www.ietf.org/mail-archive/web/hybi/current/msg04861.html

Do those messages reflect your current position?

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 09:41:00 +0100

> Do those messages reflect your current position?

Maciej,

I'm not opposed to a CONNECT solution and I'd like to see a clear
proposal made that separates out CONNECT from the other issues (bogus
hosts etc).  I'd like to see if such a proposal can be made acceptable
to the objections raised by Roy and others.   The proposals made
yesterday do look like a good way to progress.

But I don't think that CONNECT is universally accepted and I think the
case still has to be made to convince others.  I'm not totally
convinced myself, but think it is worth considering and asking for
concrete proposals without bogus header and encoding distractions is a
good step towards evaluating those concerns.

Most importantly  I do *not* think that CONNECT is our only available
solution.  My recent posts have been to simply correct some
overstatements about the vulnerability (or otherwise) of GET+Upgrade.
  I still think that  GET+Upgrade together with robust framing is a
viable solution - but I recognise that some have concerns with a
defence based only on framing.   Thus I'm happy to try form a
consensus around CONNECT, but at the same time I'm talking about
misconceptions of the framing defence because that applies regardless
of handshake and may be important if we can't overcome objections to
CONNECT.

Personally, my current favourite option now is for a dedicated WS port
and the use of CONNECT to access it via port 80 if it cannot be
reached directly.       But it looks like CONNECT has some serious
objections to its use and if they cannot be overcome, then I'm not
opposed to using GET+Upgrade.    IE I don't really care which
handshake we use so long as we can agree on one or the other.  I care
about HTTP compatibility, valid routing data and robust extensible
framing and I think these concerns.

cheers

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 03 Dec 2010 09:08:01 -0800

Greg Wilkins wrote:
> reached directly.  

+1

>      But it looks like CONNECT has some serious
> opposed to using GET+Upgrade.  

I'd think using CONNECT as a proxy to WS port 8888 would be valid, since 
that should be a normal use of CONNECT without needing any magic.

-- Scott

---

From: Brian McKelvey <[email-masked]>
Date: Sat, 4 Dec 2010 10:58:16 -0800

But that's only appropriate to send to a proxy server that the user has configured their browser to use.  Sending a CONNECT request without knowing that it's being sent to a proxy is inappropriate, as Roy pointed out, and if you did send it to a real proxy server, then that part of the handshake would be stripped anyway by the time the connection reached our websocket destination.  The proxy would open a new empty tunnel and not forward the HTTP CONNECT request itself.

Sent from my iPhone

On Dec 3, 2010, at 9:08 AM, Scott Ferguson <[email-masked]> wrote:

> I'd think using CONNECT as a proxy to WS port 8888 would be valid, since that should be a normal use of CONNECT without needing any magic.
> 
> -- Scott
>

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 5 Dec 2010 09:59:09 +0100

On 4 December 2010 19:58, Brian McKelvey <[email-masked]> wrote:
> But that's only appropriate to send to a proxy server that the user has configured their browser to use.
> Sending a CONNECT request without knowing that it's being sent to a proxy is inappropriate, as Roy
> pointed out, and if you did send it to a real proxy server, then that part of the handshake would be stripped
> anyway by the time the connection reached our websocket destination.  The proxy would open a
> new empty  tunnel and not forward the HTTP CONNECT request itself.

Which I think is exactly the semantics we want iff the CONNECT was to
a dedicated WS port.

If a dedicated port existed, I think browsers could get around the
configured proxy semantics in several ways.   Firstly they can add a
WS proxy to their configuration, so that if one was configured it
would be used for all WS connections.

We could then say that the default proxy for WS connections is to
assume that their is a proxy running on port 80 of the target host.
Further, if WS fails in a direct connect attempt, it should attempt to
use the configured or default proxy.

The key problem I see with this, is that it would allow a WS browser
to send connects to non-ws hosts and thus CONNECTs to their port 80
when they are not expecting them.    This, as Roy points out, may
trigger alarms as a known attack.   But perhaps this is good, because
if WS connections are being attempted on a server that does not expect
them, then alarms probably should be raised?

cheers