Subject: Re: [hybi] WebSocket, TLS and intermediaries

From: Ian Hickson <[email-masked]>
Date: Tue, 20 Jul 2010 22:03:32 +0000 (UTC)

On Mon, 19 Apr 2010, Vladimir Katardjiev wrote:
> 
> But wouldn't it be better if [the whole point of having a standard like 
> websocket, is so that the network infrastructure can implement the RFC 
> and then we have known semantics over that connection that 
> intermediaries can at least handle correctly, but may also do clever 
> stuff with]?

IMHO no, not really. I think experience has taught us that while having 
intelligence in the network seems like a great idea, in practice having 
all the intelligence at the edge of the network is far superior. Indeed 
some of the biggest problems with deploying new technologies have been 
"intelligent" intermediaries -- for example, deployment of pipelining in 
HTTP has been delayed on the order of a decade because of man-in-the- 
middle proxies. Indeed, this insight -- that the core network should be 
dumb and the clverness should be in the edge -- is one of the underlying 
principles on which TCP/IP was built.

> Let's take a couple of hypothetical scenarios where an amateur 
> programmer can _transparently_ gain additional, desirable functionality 
> by an intermediary "messing" with a well-defined protocol.
> 
> 1. Compression
> For all the talk about transferring 5gb using WebSockets, the FCC still 
> maintains a fairly conservative definition of "broadband" at 768kbps. I 
> believe the US average broadband speed was a handy contribution to the 
> speed savings that SPDY's compression of data made possible. In other 
> words, it's not unlikely that a future iteration of WebSockets adds 
> compression (let's say gzip) to the equation.
> 
> Obviously, for this to work it needs to be optional, because otherwise 
> we raise the barrier of entry. But in some deployments it is a desirable 
> thing for the ISP to add. An ISP might want to compress the traffic to 
> lower the load on their edge network (or their 768kbps broadband 
> customers) In the case of a mobile network, the shorter time the radio 
> is kept alive, the more battery savings, so it's highly desirable to 
> compress WebSocket connections if appropriate.

Why would they compress jsut WebSockets and not simply compress all IP 
traffic?

Suppose though that there is some reason why WebSocket traffic should be 
compressed specifically. Nothing prevents a man-in-the-middle proxy from 
intercepting WebSocket handshakes and traffic and implementing the 
compression for the client transparently. The proxy would just act as a 
server for the real client, and act as a client for the real server. 
Neither side need know that anything is going on.

So this case appears to already be handled. (Or will be, once we define 
how to opt into compression at the WebSocket layer, if we add that.)

> 2. Optimisation
> Suppose we had a way of determining that a client cannot receive 
> messages larger than 1mb, due to memory restrictions. (Feel free to 
> substitute it with 5gb for the 2020 scenario). Wouldn't it be a waste of 
> time to send all that data to the remote client only for it to discover 
> that state on its own? For a mobile client, if the behaviour on too 
> large frames is well-defined, we may even handle it without waking the 
> client up (or close the connection if that's the defined behaviour) 
> potentially saving a lot of bandwidth and battery.

If you have control over the device, then there's no reason not to define 
the client as being a distributed piece of software that spans the network 
from the base station to the device, and thus there's no need to wake the 
device to handle an incoming frame that's too big.

(That seems like a weird optimisation though. You're likely going to have 
to wake the device anyway to let it know the connection was terminated, 
since that can run some JS.)

> 3. Error checking
> A properly written intermediary is also expertly written. So it would be 
> able to prevent protocol exploits that may appear. In addition, these 
> types of intermediaries would be maintained for the purpose of security 
> and thus updated faster than you could hypothetically update client(s) 
> and server(s). While it doesn't eliminate the spread of exploits and 
> malware, it can reduce it. If we are supposed to be as concerned as we 
> are about security, we should also cover the possibility that there may 
> be a security hole in the future (one that we couldn't consider now) and 
> make sure it can be protected. An intermediary can be a part of such a 
> solution, but that also implies we must have intermediaries that can 
> talk WebSockets.

That's such a hypothetical case (a well-written intermediary that is 
updated faster than the client is something that I've never heard of 
before, and fixing a security bug in the protocol or in the client might 
well require violating the protocol by definition) that I'm not really 
sure how to apply it concretely.

> 4. Deployment
> This falls into the category of "making the protocol actually work", but 
> I do count it as a benefit to the amateur programmer because, yeah, I 
> don't quite think TLS is so trivial to implement either. I don't 
> remember where the original reference is, but I don't think creating a 
> protocol whose design goal is to be wrapped end to end in TLS to 
> simulate having open network access is going to work.
> 
> For one, you can't guarantee having no intermediaries with TLS (for the 
> amateur programmer). All you need to do to MITM a TLS connection is, 
> well, an intermediary box (I'd wager most ISPs can arrange one of those) 
> and a signed certificate (shouldn't be an issue either). Since the 
> WebSocket connection is user transparent they won't notice that the 
> certificate no longer says Bank.com but ISP.com. Or Corporation.com for 
> that matter. (I won't discuss ethics/morality/legality, just technology)

The protocol would reject such a connection if the certification didn't 
match the host name.

> I don't think working on bypassing intermediaries for all traffic is a 
> productive avenue. Rather, I think it'd just lead to a pointless arms 
> race.

The spec's design doesn't intentionally bypass intermediaries, it just 
doesn't consider compatibility with intermediaries or supporting 
intermediaries to be a priority. Having seen what a mess HTTP has had to 
contend with when it comes to intermediaries, this seems wise, IMHO.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Ian Hickson <[email-masked]>
Date: Tue, 20 Jul 2010 22:14:04 +0000 (UTC)

(By the way, I spoke to Joe and he suggested that I post an e-mail to each 
thread to which I was replying, rather than replying to all threads in one 
bulk e-mail or replying to all the e-mails I'm replying to individually. I 
hope this compromise works for everyone; I know there was some controversy 
when I replied in bulk a few months ago. I apologise in advance if this 
causes a spike in traffic to the list.)

On Mon, 19 Apr 2010, Roberto Peon wrote:
> On Mon, Apr 19, 2010 at 2:20 AM, Julian Reschke wrote:
> > > On 19.04.2010 10:48, Ian Hickson wrote:
> > > >
> > > > There are many many implementations of HTTP. Some fast, some not 
> > > > so.  Some complete, some not so.
> > >
> > > I think we can get orders of magnitude more complete implementations 
> > > of Web Sockets than of HTTP if we keep the protocol trivial.
> >
> > Yes. That's a given. Make it less complex, and it will be easier to 
> > completely implement.
> 
> This isn't true! Make it (the protocol) less complex and it will be easy 
> to implement something which *conforms to the spec*, but not necessarily 
> something which scales and is robust, reliable, and scalable in the face 
> of all the stuff that happens out there.

Not all implementations have to scale to a million QPS or withstand DDOS 
attacks for weeks at a time. Most implementations of Web Sockets will 
likely be for small amateur projects that are lucky if they average 1 QPH, 
let alone 1 QPS.

> The latter part is what really worries me. We really need to be sure 
> that the protocol that we create allows for an implementation of a 
> server to do these things.

I agree that we shouldn't make the protocol impossible to scale. If there 
are specific features in the protocol that make it impossible to scale, 
please do raise these as issues. However, one doesn't have to make a 
protocol complex to make it scale.

> If the (hopefully small) added complexity is too much for the amateur 
> programmer, then they should use the API level.

What I'm interested in personally is writing a protocol that amateur 
programmers can implement easily. If we say they have to use someone 
else's code to do this, then that's a failure, IMHO. (Though as I've 
mentioned before, if people have different goals or priorities, I have no 
problem with separate protocols also being designed to address those -- 
Web Sockets doesn't have to be everything for everybody.)

---

From: John Tamplin <[email-masked]>
Date: Tue, 20 Jul 2010 18:21:02 -0400

> Web Sockets doesn't have to be everything for everybody.)

I still don't see the argument for servers written by amateurs.  I have used
far more quick-and-dirty web clients (telnet, socket/connect/write in C,
wget/curl, etc) than I have quick-and-dirty servers.  If some amateur needs
to write a server anyway, they aren't going to write it from scratch even if
it were possible for them to do so -- they will find some library.

Contrast this with embedded clients, which might well have tighter
constraints than most clients, and could benefit from having optional
features they could leave out that would be useful for more powerful
clients.  Meanwhile, the servers are going to almost certainly be on more
powerful machines, though granted they have more connections to support.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Roberto Peon <[email-masked]>
Date: Tue, 20 Jul 2010 15:33:55 -0700

> protocol complex to make it scale.

We need browser-owned connections with a security model that allows multiple
tabs to share the connection safely.
This is probably the most important requirement for scalability as far as
I'm concerned. This requires fewer keep-alives (both because of fewer TCP
connections and because the TCP connection is more likely to have real
content sent on it instead of near-zero value'd keep-alives), it requires
fewer file descriptors, it requires fewer handshakes.

-=R

---

From: Mike Belshe <[email-masked]>
Date: Tue, 20 Jul 2010 16:10:07 -0700

> Web Sockets doesn't have to be everything for everybody.)

For as adamantly as Ian states that it should be a requirement, I am just as
adamant that it should not.

But more importantly, this single issue has been holding protocol progress
hostage.  Naturally, any feature has some amount of complexity.   But this
requirement creates an invisible, and subjective barrier to each feature.
 Is the feature too complex for an amateur programmer?  Nobody knows, and
everyone disagrees, because it is a subjective criteria.  So we spin and
can't agree.

Every protocol expert I've spoken with agrees that amateur protocol
implementors should not be a requirement.

Is there some way we can vote to either keep or nullify this requirement now
and never come back to it again?  I'm tired of this obstacle holding
everything up.

Mike

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Jack Moffitt <[email-masked]>
Date: Tue, 20 Jul 2010 17:16:25 -0600

> and never come back to it again?  I'm tired of this obstacle holding

It seems to me that any protocol that requires security is already
outside the reach of amateur developers without library support.  If
they must rely on libraries written by professionals for security, why
is it so bizarre to ask them to rely on libraries for other functions
that are also complex and hairy?

I think this is getting at the root of the issues. We either need to
strike this requirement or give it some concrete meaning. As I'm not
sure such meaning can be easily provided, striking the 'amateur
programmer' requirement seems prudent.

jack.

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 20 Jul 2010 16:22:06 -0700

Mike Belshe wrote:
>
> For as adamantly as Ian states that it should be a requirement, I am 
> just as adamant that it should not.
>
> Is there some way we can vote to either keep or nullify this 
> requirement now and never come back to it again?  I'm tired of this 
> obstacle holding everything up.
+1

-- Scott

---

From: Mike Belshe <[email-masked]>
Date: Tue, 20 Jul 2010 16:29:07 -0700

On Mon, Apr 19, 2010 at 7:47 AM, Vladimir Katardjiev <[email-masked]>wrote:

> On 16 apr 2010, at 22.13, Ian Hickson wrote:
> > On Fri, 16 Apr 2010, Greg Wilkins wrote:
> >>
> >> The whole point of having a standard like websocket, is so that the
> >> network infrastructure can implement the RFC and then we have known
> >> semantics over that connection that intermediaries can at least handle
> >> correctly, but may also do clever stuff with.
> >
> > That wasn't even on my list of reasons for originally coming up with
> > TCPConnection, let alone the main reason.
>
> >
> > The main reason for the Web Socket protocol is to have one protocol that
> > the browsers can all implement in an interoperable fashion so that
> servers
> > can implement things once and have them work with all browsers. In the
> > ideal deployment the connection is wrapped in end-to-end TLS, so the
> > intermediaries can't do anything with it. There were only two reasons for
> > even allowing unencrypted Web Socket originally: requiring amateurs to
> > figure out TLS seemed like too high a bar, and there is still some
> concern
> > that when the connection is being used for entirely public information,
> > the TLS connection overhead is too expensive to justify.
>
> Fair enough, it wasn't originally. But wouldn't it be better if it did?
> Let's take a couple of hypothetical scenarios where an amateur programmer
> can _transparently_ gain additional, desirable functionality by an
> intermediary "messing" with a well-defined protocol.
>
> 1. Compression
> believe the US average broadband speed was a handy contribution to the speed
> savings that SPDY's compression of data made possible. In other words, it's
> not unlikely that a future iteration of WebSockets adds compression (let's
> say gzip) to the equation.
>

For the SPDY data point - for low speed links, the header compression
matters.  For high speed links, it doesn't.  SPDYs gains mostly come from
other factors.  I wish I had the graphs handy, but I do not.

>
> Obviously, for this to work it needs to be optional, because otherwise we
> raise the barrier of entry. But in some deployments it is a desirable thing
> for the ISP to add. An ISP might want to compress the traffic to lower the
> load on their edge network (or their 768kbps broadband customers) In the
> case of a mobile network, the shorter time the radio is kept alive, the more
> battery savings, so it's highly desirable to compress WebSocket connections
> if appropriate.
>
> 2. Optimisation
> Suppose we had a way of determining that a client cannot receive messages
> larger than 1mb, due to memory restrictions. (Feel free to substitute it
> with 5gb for the 2020 scenario). Wouldn't it be a waste of time to send all
> that data to the remote client only for it to discover that state on its
> own? For a mobile client, if the behaviour on too large frames is
> well-defined, we may even handle it without waking the client up (or close
> the connection if that's the defined behaviour) potentially saving a lot of
> bandwidth and battery.
>
> 3. Error checking
> able to prevent protocol exploits that may appear. In addition, these types
> of intermediaries would be maintained for the purpose of security and thus
> updated faster than you could hypothetically update client(s) and server(s).
> While it doesn't eliminate the spread of exploits and malware, it can reduce
> it. If we are supposed to be as concerned as we are about security, we
> should also cover the possibility that there may be a security hole in the
> future (one that we couldn't consider now) and make sure it can be
> protected. An intermediary can be a part of such a solution, but that also
> implies we must have intermediaries that can talk WebSockets.
>

Let me try to be objective about the tradeoff here:  When everything is
encrypted and authenticated at the endpoints, it prevents an intermediary
(who's motivations may be different than yours) from taking action.  When
data is not encrypted & authenticated, intermediaries, which have different
motivations (good or bad) are possible.

So the sad news is that for every good motivation, there is an equal and
opposite bad motivation as a counter argument...

>
> 4. Deployment
> This falls into the category of "making the protocol actually work", but I
> do count it as a benefit to the amateur programmer because, yeah, I don't
> quite think TLS is so trivial to implement either. I don't remember where
> the original reference is, but I don't think creating a protocol whose
> design goal is to be wrapped end to end in TLS to simulate having open
> network access is going to work.
>

> amateur programmer). All you need to do to MITM a TLS connection is, well,
> an intermediary box (I'd wager most ISPs can arrange one of those) and a
> signed certificate (shouldn't be an issue either). Since the WebSocket
> connection is user transparent they won't notice that the certificate no
> longer says Bank.com but ISP.com. Or Corporation.com for that matter. (I
> won't discuss ethics/morality/legality, just technology)
>

Actually, if you do that, the browser throws up a huge red box saying
"danger".  The MITM will not have a signing cert for a CA trusted by the
browser.  So, the cert will be signed by the ISP or other transparent
intermediary, which won't match the trusted CA list, and the browser will
complain.  Corporations can get around this by forcing a CA cert onto the
browser; but that is no longer a transparent intermediary - it was
specifically granted permission at the endpoint.  Of course, the user, not
knowing what the hell is going on, will click through anyway, think "that's
weird", and everything else will still work.

BTW - there is another data point here; deployment of WebSockets over port
80 was measured in Chrome to have ~67% success rate today.  Deployment over
port 443 (with TLS) has a >95% success rate.  So, if you don't use TLS, then
browsers and websites will need to be made more complex to deal with the
edge case of WebSockets failing in weird ways due to existing intermediaries
which fail, even after the WebSocket handshake.

Mike

>
> (course, this won't work if the TLS box shares https and wss traffic, but
> then we're not dealing with the amateur programmer are we?)
>
> productive avenue. Rather, I think it'd just lead to a pointless arms race.
> It was my impression that what this wg was out to make was a protocol that
> played "nice" with both nonexistent, current and future intermediaries.
>
> Don't get me wrong, the actual data passed through the WebSocket connection
> should be (possible to be?) opaque to any intermediaries, but the fact that
> it is a message, or a keepalive, or a graceful close, should be information
> available for intermediaries.
>
> Vladimir
>

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 20 Jul 2010 17:01:02 -0700

On Jul 20, 2010, at 4:29 PM, Mike Belshe wrote:

> 
> 3. Error checking
> A properly written intermediary is also expertly written. So it would be able to prevent protocol exploits that may appear. In addition, these types of intermediaries would be maintained for the purpose of security and thus updated faster than you could hypothetically update client(s) and server(s). While it doesn't eliminate the spread of exploits and malware, it can reduce it. If we are supposed to be as concerned as we are about security, we should also cover the possibility that there may be a security hole in the future (one that we couldn't consider now) and make sure it can be protected. An intermediary can be a part of such a solution, but that also implies we must have intermediaries that can talk WebSockets.
> 
> Let me try to be objective about the tradeoff here:  When everything is encrypted and authenticated at the endpoints, it prevents an intermediary (who's motivations may be different than yours) from taking action.  When data is not encrypted & authenticated, intermediaries, which have different motivations (good or bad) are possible.
> 
> So the sad news is that for every good motivation, there is an equal and opposite bad motivation as a counter argument...

Also, in practice it does not seem to be the case that intermediaries are updated faster than clients or servers. One of the challenges to deploying new protocols or protocol features is old intermediaries that are not updated.

> 
> Actually, if you do that, the browser throws up a huge red box saying "danger".  The MITM will not have a signing cert for a CA trusted by the browser.  So, the cert will be signed by the ISP or other transparent intermediary, which won't match the trusted CA list, and the browser will complain.  Corporations can get around this by forcing a CA cert onto the browser; but that is no longer a transparent intermediary - it was specifically granted permission at the endpoint.  Of course, the user, not knowing what the hell is going on, will click through anyway, think "that's weird", and everything else will still work.

Browsers don't have to accept broken certs for WebSocket with a clickthrough the way many still do for Web sites. I would argue that they should not, because the security consequences of the decision cannot possibly be understood by the typical user.

> 
> BTW - there is another data point here; deployment of WebSockets over port 80 was measured in Chrome to have ~67% success rate today.  Deployment over port 443 (with TLS) has a >95% success rate.  So, if you don't use TLS, then browsers and websites will need to be made more complex to deal with the edge case of WebSockets failing in weird ways due to existing intermediaries which fail, even after the WebSocket handshake.

This point is very important. Building on top of TLS has huge practical benefits. I think this outweighs the desire to more easily build transparent intermediaries. Any mechanism that allows intermediaries without being authorized by either endpoint is by definition a security vulnerability in the protocol.

I think the benefits of TLS also outweigh the "amateur server implementor" argument. I don't think we want to make it easy to implement a security hole.

Regards,
Maciej

---

From: John Tamplin <[email-masked]>
Date: Tue, 20 Jul 2010 20:04:46 -0400

On Tue, Jul 20, 2010 at 8:01 PM, Maciej Stachowiak <[email-masked]> wrote:

> This point is very important. Building on top of TLS has huge practical
> benefits. I think this outweighs the desire to more easily build transparent
> intermediaries. Any mechanism that allows intermediaries without being
> authorized by either endpoint is by definition a security vulnerability in
> the protocol.
>
> I think the benefits of TLS also outweigh the "amateur server implementor"
> argument. I don't think we want to make it easy to implement a security
> hole.
>

How would requiring TLS impact games over WebSocket, such as GWT Quake?
 Maybe one day we will have a connection-oriented datagram protocol for WS,
but until then we have to make do with running over TCP.  Adding encryption
overhead might render WS unusable for this purpose.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 21 Jul 2010 10:10:48 +1000

On 21 July 2010 08:33, Roberto Peon <[email-masked]> wrote:

> We need browser-owned connections with a security model that allows
> multiple tabs to share the connection safely.

+1

Currently websocket solves none of my pain points implementing large scale
cometd solutions, namely the issues associated with the number of
connections, keeping them alive and detecting when they fail.

The only advantage websockets currently have over long polling is that they
have marginally better latency and better data density, but neither of these
are killer issues for a large class of application.

Surely we can come up with a solution that has good latency, good data
density and address some if not all of the issues associated with connection
management.  Delegating the connection management to the application
developer does not make that problem any simpler, it just means that it will
be done badly more often than not.

cheers

---

From: Roberto Peon <[email-masked]>
Date: Tue, 20 Jul 2010 17:11:16 -0700

> overhead might render WS unusable for this purpose.

I'd be pretty surprised if SSL added enough overhead that it made WS
unsuitable for games. It is far, far, far, more likely that the fact that
we're using TCP renders it useless for certain classes of games.

Also, while perhaps I don't count as an 'amateur' programmer, I did add
SSL/TLS support to a server recently. It tool no more than a day to do! t
was surprisingly easy!

-=R

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 21 Jul 2010 10:11:23 +1000

On 21 July 2010 09:22, Scott Ferguson <[email-masked]> wrote:

> Mike Belshe wrote:
>
>>
>> For as adamantly as Ian states that it should be a requirement, I am just
>> as adamant that it should not.
>>
>> Is there some way we can vote to either keep or nullify this requirement
>> now and never come back to it again?  I'm tired of this obstacle holding
>> everything up.
>>
> +1

+1

---

From: Roberto Peon <[email-masked]>
Date: Tue, 20 Jul 2010 17:15:02 -0700

On Tue, Jul 20, 2010 at 5:01 PM, Maciej Stachowiak <[email-masked]> wrote:

> Also, in practice it does not seem to be the case that intermediaries are
> updated faster than clients or servers. One of the challenges to deploying
> new protocols or protocol features is old intermediaries that are not
> updated.
>

Hear, hear. This is the part about intermediaries that is really terrible.
They ossify the net.
There is another concern w.r.t. intermediaries.
Schools and other institutions will block port 443 if they feel unacceptable
content is flowing over it and they have other means of dealing with it.
I believe that this means that we need an 'authorized proxy' model. No proxy
would be fully transparent (unless it was a reverse proxy representing the
real endpoint), however it should be exceptionally easy to install a policy
allowing the proxy explicitly. It is a hair more annoying for the user than
no proxy, but it gives schools, etc. a way to control the computers that
they own without blocking port 443 for everyone and everything.

> Browsers don't have to accept broken certs for WebSocket with a
> clickthrough the way many still do for Web sites. I would argue that they
> should not, because the security consequences of the decision cannot
> possibly be understood by the typical user.
>
>

---

From: Roberto Peon <[email-masked]>
Date: Tue, 20 Jul 2010 17:16:18 -0700

>>> now and never come back to it again?  I'm tired of this obstacle holding

yes, please. +1.
-=R

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 21 Jul 2010 10:35:42 +1000

On 21 July 2010 10:15, Roberto Peon <[email-masked]> wrote:

>
> Schools and other institutions will block port 443 if they feel
> unacceptable content is flowing over it and they have other means of dealing
> with it.
> I believe that this means that we need an 'authorized proxy' model. No
> proxy would be fully transparent (unless it was a reverse proxy representing
> the real endpoint), however it should be exceptionally easy to install a
> policy allowing the proxy explicitly. It is a hair more annoying for the
> user than no proxy, but it gives schools, etc. a way to control the
> computers that they own without blocking port 443 for everyone and
> everything.
>

+1

As Mike has said, there are good motives and bad motives for intermediaries.
We have to find a way to allow the good motives to be supported, even if
that costs a bit of complexity in the handshake/protocol.

Having explicit intermediaries would be very useful for knowing  timeouts,
optimizing keepalives and security considerations.

cheers

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 21 Jul 2010 08:39:38 +0800

> For as adamantly as Ian states that it should be a requirement, I am just as adamant that it should not.
> Every protocol expert I've spoken with agrees that amateur protocol implementors should not be a requirement.
> Is there some way we can vote to either keep or nullify this requirement now and never come back to it again?  I'm tired of this obstacle holding everything up.

+1

That shouldn't mean that simplicity is not a design goal, just that it's not an overriding constraint.

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 21 Jul 2010 08:47:28 +0800

> > Schools and other institutions will block port 443 if they feel unacceptable content is flowing over it and they have other means of dealing with it.
> > I believe that this means that we need an 'authorized proxy' model. No proxy would be fully transparent (unless it was a reverse proxy representing the real endpoint), however it should be exceptionally easy to install a policy allowing the proxy explicitly. It is a hair more annoying for the user than no proxy, but it gives schools, etc. a way to control the computers that they own without blocking port 443 for everyone and everything.
>
> +1
>
> As Mike has said, there are good motives and bad motives for intermediaries. We have to find a way to allow the good motives to be supported, even if that costs a bit of complexity in the handshake/protocol.
>
> Having explicit intermediaries would be very useful for knowing  timeouts, optimizing keepalives and security considerations.

+1

Despite the fact that a policy-enforcing proxy often causes a lot of grief, the alternative is worse.  We don't need another arms race.  Recognizing that people will enact policy, and providing ways that clients can cooperate, is more constructive than trying to circumvent policy measures.

--Martin

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 21 Jul 2010 01:50:38 +0100

John Tamplin wrote:
>    I still don't see the argument for servers written by amateurs. Â I
>    have used far more quick-and-dirty web clients (telnet,
>    socket/connect/write in C, wget/curl, etc) than I have quick-and-dirty
>    servers. Â If some amateur needs to write a server anyway, they aren't
>    going to write it from scratch even if it were possible for them to do
>    so -- they will find some library.
>    clients. Â Meanwhile, the servers are going to almost certainly be on
>    more powerful machines, though granted they have more connections to
>    support.

But I must say I don't see a problem with keeping it in reach to
amateur programers, at the same time as supporting smarter features.
The two requirements are easily reconciled.

I'm a big fan of the "complex protocols" end of the spectrum - because
I consistently advocate multiplexing, feature negotiation, proxy
opt-ins, message splitting and other "complex" things; all of them are
driven by the desire for good performance (over a variety of
measures), and providing a consistently simple "it just works" API to
the API users.

All that's needed, for amateur programmer compatibility, is for the
"complex" performance-enhancing features to be optional negotiated
features on top of an extremely simple base protocol.  And for the
negotiation to be really simple too.

That's a good idea anyway.

We have a great analogue of this in HTTP's optional persistent
connections.  Simple clients and simple servers don't handle
persistent connections; better ones do, and every combination
works just fine.

Here's how I imagine it working:

The big browsers get decent WS implementations, and are good at
performance-related things that are "hidden under the API", like
automatic sharing of connections, merged keepalives, and maybe even
things like network-signalled-callback on phones.  They request the
most useful enhancers in negotiation, but they work if the answer is
"no".  They can continue to be extended if we think of newer techniques.

Well known servers (Apache etc.) get decent implementations, and can
handle the performance-enhancing features.  They negotiate "yes"
to all the most useful enhancers.

Some proxies like those on a mobile network recognise negotiation and
can add enhancements like network-signalled-callback mentioned above,
and even connection fan-in / fan-out if that proves to be technically
workable.  Other proxies, including HTTP ones, don't participate in
negotiation and are simply transparent.

Joe Random wants to write a quick WebSocket server in half a page of
Python.  That's fine: He implements the minimum base protocol, which
is designed to be very simple (providing the minimum needed for the
WebSocket API to work) and it *just works* with all browsers and other
clients.  Negotiation is basically "no" to all features.

Same when Joe Random wants to write a quick client.

The most important part of this model is that negotiation is sane and
extensible right from the start, and that it has room for all the
interested parties: "below the API" client requested features, proxy
requested/recognised features, server requested features.  The
existing subprotocol field provides the fourth: "above the API" client
requested features.

As setup latency is an issue, it is also important that negotiation
does not incur avoidable round trips.

A second important thing is subtler: It is important that features
negotiated transparently "below the API" are carefully designed to be
*invisible* to WebSocket API users.

Connection sharing is the best example (but it also applies to
keepalive merging): Sending an erroneous message on one channel of a
shared connection must not break all the others; if a receiver event
queue fills, thereby stalling that channel, it must not block messages
from flowing to the others; and sending a very large message on one
channel must not block others for a long time.  Otherwise the
behaviour isn't transparent enough as an *automatic* under-the-hood
replacement for separate connections.

-- Jamie

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 20 Jul 2010 18:20:46 -0700

> Having explicit intermediaries would be very useful for knowing  timeouts, optimizing keepalives and security considerations.

In the TLS case, authorizing an intermediary has to require some form of explicit configuration on at least one of the client or the server. Otherwise it is vulnerable to MITM. On the server side, the intermediary could plausibly have access to the server's certificate, so it can just pretend to be the endpoint to the client. On the client side, there has to be some way to authorize a root cert for use with WebSocket, so the intermediary can MITM you. If it needs to be easier than this, then that seems like primarily a browser UI issue, not a protocol issue.

We could do other things for the non-TLS case, but I am increasingly convinced that the non-TLS case is uninteresting.

Regards,
Maciej

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 06:44:56 +0200

> which fail, even after the WebSocket handshake.

I'm not surprized at all by this, the handshake it build so that most
cases which would transparently work by default will fail ! This is the
big failure of this protocol right now. We're trying to break compatibility
with 10 years of work towards interoperability between components that
try their best to be transparent to each other, and everything will have
to be reinvented from scratch.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 06:48:16 +0200

> I think the benefits of TLS also outweigh the "amateur server implementor" argument. I don't think we want to make it easy to implement a security hole.

There's no "security hole" here. If you don't need security, use port 80,
if you need security, use port 443. That's been working that way for the
web for decades and that's no secret even for non-technical people. Clear
traffic offers reliability and large possibilities while ciphered traffic
offers security with less possibilities. Nothing new here.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 06:52:02 +0200

> we're using TCP renders it useless for certain classes of games.

SSL adds a lot of overhead due to the block mode for protocols exchanging
a few bytes in each direction. Think about chats where each typed character
will be sent as a few tens of bytes. This costs in terms of network bandwidth,
CPU and perceived latency for the user. Compare SSH with telnet over slow
links.

Also, SSL will limit adoption because at many places it will simply not
be available. It's very common to white-list only a handful of SSL sites
in many enterprises, schools, etc...

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 07:06:43 +0200

> that costs a bit of complexity in the handshake/protocol.

In my opinion, supporting the good ones should exactly cost nothing : just
rely on what they correctly support : HTTP. Many intermediaries are there
because they're required in the infrastructure they're deployed in :
  (client side)
  - protocol-based proxy farm selection (eg: use the WS-compatible farm 
    for WS and the other one for HTTP)
  - proxy load-balancers
  (server side)
  - IDS/Protocol inspection
  - reverse-proxy load balancers
  - reverse-proxies (SSL, compression, WAF, inter-DMZ bounce)
  - URL switching to select server pools

The "bad motives" intermediaries are generally the ones that do transparent
caching and such features. But they're not really "bad motives" in that
they're here to improve users' bandwidth. However, they're often doing
badness because the users can't decide to bypass them. But since they're
themselves generally load-balanced, it's not difficult to add information
in the request (eg: "Upgrade: WebSocket") that the load balancers can use
to bypass the transparent proxy farm. There are generally a lot of cache
bypass rules on such farms because ISPs are used to receive users complaints
for site XXX or YYY which does not work. So adding bypass rules to such
farms is just a matter of days at most places, once the first issues are
encountered. And contrary to a common belief, it's probably the place
where the fixes/workarounds can be implemented the fastest, because there
are no security considerations and we know that ISPs don't take too much
time validating products.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 07:11:03 +0200

On Wed, Jul 21, 2010 at 01:50:38AM +0100, Jamie Lokier wrote:
> John Tamplin wrote:
> >    I still don't see the argument for servers written by amateurs. ? I
> >    servers. ? If some amateur needs to write a server anyway, they aren't
> >    clients. ? Meanwhile, the servers are going to almost certainly be on

+1

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 07:13:59 +0200

+1

Willy

---

From: Roberto Peon <[email-masked]>
Date: Tue, 20 Jul 2010 22:37:36 -0700

I have some experience with dealing with broken HTTP client implementations
in nearly-impossible-to-update devices such as televisions, phones, etc.
We're starting from scratch, so hopefully we won't have the confusion of
poor implementations that HTTP does, but I'd end up really sad if I had to
make my server not use advanced features of a *new* protocol because some
clients identify their capabilities wrongly.

Yes, we've had to do this. It is really, really, painful.

To be clear: I'm not motivated to make it possible for amateurs to make
servers or clients.
I am motivated to ensure that the protocol *just* complex enough to do what
we want, and otherwise as simple and unambiguous as possible.

If that enables "amateur programmers" to write clients or servers that speak
a subset of the protocol... great!
If not, hopefully they can hack on a reference implementation!

I strongly hope that we'll end up with a 100% functional reference
implementation that we publish with the spec.
I also strongly hope that we'll end up with a conformance suite (as a result
of creating the reference implementation).

-=R

On Tue, Jul 20, 2010 at 5:50 PM, Jamie Lokier <[email-masked]> wrote:

> John Tamplin wrote:
> >    I still don't see the argument for servers written by amateurs. � I
> >    servers. � If some amateur needs to write a server anyway, they aren't
> >    clients. � Meanwhile, the servers are going to almost certainly be on

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Jul 2010 08:07:42 +0200

On 21.07.2010 02:01, Maciej Stachowiak wrote:
> ...
> Also, in practice it does not seem to be the case that intermediaries
> are updated faster than clients or servers. One of the challenges to
> deploying new protocols or protocol features is old intermediaries that
> are not updated.
> ...

The same has been said about clients (IE6) and servers (Apache httpd 
prior to implementing DefaultType None) just within a week :-)

---

From: Simone Bordet <[email-masked]>
Date: Wed, 21 Jul 2010 09:56:05 +0200

>>>> now and never come back to it again?  I'm tired of this obstacle holding

+1 as well.

Simon
-- 
http://bordet.blogspot.com
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz

---

From: Simone Bordet <[email-masked]>
Date: Wed, 21 Jul 2010 10:07:12 +0200

Hi,

> of creating the reference implementation).

I could not have said it better.

Why cannot "amateur programmers" be exposed to WebSocket via an API
that is a higher level API than the socket API ?
This is already the way it is done in JavaScript where "amateur
programmers" do not need to care about HTTP headers during handshake,
nor to add 0x00 and 0xFF at start and end of their string messages.
The same can be offered on server side, making it easy for "amateur
programmers", but allowing the protocol to gain all the features
discussed.

I would stretch my luck even further and bet that even if an API will
not be designed for server-side, it will be eventually written, and
that "amateur programmers" will use that API in their applications
rather than dealing with sockets and having to read the protocol spec
to understand what to read/write from the socket streams.

Simon

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 21 Jul 2010 02:13:54 -0700

On Jul 20, 2010, at 11:07 PM, Julian Reschke wrote:

> The same has been said about clients (IE6) and servers (Apache httpd prior to implementing DefaultType None) just within a week :-)

I can't say for sure which has the fastest rate of being updated, I just reject the premise that intermediaries are in general updated faster to a significant degree. HTTP pipelining is a clear example of a feature that is largely blocked by intermediaries not being updated enough. When a protocol feature requires client or serve updates, then often it can be deployed incrementally when only a subset of clients or servers is updated, since they can come up with a way to signal to each other. However, if it is possible for intermediaries to get in the chain completely transparently, without any way for the client or server to know they exist, then it's quite difficult, perhaps impossible to come up with a reliable signal. In my opinion, to the extent we support intermediaries, the design should require at least one of the client or server to be explicitly aware of the intermediary. We should not support magical transparent intermediaries where neither the client nor server has an obvious way to detect its existence; in fact we should design the protocol to prevent such intermediaries from being created.

Regards,
Maciej

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 21 Jul 2010 14:21:01 +0200

On Wed, Jul 21, 2010 at 02:13:54AM -0700, Maciej Stachowiak wrote:
> HTTP pipelining is a clear example of a feature that is largely blocked by intermediaries not being updated enough.

It's not always a matter of not being "updated enough", but a matter of
"not always being implementable at all at a reasonable cost". Supporting
pipelining on intermediaries can require a very complex software architecture
with complex resource management to support multiple requests in flight,
sometimes for little added value. What is important though is that pipelined
requests all get processed in the correct order just as if they were not
pipelined. But are their still that many products that are incompatible
with pipelining ?

> When a protocol feature requires client or serve updates, then often it can be deployed incrementally when only a subset of clients or servers is updated, since they can come up with a way to signal to each other.

This is what was done with early WS implementations and multiple client and
server implementations have existed, BTW.

> However, if it is possible for intermediaries to get in the chain completely transparently, without any way for the client or server to know they exist, then it's quite difficult, perhaps impossible to come up with a reliable signal. In my opinion, to the extent we support intermediaries, the design should require at least one of the client or server to be explicitly aware of the intermediary.

Not necessarily, if we rely on standards and the intermediaries support those
standards, there is no need for the ends to know about them. Neither your
client nor your servers are aware of the number of routers and firewalls
in the chain because the service they offer relies on well established
standards and they act transparently. The same must be true with intermediaries
here. By using a standard HTTP handshake, standard-compliant intermediaries
can work without either of the client or server being aware of it. But if we
use non-standard derivative of well-established standards, of course we're
impacted a lot by standard-compliant intermediaries.

> We should not support magical transparent intermediaries where neither the client nor server has
>   an obvious way to detect its existence; in fact we should design the protocol to prevent such intermediaries from being created.

I disagree here. Intermediaries will be created whatever efforts you put to
try to make them fail. They will exist because people who deploy them need
them, and the quality of their implementation is no problem for them. The
dirtier we proceed, the uglier they will look like and the poorest the
resulting quality of service for the client.

There must not be "magical transparent intermediaries", there must be some
natural and implicit support for standard-compliant intermediaries so that
any new requirement can happen within a standard, and involves no "magics".

Regards,
Willy

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 21 Jul 2010 05:33:49 -0700

On Jul 21, 2010, at 5:21 AM, Willy Tarreau wrote:

> standards and they act transparently.

Yes, but the core protocols of the Internet (IP, TCP, UDP, ICMP, etc) have update time scales on the order of decades. Even with exhaustion of the address space staring us down, rolling out IPv6 is a massive engineering undertaking. And indeed, if you try to use IPv6 on your home network today, it will matter to you whether the intermediaries between you and your other endpoint support it.

The bottom line is this: if you allow intermediaries to participate without the explicit knowledge of either endpoint, it may take decades to roll out significant protocol changes. All communication will be reduced to the least common denominator of the worst-behaving intermediaries. That may be a cost worth bearing for low-level protocols, but it is completely unacceptable for an application-level protocol in an area of high innovation.

> The same must be true with intermediaries
> can work without either of the client or server being aware of it.

And likewise they can (and apparently often will) fail without either the client or the server being aware of it. This mistake has already been made with HTTP, let's learn from it instead of repeating it.

> them, and the quality of their implementation is no problem for them.

That's exactly the problem - low-quality intermediaries are not a problem for the people running them, but they create global negative externalities.

Regards,
Maciej

---

From: Ian Hickson <[email-masked]>
Date: Wed, 21 Jul 2010 22:17:02 +0000 (UTC)

On Tue, 20 Jul 2010, John Tamplin wrote:
> > mentioned before, if people have different goals or priorities, I have 
> > no problem with separate protocols also being designed to address 
> > those -- Web Sockets doesn't have to be everything for everybody.)
> 
> I still don't see the argument for servers written by amateurs.  I have 
> used far more quick-and-dirty web clients (telnet, socket/connect/write 
> in C, wget/curl, etc) than I have quick-and-dirty servers.

That's largely because most protocols are such that you can't make a 
compliant server quickly. The one exception (which isn't a network 
protocol but is relatively similar for the purposes of this discussion) is 
CGI. That's a simple protocol, and there are many implementations of the 
server side of the protocol (that is, CGI scripts; the HTTP server is the 
client of the CGI protocol). Now, most people use libraries, of course, 
and that's fine. But the protocol is simple enough that you don't _have_ 
to use a library, and some people don't.

> If some amateur needs to write a server anyway, they aren't going to 
> write it from scratch even if it were possible for them to do so -- they 
> will find some library.

Yes, for most existing protocols that is the sad reality. I think that's a 
failure on our part (the part of protocol designers). We can enable much 
greater experimentation, innovation, learning, and so forth, if we make 
lower the bar to the level of CGI scripts rather than keeping it at the 
level of HTTP servers.

> clients.  Meanwhile, the servers are going to almost certainly be on 

In the case of Web Sockets, most clients (by usage) are going to be Web 
browsers, so if we are forced to put complexity into the protocol, we 
should move it to the browser side. But in general we should keep that 
side simple too, so that custom clients can also be written.

> negotiation to be really simple too.

Once the protocol is proven, I think it would make sense to add optional 
features. The protocol is designed to make that easy for us.

> fewer file descriptors, it requires fewer handshakes.

This is pretty easily implemented at the application layer using shared 
workers.

> just as adamant that it should not.

It's quite possible that this working group should be working on two (or 
more) protocols, rather than trying to solve all our problems with Web 
Sockets. Or it could be that Web Sockets isn't the right solution for the 
problems that most people in this working group are trying to solve.

Web Sockets is a trivially-implementable TCP-for-browsers protocol for 
long-lived connections. If that doesn't sound like the protocol you need, 
then you need a different protocol. If that isn't the protocol this 
working group wants, then this working group wants another protocol. I 
don't think there's anything wrong with that, but then we should work on 
that protocol, not Web Sockets.

> But more importantly, this single issue has been holding protocol 
> progress hostage.  Naturally, any feature has some amount of complexity.  
> But this requirement creates an invisible, and subjective barrier to 
> each feature. Is the feature too complex for an amateur programmer?  
> Nobody knows, and everyone disagrees, because it is a subjective 
> criteria.  So we spin and can't agree.

I disagree that the protocol's development has been in any way held up by 
this. I agree that we've talked about it a lot in the working group, but 
the spec itself hasn't been affected by this as far as I can tell.

> implementors should not be a requirement.

And what did the protocol amateurs you spoke to say? It seems unsurprising 
that people whose careers are based on being able to solve complicated 
protocol problems should want protocols to be complicated.

---

From: Ian Hickson <[email-masked]>
Date: Wed, 21 Jul 2010 22:47:12 +0000 (UTC)

On Wed, 21 Jul 2010, Maciej Stachowiak wrote:
> 
> In my opinion, to the extent we support intermediaries, the design 
> should require at least one of the client or server to be explicitly 
> aware of the intermediary.

Agreed. I think we should add this to the requirements.

> We should not support magical transparent intermediaries where neither 
> the client nor server has an obvious way to detect its existence; in 
> fact we should design the protocol to prevent such intermediaries from 
> being created.

The only way I can see to do this would be to require TLS (or some 
equivalent asymmetric encryption scheme), which I think is too high a 
cost. Otherwise, in principle, I agree with this too.

---

From: Mike Belshe <[email-masked]>
Date: Wed, 21 Jul 2010 16:28:38 -0700

> the spec itself hasn't been affected by this as far as I can tell.

To be specific, our earlier discussions about error handling, large frames,
and binary frames were all cut short under the "too hard for amateur
programmers".

Based on the +1s in this thread, I think many agree that this has occurred.
 I don't yet know if we mostly agree that amateur programmers is not an
overriding criteria, so how do we decide?  Can we get a group decision on
that now?

Once we have a decision on that, we can finish looking at the other issues
on the table.  If we agree that "amateur programmer support" is not a
feature, then it cannot be held up as an argument for or against any
proposed solution once we've decided upon it.  Or, if it is a priority, then
we should weigh all proposed changes against it to ensure that the
requirement is met.

Can we call a vote on this issue alone and settle it?  Chair-people, can you
offer guidance?  Without a decision on the importance of this issue, I know
we'll go circular again when deciding on other issues.

> protocol problems should want protocols to be complicated.

I have yet to meet anyone that wants the protocol to be complicated, so
please don't say that I asked for that.  What I asked for is that the
protocol not be constrained by the need to be implemented by "amateur
programmers".

Mike

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 22 Jul 2010 09:38:31 +1000

> and binary frames were all cut short under the "too hard for amateur
... and it was not just that.  This list woke up again and was just starting
to make some good technical discussions.....  Then all of a sudden the whole
conversation is back to talking about amateur programmers, HTTP-like
handshakes, no extensions unapproved by central control, what do you mean I
can't edit the draft 37 times a day? do we really need to talk about changes
first when it is so obvious to me that I'm right...

welcome to Hybi ground hog day!

---

From: Mike Belshe <[email-masked]>
Date: Wed, 21 Jul 2010 16:42:47 -0700

On the TLS issue, we're going in circles.  Both sides are correct on the
facts about the pros and cons of TLS (more or less).  I don't think the spec
should require TLS.  But, would it be possible to define alternate
handshakes based on whether TLS is used?

If we use the currently defined handshake, TLS will be at a performance
disadvantage to insecure WebSockets, because TLS will require more
round-trips.

Could we define, as part of the WebSocket protocol, that when WebSocket is
negotiated over TLS it uses the TLS/NPN extension to effectively fold the
WebSocket Upgrade handshake into the TLS handshake?  This removes a round
trip from the WebSocket startup time.  The next-protocol-negotiation
extension is defined here:
http://tools.ietf.org/html/draft-agl-tls-nextprotoneg-00

If we don't do this, websites in the future will be discouraged from using
secure websockets because secure websockets will be slower than insecure
ones.  By contrast, with the improvements in TLS (if they come to fruition),
if WebSockets defines a mechanism for hanshaking over TLS/NPN, WebSockets
over TLS will negotiate with fewer round trips than WebSockets over HTTP.
 Hopefully this will encourage more use of the secure facility, without
making it a requirement for those that really don't want it.
<http://tools.ietf.org/html/draft-agl-tls-nextprotoneg-00>
Mike

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 22 Jul 2010 09:55:50 +1000

+1

there has been on opinion expressed that we can only ever have 1 websocket
handshake.  Thus because we are starting with port 80, the handshake is
currently some mutant child of HTTP that must be sent even if another port
is used and no HTTP is involved.

This approach is  stuck trying to fixate on the exact syntax of the
handshake, literally down to counting spaces!

Instead we need to establish the semantic context in which a websocket
connection can be established, and then allow multiple ways to establish
that context.   HTTP upgrade is one such way, TLS/NPN is another.  I'm sure
there will eventually be more (maybe SPDY will support a DOWNGRADE method :)

If the semantic context is secure (eg contains random token generated after
any URL was formed by application), then does it really matter what syntax
was used to establish the context?

regards

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 02:11:12 +0200

On Thu, Jul 22, 2010 at 09:55:50AM +1000, Greg Wilkins wrote:
> that context.   HTTP upgrade is one such way, TLS/NPN is another.

+1 !

Regards,
Willy

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 21 Jul 2010 17:15:18 -0700

On Jul 21, 2010, at 4:28 PM, Mike Belshe wrote:

> To be specific, our earlier discussions about error handling, large frames, and binary frames were all cut short under the "too hard for amateur programmers".
> 
> Based on the +1s in this thread, I think many agree that this has occurred.  I don't yet know if we mostly agree that amateur programmers is not an overriding criteria, so how do we decide?  Can we get a group decision on that now?
> 
> Once we have a decision on that, we can finish looking at the other issues on the table.  If we agree that "amateur programmer support" is not a feature, then it cannot be held up as an argument for or against any proposed solution once we've decided upon it.  Or, if it is a priority, then we should weigh all proposed changes against it to ensure that the requirement is met.
> 
> Can we call a vote on this issue alone and settle it?  Chair-people, can you offer guidance?  Without a decision on the importance of this issue, I know we'll go circular again when deciding on other issues.

The IETF generally does not use voting as a means of decision-making. I don't have personal experience in how decision-making is done when there is not clear consensus, but these documents describe how it's supposed to work.

http://tools.ietf.org/html/rfc2026
http://tools.ietf.org/html/rfc4677

Regards,
Maciej

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 21 Jul 2010 17:30:40 -0700

Mike Belshe wrote:
>
>
> On Wed, Jul 21, 2010 at 3:17 PM, Ian Hickson <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
> To be specific, our earlier discussions about error handling, large 
> frames, and binary frames were all cut short under the "too hard for 
> amateur programmers".
The entire spec has frozen in a broken state for months because of Ian's 
intransigence (frozen = "not affected"), and the chairs' unwillingness 
to address the issue.

* It doesn't have any support for binary data.

* It doesn't have any support for basic protocol concepts like chunking.

* It doesn't have any way of extending to the multiplexing/flow control 
that's been brought up numerous times.

* It doesn't have any to handle error frames.

* It doesn't address the keepalive/ping issue at all.

* It doesn't address the RTT issues.

* It doesn't even conform to the HTTP specification correctly.

The current spec is a joke.

The only reason discussion started up again was because of the 
possibility of getting a real editor for the spec and moving on.

If the chairs decide to do nothing -- again --  we'll be stuck with a 
bad spec that solves no problem well.

>     protocol problems should want protocols to be complicated.
FFS. Nice combination ad-hom/strawman there.  [Chairs? This is your editor.]

The counterproposal at

http://hessian.caucho.com/websockets/draft-ferg-hybi-websockets-latest.html

not complicated at all: it's a trivial protocol, and yet it solves the 
HyBi requirements, including requirements that the current draft 
ignores, as well as being extensible to muxing requirements.

A dozen members of this list could come up with proposals as simple as 
that one and most would improve on it.

-- Scott

---

From: Adam Barth <[email-masked]>
Date: Wed, 21 Jul 2010 17:39:41 -0700

On Wed, Jul 21, 2010 at 5:30 PM, Scott Ferguson <[email-masked]> wrote:
> not complicated at all: it's a trivial protocol, and yet it solves the HyBi
> requirements, including requirements that the current draft ignores, as well
> as being extensible to muxing requirements.
>
> A dozen members of this list could come up with proposals as simple as that
> one and most would improve on it.

This protocol is highly insecure.  First, the server offers no proof
that it understands the protocol, so it's very likely that it's
vulnerable to cross-protocol attacks.  Second, the client doesn't even
require the server to respond at all before sending (almost) arbitrary
content over the socket, which is very likely to lead to disaster.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 21 Jul 2010 17:43:34 -0700

On Jul 21, 2010, at 5:30 PM, Scott Ferguson wrote:

> not complicated at all: it's a trivial protocol, and yet it solves the HyBi requirements, including requirements that the current draft ignores, as well as being extensible to muxing requirements.

Can you explain how this proposal provides an HTTP-compatible upgrade in the non-TLS case? Or how it provides protection against cross-protocol attacks? I think those are two pretty important requirements, and it is not obvious to me how they are met.

Regards,
Maciej

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 21 Jul 2010 18:20:36 -0700

> content over the socket, which is very likely to lead to disaster.
Like I said, most people on the list could improve on it. (Since my 
interest is on the stream encoding itself, I'm perfectly willing to 
defer to others expertise on fixing the handshake.) That said,

1. The server's proof is its initial HELLO control frame. From a 
security standpoint, I don't see how this is different than a HTTP 
server's proof that it understands HTTP. If you want a hash of the 
input, it's trivial to add a nonce header for the client and a 
calculated hash for the response.

1a. How so? I'd like to see an actual attack.

2. No different from HTTP POST. The client may wait for the response if 
it it wants, but clients which want the equivalent responsiveness of 
HTTP POST can send data immediately. It's up to the requirements of the 
application. This design allows both application models.

-- Scott

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 21 Jul 2010 18:31:57 -0700

> Can you explain how this proposal provides an HTTP-compatible upgrade in the non-TLS case? Or how it provides protection against cross-protocol attacks? I think those are two pretty important requi…
The HTTP negotiation was in an earlier draft:

  http://hessian.caucho.com/websockets/draft-ferg-hybi-websockets-m1.html

The previous non-HTTP link was based on the discussion we had a few 
months ago for the non-HTTP case, just showing how you could have the 
handshake as part of the protocol syntax itself.  (And as I mentioned in 
the other response, my main focus was on the stream part of the 
protocol. There's nothing interesting or innovative in the handshake.)

The only thing I changed in the HTTP portion was to make it actually HTTP.

But focusing on the handshake is really missing the point of that proposal.

-- Scott

---

From: Adam Barth <[email-masked]>
Date: Wed, 21 Jul 2010 18:34:55 -0700

On Wed, Jul 21, 2010 at 6:20 PM, Scott Ferguson <[email-masked]> wrote:
>>> HyBi
> Like I said, most people on the list could improve on it. (Since my interest
> is on the stream encoding itself, I'm perfectly willing to defer to others
> expertise on fixing the handshake.) That said,
>
> 1. The server's proof is its initial HELLO control frame. From a security
> standpoint, I don't see how this is different than a HTTP server's proof
> that it understands HTTP. If you want a hash of the input, it's trivial to
> add a nonce header for the client and a calculated hash for the response.
>
> 2. No different from HTTP POST. The client may wait for the response if it
> it wants, but clients which want the equivalent responsiveness of HTTP POST
> can send data immediately. It's up to the requirements of the application.
> This design allows both application models.

I don't think you understand how cross-protocol attacks work, and I'm
not that interested in educating you about them.  HTTP POST requests
are also very poorly designed in this way and have been the cause of a
number of cross-protocol attacks, most recently against IRC.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 21 Jul 2010 18:43:30 -0700

On Jul 21, 2010, at 6:31 PM, Scott Ferguson wrote:

> The previous non-HTTP link was based on the discussion we had a few months ago for the non-HTTP case, just showing how you could have the handshake as part of the protocol syntax itself.  (And as I mentioned in the other response, my main focus was on the stream part of the protocol. There's nothing interesting or innovative in the handshake.)
> 

You said it solved all the requirements, so I was interested to find out if this was really so. It looks like that is not the case with your current draft, although it does have some nice properties.

Regards,
Maciej

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 21 Jul 2010 19:14:20 -0700

> You said it solved all the requirements, so I was interested to find out if this was really so. It looks like that is not the case with your current draft, although it does have some nice propertie…
You're right about that. I do apologize for the overreach.

The missing binary/framing/control/stream requirements are the issue I'm 
worried about, and wanted to show it was possible to satisfy with a 
simple protocol, creating a more formal writeup of the related framing 
discussions on the list. The handshake was tacked on only for completeness.

Basically a proof of existence, because those requirements were being 
written off as too difficult. (And because no one else was stepping up 
to do it and they're trivial.)

Also, some of the related discussions were not grounded in an actual 
protocol.  For example, Jamie Lockier's ideas to reduce round trips 
sound like they're impossible to reconcile with the cross-site scripting 
issues. In the concrete proposal, both you and Mike saw problems 
immediately, but I don't recall his ideas being dismissed in that 
thread. (Although it was a few months ago.)

-- Scott

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 21 Jul 2010 21:14:40 -0700

> number of cross-protocol attacks, most recently against IRC.
Well, we have two conflicting desires that need to be resolved:

  1) To make WebSockets initial latency no worse than HTTP. (If I 
understand correctly, this is the issue raised by Jamie and also Greg.)

  2) cross-site scripting

I see only 3 possible resolutions:

  a) That 1 & 2 are irreconcilable and WebSocket performance must be 
worse than HTTP/comet because its security must be better than HTTP.

  b) WebSocket performance must be as good as HTTP/comet as long as its 
security is no worse than HTTP.

  c)  Both 1 & 2 are resolvable and the solution is XXX.

If (a) or (b) are selected, it should be a conscious decision, not 
chosen by default. I assume you're in camp (a)?

If you believe (c), than it seems to me you either need to propose an 
XXX or suggest fixes to a flawed XXX proposal to make it work.

The current official draft proposes (a), but it seemed to me that the 
group is divided on the issue, and I made an attempt at an XXX. Perhaps 
I misunderstood, everyone's in camp (a).

-- Scott

(I suppose there might be a (d): browser clients are restricted to (a) 
but non-browser clients can use (b). But that would probably be too 
messy for a spec.)

---

From: Adam Barth <[email-masked]>
Date: Wed, 21 Jul 2010 21:40:00 -0700

The more I read on this list, the more I think that folks here don't
understand the requirements on the protocol, especially the security
requirements.  That's understandable if you haven't worked extensively
with the browser security model.  Fortunately, Ian is an expert on
these topics and has more time to reply to this list than I do.  I'll
try to address some of the confusion in this email, but I might not be
able to answer all the followup questions.

On Wed, Jul 21, 2010 at 9:14 PM, Scott Ferguson <[email-masked]> wrote:
>  1) To make WebSockets initial latency no worse than HTTP. (If I understand
> correctly, this is the issue raised by Jamie and also Greg.)
>
>  2) cross-site scripting

There is surprisingly little interaction between cross-site script and
the security requirements on WebSockets.  However, there is a large
interaction between the security requirements on WebSockets, the
browser's same-origin policy, and cross-protocol vulnerability.
Please note that cross-protocol vulnerabilities have nothing to do
with cross-site scripting.

>  a) That 1 & 2 are irreconcilable and WebSocket performance must be worse
> than HTTP/comet because its security must be better than HTTP.

I don't understand what this means.  How could latency requirements
for a network protocol be irreconcilable with cross-site scripting?
The two operate at completely different layers of abstraction.

> security is no worse than HTTP.

What does it mean for security to be no worse than HTTP?  When
thinking about security, it's important to consider what security
goals you're trying to achieve in the presence of what sort of
adversary.  It's far from a one dimensional quantity that can be
"better" or "worse" than another.  Your statement is basically
nonsensical.

>  c)  Both 1 & 2 are resolvable and the solution is XXX.

It's clear to me that we can achieve both performance and security.
We've seen several proposals that achieve that.  Of course, those
aren't the only requirements we're aiming to meet...

> If (a) or (b) are selected, it should be a conscious decision, not chosen by
> default. I assume you're in camp (a)?

What's the difference between choosing something by default versus a
conscience choice?  From my statements above, I think you'd put me in
camp (c).

> If you believe (c), than it seems to me you either need to propose an XXX or
> suggest fixes to a flawed XXX proposal to make it work.

I've already did so back in May:

http://www.ietf.org/mail-archive/web/hybi/current/msg01948.html

This is different from the protocol in the current working draft
because it requires TLS.  I think TLS+NPN is a much better design than
the current HTTP-ish design, for the reasons outlined in that email.
The main (only?) disadvantage is that it requires amateur server
implementations to use a TLS library.

> The current official draft proposes (a), but it seemed to me that the group
> is divided on the issue, and I made an attempt at an XXX. Perhaps I
> misunderstood, everyone's in camp (a).

It's fine to propose designs as food for thought, but claiming that
you've addressed all the requirements when you haven't even met the
basic security requirements is a bit brash.

> (I suppose there might be a (d): browser clients are restricted to (a) but
> non-browser clients can use (b). But that would probably be too messy for a
> spec.)

I don't care what non-browser clients do.  I'm here to help design a
protocol for use by browsers.

Adam

---

From: Roberto Peon <[email-masked]>
Date: Wed, 21 Jul 2010 22:09:51 -0700

+1
-=R

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 07:54:52 +0200

Hi Adam,

On Wed, Jul 21, 2010 at 09:40:00PM -0700, Adam Barth wrote:
> these topics and has more time to reply to this list than I do.

Unfortunately, if neither of you two can take some time to give concrete
examples of what you're trying to protect against, you will constantly
see proposals that don't fit your requirements.

My understanding of cross-protocol attacks (please correct me if I'm
wrong) is the fact that a browser can be tricked into sending specially
crafted data to an IP:port which it believes is one protocol while it's
another one, the typical example being a form designed to send a POST
to an SMTP server, which will ignore all the HTTP part it does not
understand and will happily apply the body's data as SMTP commands. It's
easy to put web sites on the net that build such forms so that browsers
which get there are accidentely used to perform that attack.

If this is indeed the issue you're talking about, then I don't see why
the HTTP-based handshake could be a problem there. It's not more than
any other common HTTP request, it's even better in that no data should
flow until the server has correctly replied indicating proper support
for the protocol.

Basically we should get this :

  - the client sends an HTTP handshake (headers only)
  - the server responds with its HTTP handshake
  - the client checks the response and only then forwards
    data.

This standard scheme protects against cross-protocol attacks (as I
understand them) as the client does not send anything unless the
server proves its ability to understand the client protocol. Also,
the request and response are different HTTP messages, so a simple
echo server can't return the valid handshake by accident.

So with this, I'm sorry I don't see what class of cross-protocol
attacks remain and which ones we should actively protect against,
so if you have good examples, it would be nice if you could take
some time to share them.

Best regards,
Willy

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 22 Jul 2010 16:04:21 +1000

> able to answer all the followup questions.

Adam,

I don't think there is any specific lack of understanding of protocol
requirements in this WG. However there may be some disagreement with what
those requirements are.

It is great that you respect Ian's opinion, but I'm afraid he has failed to
convince many in this WG of the need for some of the more esoterica
"features" in the -76 draft.

Take the security issue for example.  There was a concern that websocket
could be used in combination with an injectable server to spoof it to
generate a legal websocket handshake response (what good that would do
anybody I don't really know, but let's assume that it is evil and dangerous
and that the injectable server cannot be exploited in other ways).    In
response to that concern, this list pretty much agreed that we should
include a random token in the handshake that was generated after the URL had
been passed to the javascript websocket API.  Since this token would be
unpredictable, then it would be impossible for anybody to craft a URL that
would cause that token to be injected in a response. Problem solved.

But when Ian codified this general agreement in the specification, he
adorned it with several extra "features".  He split the random number up and
added a pseudo secure algorithm involving count spaces, taking away the
number you first thought of and divide by your grandmothers birthday.  This
is totally unnecessary complexity and confusion because the random token
would be generated after any client had the chance to inject the URL, so no
further protection is needed from injection.  It is another silly example of
being fixated on the exact bytes of a message rather than the semantics that
the message is carrying.

Ian further "enhanced" the solution by sending the random token, not in
headers as had been discussed, but in extra bytes that comply neither with
HTTP nor with websocket framing.  The motivation for these bytes is
apparently to fast fail if there in an intermediary that does not support
websocket.   However, among many other problems with this mechanism, it has
now been shown that this mechanism slow fails with an intermediary that
would support websocket if not for the extra bytes.

Fast fail is indeed a very desirable requirement (although not always
achievable).   Protection from injection is also a desirable requirement.
But there was no need to mix the solution to these two requirements into a
single solution (with extra pseudo security).  If Ian had simply added the
random token as a header as had been discussed and widely accepted, then we
would no longer be talking about it.

He could also have raised the fast fail requirement (ideally by proposing it
as an addition to the requirements document), and then we could be using the
considerable experience available in this WG (and elsewhere) to come up with
the best solution to that problem.

We will only make progress is we methodically break down requirements and
consider multiple solutions.   Intransigent insistence of the first
one-size-fits-all "solution" does not help at all.

regards

---

From: Adam Barth <[email-masked]>
Date: Wed, 21 Jul 2010 23:11:58 -0700

Thanks.  Your email is much more lucid.

> see proposals that don't fit your requirements.

A cross-protocol vulnerability is a very general class of
vulnerability in which the client believes it's speaking one protocol
and the server believes it's speaking another protocol.  The ensuing
confusion leads to a miscommunication that an attacker can exploit to
do something undesirable.

> another one,

It's not a matter of being tricked.  It's a deliberate ability the
browser grants to guest code (the merits of which aren't really up for
discussion in this forum).

> the typical example being a form designed to send a POST
> understand and will happily apply the body's data as SMTP commands.

Yes.  HTTP => SMTP has a particularly bad cross-protocol
vulnerability.  The "solution" to the vulnerability is that browsers
forbid HTTP connections to port 25, which is deeply unsatisfying but
effective in practice.

> It's easy to put web sites on the net that build such forms so that browsers

Again, there's no accident involved.  It's entirely reasonable for
users to visit hostile web sites.  It's also entirely reasonable for
those hostile web sites to use these APIs.  It's the browsers job to
make sure the user isn't sad because of this arrangement.

> the HTTP-based handshake could be a problem there.

Well, it certainly *could* be a problem if improperly designed.  As a
community, we don't have a good science of cross-protocol
vulnerabilities.  We certainly can't understand all possible
interactions between all protocols.  We have to reason about these
attacks indirectly, for example by reduction.  It's possible to reason
about the security of an HTTP-based handshake by reduction to already
existing abilities the attacker has, provided the protocol is designed
with that sort of reasoning in mind.  Blithely hoping "HTTP likeness"
will avoid all issues isn't sufficient.

> It's not more than any other common HTTP request,

Keep in mind that attacker can only generate a very stylized subset of
HTTP requests in browsers today.  Stepping outside that subset
requires careful consideration.

> it's even better in that no data should

That's also a useful property.  However, we need to be careful when
thinking about what "proper" means in this context.  In many
protocols, the attacker can put words in the server's mouth, so to
speak.  For example, in DNS, the attacker can request TXT records from
domains he control, or in NNTP, the attacker can request posts he
authored.  That's why the server's proof of understanding the client's
hello message is convoluted in current protocol.

> echo server can't return the valid handshake by accident.

We're worried about more complex servers than just an echo server.
That's why the proof of understanding is more complex.  Certainly an
identical client -> server and server -> client message would be a
poor design.  However, just because they're not identical doesn't mean
we're out of the woods.

> so if you have good examples, it would be nice if you could take

Designing a secure protocol doesn't mean enumerating all attacks and
making sure we avoid them all.  We're not smart enough to envision all
attacks.  Instead, we need to convince ourselves that no attacks are
possible, which is much the opposite problem.  I don't have a pile of
attacks in my back pocket.  Instead, I can tell you that the protocol
presented earlier in this thread is unlikely to be free of
cross-protocol vulnerabilities because it lacks a number of defenses
and mitigations, some of which you've listed in your message.

Adam

---

From: Adam Barth <[email-masked]>
Date: Wed, 21 Jul 2010 23:21:46 -0700

> would cause that token to be injected in a response. Problem solved.

I don't think these issues are as simple as you seem to believe.
Security is rarely as simple as blithely tossing in a nonce and hoping
for the best.  Also, I'm skeptical about using the work "impossible"
in the context of what an attacker can and cannot do.  It smacks of
the kind of hubris that leads folks to a false sense of security.

> one-size-fits-all "solution" does not help at all.

You seem to essentially be saying that you don't understand the
rationale for all the cross-protocol mitigations in the current
handshake.  I'm not sure I understand them all either.

The essential problem is that we don't have a science of
cross-protocol attacks in the same way we have a science of
cryptographic primitives in network protocols.  It seems likely that
Ian has layered these mitigations into the handshake as defense in
depth.  Without a way to evaluate cross-protocol attacks, it seems
prudent to use a bunch of mitigations and hope that at least one of
them saves us in any given situation.

As an aside, that's why I prefer the TLS+NPN approach.  It's a simple
mechanism that I can convince myself resists cross-protocol attacks.
Moreover, I understand exactly what role each piece of the protocol is
playing, which means we don't need to layer together a string of
half-defenses and hope the holes in the cheese don't align.

Adam

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 22 Jul 2010 16:48:36 +1000

Adam,

I am not claiming that a random nonce is a solution to all the security
problems.
But I believe it is a very good solution to the specific concern expressed
that an injectable HTTP server might be able to be tricked into sending and
acceptable websocket hand shake response.

If the token is unpredictable and generated after the URL is passed, then I
do believe that we can say that it is impossible for a URL to be crafted
that will inject that token into a response. Of course "unpredictable" is a
relative measure, but I think good server developers are already on top of
that one when it comes to the generation of session IDs and similar
(although it is surprising how difficult that is).

Conversely, I think the count spaces and divide results is exactly the type
of algorithm that can have unexpected numeric effects that might result in
the number of truly random bits being significantly reduced and this more
predictable, specially if the same random generator(algorithm) is used to
produce the numbers and to place the spaces.  It would not surprise me if
such an algorithm significantly reduced the search space.

> handshake.  I'm not sure I understand them all either.

Using a pseudo secure algorithm that has not been subjected to any rigorous
analysis is not going to help either.

> them saves us in any given situation.

I'm all for defence in depth.
Note that the random token is already the second line of defence as you
firstly need a scriptable browser, that can talk to an injectable server.

Can you tell me that the count spaces algorithm really adds another layer of
defence? or does it just make the random nonce more guessable?

> half-defenses and hope the holes in the cheese don't align.

Similarly a random nonce is a simple mechanism, which good developers will
understand and know that they need to provide sufficient random bits that
are unpredictable and protected from discovery by the javascript client.

regards

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 08:49:45 +0200

> Thanks.  Your email is much more lucid.

Thanks ;-)

> do something undesirable.

OK so that was my understanding. I've been using that for some time to
telnet home through proxies from closed enterprise networks ;-)

(...)
> make sure the user isn't sad because of this arrangement.

Yes indeed. I mean "accidentely" from the user's point of view, in
that it is not the behaviour he would have expected.

> Well, it certainly *could* be a problem if improperly designed.

Indeed, and as such the -76 one is, because the first bytes can be used
to send a second HTTP request to a server through a reverse-proxy while
at this point this should only be WebSocket and not HTTP.

> As a

That's why I'm trying to reduce the amount of blind emissions during the
handshake.

> requires careful consideration.

My concern is also that we're becoming too much different from this
stylized subset, by introducing special features that nobody cares
to qualify possible negative impacts nor means of exploitation. I'd
rather stick to something well mastered which shares the same class
of possible vulnerabilities and attacks as the other ones targetting
the same agents, so that when we spot one and fix it, the issue is
fixed for all uses.

> hello message is convoluted in current protocol.

Yes, indeed ! Remember spammers using FTP with a RETR command to send
a previously uploaded mail from port 20 to another host port 25 ?

> poor design.  However, just because they're not identical doesn't mean

I agree, but at least we can enumerate the strict requirements for
the client to accept to send data to the server, which reduce the
exposure. Right now with a properly designed HTTP upgrade handshake,
you require a very precise set of criteria in the response that only
a server understanding the protocol could send because a non-WS
server would not have invented them. And one advantage of HTTP upgrade
is that the response MUST start with "HTTP/1.1 101". Anything else
won't match, which considerably limits the ability to inject the
response into the server's mouth. For instance, my SMTP server sends
me a "220 mail.home.local" first, which does not match HTTP/1.1. Some
other protocols will for instance send "Error: GET /xxx unknown command".
In fact, the parts we expect from the response cannot be guessed from
the command by non-WS aware servers. This is what makes it quite robust
already.

> attacks.

Agreed. Otherwise it's a lost game. But they can serve as examples to
try to defeat a design however.

> Instead, we need to convince ourselves that no attacks are

Yes, but as Scott said it, it was a proposal to work on something with
a constructive approach. Not everyone is perverted enough to think how
a protocol can be abused, and that's why we need a variety of people to
work on the same protocol.

Regards,
Willy

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jul 2010 00:08:02 -0700

> at this point this should only be WebSocket and not HTTP.

Interactions with both proxies and reverse proxies are important to
consider for HTTP-based handshakes because we're likely to get deeper
into their state machines.  I'm not sure I understand precisely the
issue you're referring to, but that's why they pay Ian the big bucks.

> That's why I'm trying to reduce the amount of blind emissions during the

Attacker-controlled blind emission is definitely a bad idea.  Fixed
once-and-for-all strings are probably not that bad.  Bytes chosen
uniformly at random for each request are best.

> the same agents, so that when we spot one and fix it, the issue is

I didn't understand the concrete implications of this paragraph.

> the command by non-WS aware servers. This is what makes it quite robust

You're describing a defense which we've been calling rigidity.  By
making the handshake rigid, it's harder to squeeze though some other
protocol's state machine.  However, you can imagine protocols that are
vulnerable to a perfectly rigid handshake.  As an example, imagine a
protocol like gopher where the attacker might be able to control all
the bytes returned by the server (e.g., because the server accepts
uploads).

There's value to having other defenses against cross-protocol attacks
in addition to rigidity.  Just as Greg's nonce isn't magic security
pixie dust, neither is rigidity.

> work on the same protocol.

I'm take some offense at being called perverted.

Adam

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 22 Jul 2010 00:18:19 -0700

> which get there are accidentely used to perform that attack.

There's at least three kinds of concerns about cross-protocol attacks and the WebSocket protocol, in the case where the client is the browser:

1) Hostile JS code running in the browser may use the browser's WebSocket client code to try to attack existing HTTP resources, if it can make a request that looks sufficiently like HTTP.
2) Hostile JS code running in the browser may use the browser's WebSocket client code to try to attack existing non-HTTP servers, if it can make a request that looks sufficiently like the service in question.
3) Hostile JS code running in the browser may use the browser's HTTP client code (e.g. via XMLHttpRequest) to try to attack newly created WebSocket servers, if it can make a request that looks sufficiently like WebSocket.

There is also another orthogonal dimension to the threat space:

A) Attacks on confidentiality, where the attacker is trying to get access to information from the resource being attacked which he or she should not be able to obtain.
B) Attacks on integrity, where the attacker is attempting to inject commands into a resource, without necessarily caring about getting a response back.

Cross-protocol attacks were much discussed earlier in the year, and I gave some sketches of attack scenarios here:
http://www.ietf.org/mail-archive/web/hybi/current/msg01198.html

> flow until the server has correctly replied indicating proper support

What's "the HTTP-based handshake"? We have had several proposals for handshakes that start out looking like HTTP. They are not all equally effective in protecting against cross-protocol attacks, though some of them do have some protection..

Although I am not an expert on cross-protocol attacks, one thing I have learned is that the issues can be *extremely* subtle. This leads me to conclude that it's better to err on the side of caution, and design the protocol to be even more robust against such threats than we think is strictly necessary. 

I believe the TLS-based handshake (as proposed by Adam) is more robust than solutions involving nonces and challenge/response built into an HTTP-like handshake. The reason for this is that it's hard for the attacker to control the bytes sent, other than to cause the initiation of a TLS handshake, which is something that all network services are already exposed to by browsers. The attacker cannot get to the next-level protocol running over TLS unless the server explicitly opts in. With HTTP-based schemes, it takes much more care to avoid sending information as the "wrong" protocol.

However, HTTP-based challenge-response solutions are definitely better than not trying to address the issue at all.

Regards,
Maciej

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 22 Jul 2010 11:21:05 +0300

Hi Adam,

first let me thank you, Willy, Maciej  for the discussion about 
cross-protocol vulnerability,
having the explanation and picture of the problem always help the 
discussion and boost the
possibility to find a good solution.

from my personal point of view,
I see more value to work on a general solution to secure the HTTP 
Upgrade mechanism
against cross-protocol vulnerability, instead of trying to draft 
something ad-hoc for WebSocket

actually that sounds at least to me as something that perfectly fit in 
the HasMat BoF.

    "The goal of this working group is to standardize a small number of
    selected specifications that have proven to improve security of Internet
    Web applications. The requirements guiding the work will be taken from
    the Web application and Web security communities.

It would be extremely good if you (or someone else) can arrange a short 
presentation about this issue during the HasMat BoF!
If nobody volunteer I can prepare a couple of slides (taking advantage 
of what you have discussed in this thread)
and ask 5 minutes to the HasMat chairs to present them.

I also think the wg should also investigate the TLS+NPN approach, that 
is quite promising but we have also consider
that it also a work in progress.

cheers
Sal

On 7/22/10 10:08 AM, Adam Barth wrote:
> On Wed, Jul 21, 2010 at 11:49 PM, Willy Tarreau<[email-masked]>  wrote:
>    
>>> On Wed, Jul 21, 2010 at 10:54 PM, Willy Tarreau<[email-masked]>  wrote:
>>>        

-- 
Salvatore Loreto
www.sloreto.com

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 11:23:22 +0200

> I didn't understand the concrete implications of this paragraph.

I was talking about the blind 8 bytes that must be forwarded. This
proposal was added to the draft without apparently anyone noticing
the impact on the HTTP protocol itself. Each time we try to divert
from working standards, we're risking to fall into traps that were
avoided by those standards.

(...)
> the bytes returned by the server (e.g., because the server accepts

Is it possible to make a gopher server respond with a WS-compatible
handshake to a valid WS request ? I don't know what the gopher protocol
looks like, but that sounds scary since in my opinion, the WS handshake
request should be small and limited enough to limit the risk of such
protocol manipulations, if not prevent them.

> pixie dust, neither is rigidity.

Yes, I agree. They can at least reduce the probability of success,
rendering them useless in practice.

> I'm take some offense at being called perverted.

Don't take offense for that, I see it more as a quality that only a few
people have to imagine how apparently reliable systems can be abused :-)

Regards,
Willy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jul 2010 02:33:57 -0700

On Jul 22, 2010, at 1:21 AM, Salvatore Loreto wrote:

> I see more value to work on a general solution to secure the HTTP Upgrade mechanism 
> against cross-protocol vulnerability, instead of trying to draft something ad-hoc for WebSocket

The HTTP Upgrade mechanism is not vulnerable to cross-protocol attacks.
Other (non-HTTP) services are vulnerable to browser misdirection if
the browser can be directed to send arbitrary bytes to an arbitrary
TCP port.  That is a problem which HTTP is actually good at avoiding
because the first line is unique to HTTP.

The issues people are raising about Upgrade are in regard to passing
through intermediaries.  Some people believe that intermediaries are
evil and must be tunneled through.  Other people believe that
intermediaries are beneficial and any protocol that deliberately
subverts them will be blocked.

There is no solution that will satisfy both of those beliefs.
Please stop trying to reconcile irreconcilable belief systems.

Implement both.  One will succeed, the other will fail.
Neither choice is mature enough for standardization at the
current time.

....Roy

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 22 Jul 2010 13:24:54 +0300

> because the first line is unique to HTTP.
Hi Roy,

thanks for the answer and to share your view.
as far I understood (but I have to admit not to be an expert on 
cross-protocol attacks),
the kind of attacks people are talking is restrict to HTTP - WebSocket 
(or viceversa) cross attack.

however if HTTP experts exclude any vulnerability in the HTTP Upgrade,
then a sort of security check right after the end of the Upgrade from 
HTTP to Websocket
and before the WebSocket starts to exchange data, could solve the problem.

> subverts them will be blocked.
passing or not through intermediaries is a completely different protocol 
aspect the wg has make a decision on;
but I would prefer the decision be made clearly, not hidden behind 
security issues
especially if they do not exist !

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 14:13:17 +0200

On Thu, Jul 22, 2010 at 01:24:54PM +0300, Salvatore Loreto wrote:
(...)
> HTTP to Websocket and before the WebSocket starts to exchange data,
> could solve the problem.

We must at least keep in mind that we should avoid round-trips as much as
possible. For that, I think that Greg's proposal of a nonce in a header is
particularly interesting because it does not add round trips and can be
made part of the Upgrade handshake itself. The request and response headers
could then be advertised in the Connection header just like the Upgrade
header right now. Such a header could possibly be suggested as general use
for the HTTP Upgrade mechanism instead of being WS-centric.

The only fear I have right now with the nonce as defined by the WS draft
is that it makes use of MD5 which comes with a cost. I know at least one
site dealing with more than 300k concurrent connections at a rate between
12 and 18k per second (long polling right now), and I think that computing
MD5 hashes can significantly impact performance at these rates. If the
goal is to ensure the response header depends on the request header, we
may reliably make use of cheaper algorithms.

Regards,
Willy

---

From: Dave Cridland <[email-masked]>
Date: Thu, 22 Jul 2010 13:27:25 +0100

> may reliably make use of cheaper algorithms.

I'd assume that an actual server can do faster than my cheap  
workstation:

$ openssl speed md5
Doing md5 for 3s on 16 size blocks: 7765276 md5's in 3.00s
Doing md5 for 3s on 64 size blocks: 6086328 md5's in 3.00s
Doing md5 for 3s on 256 size blocks: 3674140 md5's in 3.00s
Doing md5 for 3s on 1024 size blocks: 1420165 md5's in 3.00s
Doing md5 for 3s on 8192 size blocks: 212112 md5's in 3.00s

FWIW, SHA-1 is only slightly slower. If you had 300k all join at  
once, then it still seems to be this would be a drop in the ocean in  
terms of loading.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 14:33:17 +0200

Hi Dave,

> once, then it still seems to be this would be a drop in the ocean in  

Nice, thanks for feeding some useful info here ! So we can assume
that a correctly sized server will be able to compute at least 2
million hashes a second, which translates into less than 1% CPU
overhead at 20000 connections per second.

You've convinced me that it's not a big deal finally.

Regards,
Willy

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 22 Jul 2010 22:48:06 +1000

Adam,

well the nonce idea was not mine (and I once was very unconvinced of it's
need), but as it's current advocate I OK if you want to label it that way.

However, I do very much like your analysis of rigidity as a protection
against security attacks.  In essence, the rigid approach is trying to focus
on the syntax off the message on the basis that it "would be impossible" for
an attacker to generate those bytes.  The nonce/token alternative is based
on the assumption that it would be impossible for an attacker to know what
bytes they need to generate (regardless of syntax).

Of the two, I believe that rigidity is the weaker approach as it is
extremely difficult to "prove" that nothing on the internet can be tricked,
subverted or exploited to produce any arbitrary bytes that an attacker
wishes to generates.

The nonce/token approach is intended to deny attackers working within the
browser having the ability to know what bytes they should generate, so that
even if they could inject them, they don't know what to object.  Thus I
think it is a stronger approach, but for more limited circumstances - as it
does not address non browser clients.

In the spirit of defence in depth, I think we probably need elements of both
approaches.   However, I would suggest that currently the specification has
placed too much emphasis on rigidity, to the extent that it will effect
interoperability and extensibility without any significant gain in security.

regards

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 16:07:26 +0200

Hi Maciej,

> 1) Hostile JS code running in the browser may use the browser's WebSocket client code to try to attack existing HTTP resources, if it can make a request that looks sufficiently like HTTP.

Maybe I'm mistaken, but why couldn't such JS code simply make use of the
browser's HTTP client to do so ? Posting a form, reloading an image from
a given URL or making an XMLHttpRequest seems perfectly possible right now,
so trying to do that via the WebSocket code would bring nothing at all.

> 3) Hostile JS code running in the browser may use the browser's HTTP client code (e.g. via XMLHttpRequest) to try to attack newly created WebSocket servers, if it can make a request that looks suff…

The same is true here : why would the JS code try to use the normal HTTP
client if the browser supports WebSocket, with which the code would be
able to have finer control over its attack ?

> What's "the HTTP-based handshake"?

I mean the HTTP Upgrade mechanism.

> We have had several proposals for handshakes that start out looking like HTTP. They are not all equally effective in protecting against cross-protocol attacks, though some of them do have some protection..

Possibly, but what ended up in the spec was finally less secure than the normal
HTTP handshake since it allows the client to send bytes to the server through
reverse proxies without that later being aware of it.

> Although I am not an expert on cross-protocol attacks, one thing I have learned is that the issues can be *extremely* subtle. This leads me to conclude that it's better to err on the side of cautio…

I agree with the *extremely subtle* point. But that's also why I'd rather
not try to reinvent within a small group things that are already efficient
and not proven wrong. We must accept that a small group is never smarter
than those outside the group and that a few hours designs have little
chance of being more robust than those which have survived exposure to the
world for 10 years or so.

It's just like when you see developers implement their own crypto algorithm
because "the public ones have not been broken yet but they're attacked, so
let's be safe and write a new one". The ones I have talked about on this
subject were really convinced their work was better than everything else,
while it was a disaster. That's what I'd prefer we avoid here.

However, enforcing a strict mode of operations based on something which
already works, or as Sal suggested, enforcing a stricter standard is the
right way to go.

> I believe the TLS-based handshake (as proposed by Adam) is more robust than solutions involving nonces and challenge/response built into an HTTP-like handshake. The reason for this is that it's har…

I agree with those points, but as suggested, there are many cases where
being limited to TLS can be a showstopper, either due to overhead, or
more simply because it will leave many users away (mostly those inside
enterprises who can only access a limited set of SSL sites).

We should spend some time trying to enumerate what aspect(s) of the HTTP-based
handshake we believe may be a problem regardless of the protocol deployed on
the target server.

> However, HTTP-based challenge-response solutions are definitely better than not trying to address the issue at all.

100% agree !

Regards,
Willy

---

From: John Tamplin <[email-masked]>
Date: Thu, 22 Jul 2010 10:12:01 -0400

On Thu, Jul 22, 2010 at 10:07 AM, Willy Tarreau <[email-masked]> wrote:

> > 1) Hostile JS code running in the browser may use the browser's WebSocket
> client code to try to attack existing HTTP resources, if it can make a
> request that looks sufficiently like HTTP.
>

JS is limited in how it can influence all the HTTP headers in the request,
and is also bound by same-origin policy.

> > 3) Hostile JS code running in the browser may use the browser's HTTP
> client code (e.g. via XMLHttpRequest) to try to attack newly created
> WebSocket servers, if it can make a request that looks sufficiently like
> WebSocket.
>

The JS cannot control anything but the data contents of the message, so is
limited in what it can do to the headers of the initial request or the
framing.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 22 Jul 2010 16:19:14 +0200

Hi John,

> and is also bound by same-origin policy.

OK thanks for this clarification.

> limited in what it can do to the headers of the initial request or the

Fine, so by making use of a same low-level protocol (HTTP), we're certain
that what is forbidden by one is also forbidden by the other one. In short,
if JS cannot manipulate HTTP headers, this will be true for both WS and HTTP
as long as WS handshake relies on HTTP. However, having a WS handshake that
"looks like" HTTP opens the way to corner cases and side effects (exactly
what happened with the extra bytes which can be parsed as HTTP requests
when directed to non-WS aware servers).

Regards,
Willy

---

From: Dave Cridland <[email-masked]>
Date: Thu, 22 Jul 2010 15:22:28 +0100

On Thu Jul 22 13:33:17 2010, Willy Tarreau wrote:
> > >The only fear I have right now with the nonce as defined by the  
> WS
> > >draft
> > >MD5 hashes can significantly impact performance at these rates.  
> If
> > >the
Well, perhaps a bit worse than that - say 1 million per second, and  
therefore 2%.

Although MD5 has some security weaknesses, its major problem is  
usually that it's too fast - consider how fast it is to exhaustively  
search the unsalted intermediate hash of Digest authentication, for  
instance.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 22 Jul 2010 15:25:30 +0100

> Well, perhaps a bit worse than that - say 1 million per second, and  

Harrumph. I misread my own figures.

Given 7,765,276 over 3.0 seconds on one core, my machine can do  
10,353,701 hashes per second using all four cores, and therefore  
20,000 connections per second equates to an overhead (due to MD5) of  
roughly 0.2%.

But what's a factor of ten between friends?

Dave.

---

From: Mike Belshe <[email-masked]>
Date: Thu, 22 Jul 2010 08:07:19 -0700

BTW - Isn't the creation of this problem really because we allow arbitrary
UTF-8 written by the JS application?  If we had a true framing layer (where
frames have headers, instead of allowing UTF-8), would these attacks be
real?  It seems to me that the simplest frame header would be sufficient to
prevent all of these attack vectors, but the frame headers could contain
nonces or whatever other security information is needed too.

The reason I mention this is because as we look to using a TLS/NPN, we can
potentially get rid of the first round trip.  There was a lot of support for
that idea within the group.  But, we can *only* do so if we don't have JS
able to write directly to the socket.  If JS is able to write UTF-8 to the
socket, then skipping the first round trip opens a security vulnerability as
depicted by Maciej & Adam here.

Mike

On Thu, Jul 22, 2010 at 12:18 AM, Maciej Stachowiak <[email-masked]> wrote:

> There's at least three kinds of concerns about cross-protocol attacks and
> the WebSocket protocol, in the case where the client is the browser:
>
> 2) Hostile JS code running in the browser may use the browser's WebSocket
> client code to try to attack existing non-HTTP servers, if it can make a
> request that looks sufficiently like the service in question.
> 3) Hostile JS code running in the browser may use the browser's HTTP client
> code (e.g. via XMLHttpRequest) to try to attack newly created WebSocket
> servers, if it can make a request that looks sufficiently like WebSocket.
>
> A) Attacks on confidentiality, where the attacker is trying to get access
> to information from the resource being attacked which he or she should not
> be able to obtain.
> B) Attacks on integrity, where the attacker is attempting to inject
> commands into a resource, without necessarily caring about getting a
> response back.
>
>
> What's "the HTTP-based handshake"? We have had several proposals for
> handshakes that start out looking like HTTP. They are not all equally
> effective in protecting against cross-protocol attacks, though some of them
> do have some protection..
>
> Although I am not an expert on cross-protocol attacks, one thing I have
> learned is that the issues can be *extremely* subtle. This leads me to
> conclude that it's better to err on the side of caution, and design the
> protocol to be even more robust against such threats than we think is
> strictly necessary.
>
> I believe the TLS-based handshake (as proposed by Adam) is more robust than
> solutions involving nonces and challenge/response built into an HTTP-like
> handshake. The reason for this is that it's hard for the attacker to control
> the bytes sent, other than to cause the initiation of a TLS handshake, which
> is something that all network services are already exposed to by browsers.
> The attacker cannot get to the next-level protocol running over TLS unless
> the server explicitly opts in. With HTTP-based schemes, it takes much more
> care to avoid sending information as the "wrong" protocol.
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 22 Jul 2010 11:18:35 -0400

> depicted by Maciej & Adam here.

The JS API just has strings.  It is up to the browser to convert these
strings into UTF8 sequences for the protocol.  If current experience is any
guide, there will be differences among browsers, particularly regarding
non-BMP characters and combining marks.

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 22 Jul 2010 09:36:34 -0700

> Can you explain how this proposal provides an HTTP-compatible upgrade in the non-TLS case? Or how it provides protection against cross-protocol attacks? I think those are two pretty important requi…
Actually, let me explain how the non-HTTP version was designed to handle 
cross-protocol attacks, because I didn't explain the reasoning behind 
the design in the draft.

I) Initial HELLO line

The initial line for both server and client responses is:

  %x80.x02.NN.NN WebSocket/1.0 LF

Deliberately designed properties of that line:

  1. Since it's a fixed-length line, there's no opportunity for URL or 
METHOD-style attacks.

  2. Since it's a line, terminated by LF, it validates against 
text-based protocols, even though this is a binary protocol. (Potential 
NN.NN attacks can be enumerated and checked.)

  3. Since the only variation is the two bytes NN.NN, 65536 values, all 
possible initial HELLO lines can be enumerated and tested against other 
protocols.

  4. Since it's not a legal HTTP line, HTTP servers and clients will 
reject it.

  5. Browser HTTP clients cannot generate that line.

  6. Since the initial byte is neither valid ASCII nor is it valid 
UTF-8, no validating ASCII/UTF-8 server will accept that line, and no 
ASCII/UTF-8 client can generate that line.

  7. The fixed "WebSocket/1.0" validates the protocol itself. Boring, 
but necessary.

I'll maintain the following properties are true of that initial line:

  1. No HTTP client can spoof that line (unless it has full binary 
control over the HTTP method, but that security hole could spoof any 
protocol).

  2. That line will be rejected by any HTTP server (all 64k variations 
of it).

  3. Clients of other protocols cannot spoof that line (again, unless 
they have full binary control over their initial bytes, and obviously 
TCP can "spoof" the line).

  4. binary and text servers which validate their initial line cannot be 
spoofed by it.

That covers validating binary and text protocols, but there's still the 
case of protocols like SMTP that discard invalid lines.

II) Handling recovery-based text protocols like SMTP

The next required lines from the request looks like:

  url: <restricted utf-8> LF

where the restrictions include forbidding LF itself.

This is a HTTP header without the line-folding.  If a SMTP 
implementation protects against HTTP headers (I checked against 
postfix), it protects against this WebSocket header. If the SMTP 
implementation accepts all invalid lines, there's really nothing you can 
do, except for the WebSocket client to wait for the server response.

III) header/url/origin issues

The "url:", "origin:", "protocol:" headers and their semantics are 
copied from the official WebSocket draft. If there's a problem with 
them, there's a problem with the official draft. (If there's an error in 
transcription, that's easily fixed.)

IV) nonce/digest

In practice, these cover validating protocols. However, it's also 
possible to force the server to prove that it's an active WebSocket 
server.  Adding a "nonce:" header to the client, and requiring the 
server to calculate a "digest:" header is proof that the server is a 
valid WebSocket server. (I personally don't think this is necessary, but 
the group may disagree.)

V) pipelining/multiple packet handshake

Pipelining data means the client is sending data before looking at the 
server's response. If this is unacceptable, the client must wait for the 
response.  There's no protocol design around this issue. You need to 
choose one behavior or the other. My draft choose pipelining. The group 
could choose the reverse.

VI) Does this cover everything?

Since the HELLO line is a fixed-length sequence (with finite 64k 
variations), it's possible to prove if it can spoof any specific 
protocol or if a specific protocol can be spoofed by it.

For example, it's possible to prove that this initial line is invalid 
HTTP and that no valid HTTP can match this line. In this case the proof 
is trivial because the initial byte is an invalid first byte of HTTP.

It's also possible to exhaustively test any specific 
non-WebSocket-protocol implementation against all possible HELLO 
attacks, because all possible HELLO lines are enumerable. You don't have 
to guess if an IRC server will reject a WebSocket HELLO; you can test it.

In a similar way, it's possible to reason about non-validating or 
partially validating servers. For example, if the HTTP server does not 
validate ASCII/UTF-8, but merely scans for SP and CR/LF, the WebSocket 
request line interpreted as HTTP is still an invalid request. And even 
if one of the NN is a SP and you have a truly brain-dead HTTP server, 
the HTTP method for the hypothetical broken server isn't a valid method. 
And if the server doesn't even validate that case, it's just plain broken.

So, even though the handshake wasn't my main focus of that draft, I do 
believe it covers cross-protocol scripting issues better than HTTP 
itself does. Obviously, my HTTP handshake version has identical security 
behavior with the official draft.

-- Scott

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 22 Jul 2010 09:40:04 -0700

On Jul 22, 2010, at 1:21 AM, Salvatore Loreto wrote:

> Hi Adam,
> 
> first let me thank you, Willy, Maciej  for the discussion about cross-protocol vulnerability,
> having the explanation and picture of the problem always help the discussion and boost the
> actually that sounds at least to me as something that perfectly fit in the HasMat BoF.
> 
> It would be extremely good if you (or someone else) can arrange a short presentation about this issue during the HasMat BoF!
> If nobody volunteer I can prepare a couple of slides (taking advantage of what you have discussed in this thread)

I'm not sure that is a good approach to the problem, for two reasons:

1) Cross-protocol vulnerabilities are highly dependent on the details, and it matters very much what the upgrade request looks like, and what protocol is used after upgrading. Upgrading to WebSocket has very different considerations from upgrading to TLS (RFC2817). It's not clear to me that this problem can be solved in a generic way.

2) Even if the problem could be solved generically for any instance of HTTP Upgrade, it would create significant delay if we add a dependency to WebSocket Protocol on a dependency that is currently undefined.

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 22 Jul 2010 09:46:34 -0700

> may reliably make use of cheaper algorithms.

I originally suggested computing a hash of the nonce with other data (or some other nontrivial computation) rather than just echoing it back. The goal here is to further reduce the risk of cross-protocol attacks by ensuring that a valid response cannot be constructed solely out of fixed pieces and echoing back portions of the request. I don't think it's essential for the algorithm to be MD5; it does not need to be a strong hash. However, it should avoid having trivial mappings for any values. For example, simply XORing two values is a bad idea because if the attacker can arrange for one value to be 0, the correct response is just the other value with no change. I believe MD5 was chosen because it does not have any such trivial flaws, and is available in popular library implementations. CRC would be adequate for this purpose but is not as readily available.

I think it would be worth doing some math to determine the expense of computing MD5 sums at a plausible connection rate. If MD5 can't keep up with your cited rate of 12k-18k connections per second on the relevant target hardware, then we need to re-evaluate.

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 22 Jul 2010 10:05:44 -0700

> so trying to do that via the WebSocket code would bring nothing at all.

All of the mechanisms you listed have significant restrictions:

1) Reloading an image from a given URL, the in-browser attacker cannot use any method other than GET or set any custom headers beyond what the UA sends normally. And the attacker definitely cannot read back the reply.
2) Posting a form, the in-browser attacker cannot use any method other than GET or POST, set any custom headers beyond what the UA sends normally, or send content in anything but a short list of existing formats. And the attacker definitely cannot read back the reply.
3) Using cross-site XMLHttpRequest, the attacker cannot use any method other than GET or POST, set any custom headers, or include a message body in anything but a small set of formats, unless the target site explicitly opts in via an OPTIONS request (cross-site XHR is specifically designed to only be able to do things you can do with form submission).

Even given all that, many sites do in fact have vulnerabilities to cross-site form posting. This class of attacks is called Cross-Site Request Forgery, or CSRF. Typically, these are attacks on integrity, not confidentiality, since the attacker cannot read the reply. WebSocket creates the possibility of extending the attack surface for these kinds of attacks, and for the possibility of attacks on confidentiality as well as integrity.

> able to have finer control over its attack ?

Because WebSocket would require a successful handshake before sending any actual messages, while an HTTP POST with XHR would blindly send the post body without validating the handshake. If the server doesn't thoroughly verify the client part of the handshake, then it may be tricked into processing  a POST body that looks like a series of WebSocket messages. If it authenticates with a cookie, then this could be used to inject control messages with a side effect, depending on the nature of the WebSocket service.

> I mean the HTTP Upgrade mechanism.

Do you mean the specific mechanism currently in <http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76> or something else?

> already works, or as Sal suggested, enforcing a stricter standard is the

I don't think anyone has previously invented a solution to this problem, so we don't have the option of using a well-vetted 10-year-old design. I also do not think passing off responsibility to another group increases our odds of creating a secure protocol.

> handshake we believe may be a problem regardless of the protocol deployed on

I think it would be worth enumerating the pros and cons of the TLS-based approach and recording them somewhere (such as on a wiki page).

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 22 Jul 2010 10:10:38 -0700

On Jul 22, 2010, at 8:07 AM, Mike Belshe wrote:

> BTW - Isn't the creation of this problem really because we allow arbitrary UTF-8 written by the JS application?  If we had a true framing layer (where frames have headers, instead of allowing UTF-8), would these attacks be real?  It seems to me that the simplest frame header would be sufficient to prevent all of these attack vectors, but the frame headers could contain nonces or whatever other security information is needed too.

I think it would be unwise to depend on a framing layer as a primary defense. First, who can say whether HTTP services would be vulnerable to data that happens to also be properly framed WebSocket messages? I wouldn't bet on it myself. Second, this does nothing to protect against using HTTP to attack WebSocket services.

> The reason I mention this is because as we look to using a TLS/NPN, we can potentially get rid of the first round trip.  There was a lot of suppor for that idea within the group.  But, we can *only* do so if we don't have JS able to write directly to the socket.  If JS is able to write UTF-8 to the socket, then skipping the first round trip opens a security vulnerability as depicted by Maciej & Adam here.

With TLS/NPN, I expect the handshake would fail at the TLS level in case of a cross-protocol attack (either WebSocket vs HTTP or HTTP vs WebSocket), and therefore no attacker-controlled content would ever be sent.

Regards,
Maciej

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jul 2010 10:16:23 -0700

On Thu, Jul 22, 2010 at 8:07 AM, Mike Belshe <[email-masked]> wrote:
> real?

Sadly, framing alone isn't sufficient.  Consider a protocol like
DNS-over-TCP that uses the following framing mechanism:

1) Read the first byte and interpret that as the size of the first frame.
2) Attempt to consume the frame.
3) If you fail to consume the frame, advance to the next frame and
return to step (1).

If you send an HTTP POST to a DNS server that's listening to TCP
(which many do), the server will interpret the "P" in POST as the
frame length.  The server will ignore the first frame (because it's
not properly formated) and then 80 bytes into the POST body and try
again.

Now, it just so happens that you can't use browser-generated HTTP POST
requests to screw with a DNS server because every interesting DNS
request must have a NULL byte in it somewhere (as far as I can tell),
and there's no way to generate a POST body from a browser that
contains a NULL byte (at least in Internet Explorer).  In other
browsers (e.g., Firefox), you can get this exact attack to work,
except that Firefox blocks port 53, so they escape disaster by another
means.

The details aren't that important.  What's important is that framing
along is insufficient to protect again cross-protocol attacks because
the other protocol might be using a different framing mechanism.  By
combining the two, you might be able to get the victim protocol to
ignore whatever traps you put in the framing.

> It seems to me that the simplest frame header would be sufficient to

You should be suspicious whenever someone says XYZ can prevent all
attack vectors.  Of the people posting to this list, only Maciej and
Ian seem to have sufficient respect for the subtleties of these
security issues.

> depicted by Maciej & Adam here.

The essential security property that we need from TLS+NPN is that the
server agrees that it wants to talk web sockets (by agreeing that it's
the next protocol) before we dump anything dangerous on the socket.
You need at least one round trip to the server to get it's consent to
receive attacker-controlled bytes.  I don't see a way around that with
our current tools.

On Thu, Jul 22, 2010 at 9:36 AM, Scott Ferguson <[email-masked]> wrote:
> cross-protocol attacks, because I didn't explain the reasoning behind the
> design in the draft.

Your analysis is wildly optimistic and doesn't consider protocols like
with framing mechanisms like DNS-over-TCP described above.  You're
just embarrassing yourself.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jul 2010 10:27:53 -0700

> because the first line is unique to HTTP.

HTTP is the source of a large number of cross-protocol attacks.  The
uniqueness of the first line often does not help because other
protocols (such as SMTP, DNS, FTP, etc) happily ignore leading junk on
the socket that they don't understand.  Claiming HTTP is not
vulnerably to cross-protocol attacks is just silly.

If you'd like some background reading about cross-protocol attacks
involving HTTP, I'd invite you to read the classic paper on the
subject:

http://www.remote.org/jochen/sec/hfpa/hfpa.pdf

There's a later paper with more interesting examples called "Extended
HTML Form Attack Revisited", but I couldn't turn up the PDF with a
quick search.  Collin Jackson and I also have a number of unpublished
results in this area, all of which use HTTP to attack other protocols.

Adam

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 22 Jul 2010 13:04:56 -0700

> The two operate at completely different layers of abstraction.
I meant cross-protocol, of course.

The issue is the number of request/response before the initial payload 
is sent. If we require the client to wait to validate the server before 
it can send its payload, the initial performance will be slower than a 
HTTP POST.

That delay may be perfectly acceptable, but some people have raised it 
as an issue in the group. I'm fine with the delay myself, but I think it 
should be resolved.
> What does it mean for security to be no worse than HTTP?  
Specifically, the fundamental issue with sending data before the server 
is validated by the client, like HTTP POST.

Is that capability acceptable for WebSockets or not? (I mean in any 
form, my specific proposal is irrelevant.)
> implementations to use a TLS library.
Well, yes, that certainly addresses the security issue. The open 
question is whether that's acceptable performance-wise. If it is 
acceptable, fine, we're done. (There is the secondary issue of NPN being 
a draft which is an issue with libraries being available in the near term.)
> basic security requirements is a bit brash.
True, and I apologize for that. I was thinking of the binary framing 
requirements that are ignored by the current spec (and are claimed to be 
too complicated), not the handshake requirements that are already 
addressed. I only added the handshake to the draft for completeness 
sake, not because it adds anything new.

-- Scott

---

From: Mike Belshe <[email-masked]>
Date: Thu, 22 Jul 2010 13:25:01 -0700

> ignore whatever traps you put in the framing.

Let's stay focused, we're not talking about securing the DNS protocol or any
protocol other than WebSockets.

You can't prevent someone from sending junk to random network ports.  Each
server needs to harden against that in its own way.  It's tough stuff - and
preventing DoS is even harder.  But WebSockets isn't at fault if a
WebSockets request goes into an existing, un-hardended server and causes
chaos.

The HTTP & Proxy case is interesting, because WebSockets does initiate
through these existing protocols, so I do agree with ensuring that these
cases don't expose new holes.  And in these cases, a frame header would
suffice.

> attack vectors.

I didn't say that - I said it specific could address all 3 of the vectors
Maciej described.

> Of the people posting to this list, only Maciej and

I'm just saying that its a bummer to put the security fixes into a
synchronous handshake (which requires a RT).  If we could put them into
frame headers which accompany the data, no RT would be necessary.  I believe
this could work.

But maybe it doesn't matter since we're currently discussing running
WebSockets only over existing ports 80 or 443, and those already require a
handshake.  Sadly, with a required handshake, WebSockets will always be
slower (for initial latency) than hanging GETs.  The extra bandwidth is
cheaper than the extra RTs.

>From a performance perspective, some times of interest:
    Average Round Trip Time:   ~100ms
    Serialization latency for HTTP request and response headers (typical
sizes of 350 & 450 bytes) over a 1Mbps link:   ~6ms

If we run WebSockets on non-standard ports, what handshake is required?
 Maybe we should address that in the spec.  It seems this is the only way to
remove a RT.

Mike

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jul 2010 13:25:42 -0700

On Thu, Jul 22, 2010 at 1:04 PM, Scott Ferguson <[email-masked]> wrote:
> The issue is the number of request/response before the initial payload is
> sent. If we require the client to wait to validate the server before it can
> send its payload, the initial performance will be slower than a HTTP POST.
>
> That delay may be perfectly acceptable, but some people have raised it as an
> issue in the group. I'm fine with the delay myself, but I think it should be
> resolved.

We need to know the server understands web sockets before spamming
them with attacker-controlled bytes.  If we don't, we'll repeat the
long and tragic history of cross-protocol vulnerabilities caused by
HTTP POST.

This is a hard requirement.

Adam

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 22 Jul 2010 13:50:44 -0700

> This is a hard requirement.
Then I think it should be added to the requirements document, because 
that's a fundamental design decision.

-- Scott

---

From: Adam Barth <[email-masked]>
Date: Thu, 22 Jul 2010 13:53:39 -0700

On Thu, Jul 22, 2010 at 1:50 PM, Scott Ferguson <[email-masked]> wrote:
> Then I think it should be added to the requirements document, because that's
> a fundamental design decision.

   REQ. 19:  WebSocket should be designed to be robust against cross-
      protocol attacks.  The protocol design should consider and
      mitigate the risk presented by WebSocket clients to existing
      servers (including HTTP servers).  It should also consider and
      mitigate the risk to WebSocket servers presented by clients for
      other protocols (including HTTP).

Adam

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jul 2010 14:03:57 -0700

> vulnerably to cross-protocol attacks is just silly.

HTTP is not vulnerable.  Please understand what that means before
misdirecting accusations.  The only things vulnerable there are the
invalid and insecure implementations of SMTP, DNS, FTP, etc.
They will continue to be vulnerable no matter what we do in hybi.

> http://www.remote.org/jochen/sec/hfpa/hfpa.pdf

I have before.  Did you read it?

  3.3	Vulnerable protocols

  Generally, all ASCII command based protocols might be vulnerable
  to this attack. This includes, but is probably not limited to:
  FTP, SMTP, NNTP, POP3, IMAP, and IRC.  Servers using binary
  protocols like SSH or DNS will generally close the connection
  if they receive anything they don’t understand. Because using
  this attack implies that there is always the HTTP protocol
  overhead at the beginning, there is no way to use this attack on
  ”binary” protocols.  Some protocols like the one used for rsh
  require that the connection originates from a TCP port smaller
  than 1024, which is generally not the case for a request
  originating from a web browser.

Just to reiterate, *other* protocol services are vulnerable to attack
from a browser *using* HTTP if those *other* protocol services do not
validate an incoming request.  This is a well-known implementation
issue with older Internet services due to excessive adherence to the
robustness principle.  It is not a vulnerability in HTTP and is not
caused by HTTP.  You can even avoid the issue entirely, using HTTP,
by implementing transfer-encoding and sending all message bodies compressed.

> results in this area, all of which use HTTP to attack other protocols.

Which means the other implementations are vulnerable, not that HTTP
is vulnerable or even that other protocols are vulnerable.  This is
a trivial thing to fix on any one of those implementations, which is
why we tell those implementations that *they* have a security
vulnerability (so that the people who can fix it, will fix it).

....Roy

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 22 Jul 2010 14:11:08 -0700

>       other protocols (including HTTP).
Yes, but that's not specific enough for your requirement because 
"consider and mitigate" doesn't necessarily imply "client must validate 
server before sending content."

As long as everyone's clear and agrees on the "client must validate 
server before sending content" requirement, then we're fine.

-- Scott

---

From: John Tamplin <[email-masked]>
Date: Thu, 22 Jul 2010 17:50:51 -0400

On Thu, Jul 22, 2010 at 5:03 PM, Roy T. Fielding <[email-masked]> wrote:

> You can even avoid the issue entirely, using HTTP,

Actually, it probably wouldn't be that hard to construct plain-text which
would produce a given attack text as a substring of the compressed output,
and it might even give an attacker away to produce binary framing that they
otherwise would be unable to.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 23 Jul 2010 00:16:44 +0200

Hi Maciej,

> 1) Reloading an image from a given URL, the in-browser attacker cannot use any method other than GET or set any custom headers beyond what the UA sends normally. And the attacker definitely cannot…

Right.

> 2) Posting a form, the in-browser attacker cannot use any method other than GET or POST, set any custom headers beyond what the UA sends normally, or send content in anything but a short list of ex…

OK.

> 3) Using cross-site XMLHttpRequest, the attacker cannot use any method other than GET or POST, set any custom headers, or include a message body in anything but a small set of formats, unless the t…

I was not aware that OPTIONS was possible there.

> Even given all that, many sites do in fact have vulnerabilities to cross-site form posting. This class of attacks is called Cross-Site Request Forgery, or CSRF. Typically, these are attacks on inte…

Yes I know them too and have experimented with that about 10 years ago, at times
I think there was no name for this practice. It was fun to put some IMG SRC links
on pages referencing printer_ip:9100 and see the face of people observing their
printers dump pages of HTTP requests when they just loaded the page ;-)

> Because WebSocket would require a successful handshake before sending any actual messages, while an HTTP POST with XHR would blindly send the post body without validating the handshake. If the serv…

OK I understand your example now, which is particularly interesting
because it's very close to the initial issue of passing some requests
in the extra bytes of the handshake.

So it seems to me that what makes WS<->HTTP cross-protocol attacks possible
is just the fact that we're using one of the GET, POST, OPTIONS method that
can be controlled from the JS running on the browser.

A few weeks ago I asked why we wouldn't use the CONNECT method in the handshake,
in conjunction with the 200 Established response. There were some objections
that this would either possibly be incompatible with some existing servers
or may cause some havoc on some reverse-proxies that also happen to support
proxy commands.

But after thinking about it, any method can be used with an Upgrade request,
what's important is that we get the 101 in response. Even a new dedicated
method would do it fine and definitely put an end to any such WS<->HTTP
confusion. Would it be something that we could dream of to have, say, a
new "UPGRADE" or "SWITCH" method that could be used only with an Upgrade:
header and Connection: upgrade in order to get a 101 Switching protocols
response ?

We could then have something like this :

        SWITCH /demo HTTP/1.1
        Host: example.com
        Connection: Upgrade
        Sec-WebSocket-Key2: 12998 5 Y3 1  .P00
        Sec-WebSocket-Protocol: sample
        Upgrade: WebSocket
        Sec-WebSocket-Key1: 4 @1  46546xW%0l 1 5
        Origin: http://example.com

        ^n:ds[4U

We could state that this method would not allow 2xx responses, only
only 101, 3xx, 4xx or 5xx would be allowed.

> Do you mean the specific mechanism currently in <http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76> or something else?

Yes/no, sorry for the confusion, I was meaning the *standard* HTTP upgrade
mechanism which makes use of Upgrade: <proto> + HTTP/1.1 101 Switch proto.
RFC2817 defines it in more details than RFC2616. This is what WS tries to
use, but since there's this absurd ambiguity whether it's HTTP or not, I'd
prefer to talk about the real spec.

> I don't think anyone has previously invented a solution to this problem, so we don't have the option of using a well-vetted 10-year-old design. I also do not think passing off responsibility to ano…

In fact I was talking about the HTTP Upgrade which is defined since
RFC2068 (01/1997) and which WebSocket tries to use. While it's not
much used, at least it's not a new design.

> I think it would be worth enumerating the pros and cons of the TLS-based approach and recording them somewhere (such as on a wiki page).

That's a good idea, and we could also do that for other suggestions
that come back from time to time (eg: adding this or that HTTP header,
or using this or that method). I have no idea where this can be put
though.

Regards,
Willy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 22 Jul 2010 15:26:26 -0700

On Jul 22, 2010, at 2:50 PM, John Tamplin wrote:

> Actually, it probably wouldn't be that hard to construct plain-text which would produce a given attack text as a substring of the compressed output, and it might even give an attacker away to produce binary framing that they otherwise would be unable to.

Interesting.  I have never seen anything like that in practice because
gzip bit sequences are rarely in the ASCII plane.  It could be prevented
by keeping control of the dictionary (mapping LF to something impossible).

....Roy

---

From: "Simon Pieters" <[email-masked]>
Date: Fri, 23 Jul 2010 09:26:43 +0200

On Fri, 23 Jul 2010 00:16:44 +0200, Willy Tarreau <[email-masked]> wrote:

>> I think it would be worth enumerating the pros and cons of the  
>> TLS-based approach and recording them somewhere (such as on a wiki  
>> page).
>

It seems http://trac.tools.ietf.org/bof/trac/wiki/HyBi#Proposals already  
has a start of such a thing.

-- 
Simon Pieters
Opera Software

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 26 Jul 2010 09:58:23 +1000

Adam,

your analysis of the cross protocol security issues continue to be very
informative.

But why does it have to be in the tone of "the WHATWG guys are the only ones
that understand anything and there is no need to change anything"?   Perhaps
you are reacting against the harsh tone that is sometimes directed towards
the WHATWG guys in this WG and feeling like you need to defend them.  Well
I'm sorry about that tone, but there has been significant stonewalling from
that quarter and there is a lot of frustration here that many other concerns
raised are being rejected.

So thanks for expanding on some of the details of the cross protocol
attacks, and you have rightly drawn attention to deficiencies in some of the
simplistic counter proposals.
However, the bit that is missing is how your analysis relates to the current
proposed solution with it's rigid content, space counting, random bytes
without frames etc.   Surely that cannot be the only solution that addresses
these issues?   Can't we try to find a solution that both reduces the cross
protocol issues, but is also less objectional to most here in the WG?

To harden a websocket server against attacks from other protocols, there are
many things that framing can help with.  For example, we could shutdown
connections after any bad frames (we are on reliable transports so no need
to forgive bad frames) and/or have a simple frame checksum.

To ensure that websocket clients are themselves not tools to attack other
protocols we could also have less minimalistic framing. Lengths and
checksums are going to be more of an obstacle to attackers than simple
sentinel framing.

I have yet to see a good description of an attack vector that is cut off by
the rigid handshake, the space counting or the unframed bytes that could not
be addressed by far simpler and less objectional means.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 26 Jul 2010 11:38:33 +1000

On 23 July 2010 02:46, Maciej Stachowiak <[email-masked]> wrote:

> I originally suggested computing a hash of the nonce with other data (or
> some other nontrivial computation) rather than just echoing it back.
>

Maciej,

I agree that avoiding a simple echo would be very good protection against
almost all forms of injection attack.

I think the handshake request contains a random token generated after the
URL has been passed to the browser. The handshake response could then
contain a random token generated after the connection has been accepted,
plus a simple hash of the combination of the request and response tokens.

This would allow a client to validate that the server had performed
websocket specific calculations (in addition to sending a 101 response) and
appears to address all the same security concerns that the current space
counting and random bytes do.

I beleive these tokens and hash should be  carried in normal HTTP headers
for the upgrade handshake, and that other mechanisms should be considered to
address the fast fail detection of non-ws handling intermediaries.

regards

---

From: Maciej Stachowiak <[email-masked]>
Date: Sun, 25 Jul 2010 18:44:37 -0700

On Jul 25, 2010, at 6:38 PM, Greg Wilkins wrote:

> I originally suggested computing a hash of the nonce with other data (or some other nontrivial computation) rather than just echoing it back.
> 
> Maciej,
> 
> I agree that avoiding a simple echo would be very good protection against almost all forms of injection attack.
> 
> I think the handshake request contains a random token generated after the URL has been passed to the browser. The handshake response could then contain a random token generated after the connection has been accepted, plus a simple hash of the combination of the request and response tokens.

I don't see the value of a random token in the handshake response. How does that provide any additional protection? Anything echoed produced by the server might well be controlled by the cross-protocol attacker, so a random token generated by the server is no more effective than a fixed string.

> 
> This would allow a client to validate that the server had performed websocket specific calculations (in addition to sending a 101 response) and appears to address all the same security concerns that the current space counting and random bytes do.
> 
> I beleive these tokens and hash should be  carried in normal HTTP headers for the upgrade handshake, and that other mechanisms should be considered to address the fast fail detection of non-ws handling intermediaries.

I would have to see a more detailed version of this proposal to evaluate its security. I think it is effectively only as strong as sending a nonce (at an arbitrary place in the request headers), and echoing back a simple hash of that nonce that doesn't include any other data. Which I think is at the weaker end of nonce-based defenses.

Regards,
Maciej

---

From: Adam Barth <[email-masked]>
Date: Mon, 26 Jul 2010 09:00:05 +0200

> your analysis of the cross protocol security issues continue to be very

Thanks.  Glad you're finding it useful.

> raised are being rejected.

I'm not sure why you consider me "one of the WHATWG guys."  I
participate about equally in WHATWG, W3C, and IETF.  At the moment I'm
more involved with IETF than the other two.

> protocol issues, but is also less objectional to most here in the WG?

I'm sure there are other solutions.  For example, I proposed a
handshake a while back that isn't rigid at all.  If you feel the
current proposal doesn't meet your requirements, please feel free to
propose something else.

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 26 Jul 2010 22:10:19 +0100

Roberto Peon wrote:
> 
>    I have some experience with dealing with broken HTTP client
>    implementations in nearly-impossible-to-update devices such as
>    televisions, phones, etc.
> 
>    We're starting from scratch, so hopefully we won't have the confusion
>    of poor implementations that HTTP does, but I'd end up really sad if I
>    had to make my server not use advanced features of a *new* protocol
>    because some clients identify their capabilities wrongly

Unfortunately that kind of breakage is inevitable at some point, if
the protocol is widely adopted with many implementations.

Which is why we should approach any identification and negotiation in
such a way that there's a way out of that trap when it happens.

There are several approaches, and we can learn from what happened with
HTTP what methods actually work.

One way is to encourage widespread use of "user-agent" style strings
which are not meant for negotiation and mostly not used for it (and
therefore there is little incentive to lie in them), but can be used
as an emergency "I don't trust this implementation" get out clause.

When those are used and not abused, they work quite well to deal with
occasionally buggy implementations.  It is important that version
numbers are included.

That said, we see from HTTP that only works if there are other ways to
recognise what people actually want to recognise at the other end so
there is no incentive to parse the user-agent all the time, and no
incentive therefore to lie in it.  HTTP user-agent is now weaker at
this because it seems to have become the de facto way of recognising
characteristics of the device, such as whether it's a mobile and what
type, so there's an incentive to lie.

If a severe WebSocket fault occurs with a widespread, non-updatable
television with a client _or_ server (yes[*]), and it does not display
any identifiable characteristic (not least because the standard
discourages it :-/), the remaining solution to that is to invent a new
capability whose sole purpose is to mean the same as the old one, but
is not used in that widespread, faulty equipment.

Then at least your server can use the advanced feature with newer and
upgradable client implementations, and vice versa.

-- Jamie

[*] Coincidentally as we speak I am developing a server which runs in
a TV and uses comet-style streamed HTTP to both local and remote
(tunnelled) browsers, and may use WebSocket if it proves usable.
Since it's an embedded implementation in C you can bet it probably
will suffer from the issues discussed. :-)

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 26 Jul 2010 23:16:06 +0100

> long and tragic history of cross-protocol vulnerabilities caused by

Compliant implementations *already* spam the server with
attacker-controlled bytes in the initial handshake.

All proposals to include early payloads are explicitly designed to
tunnel it safely in a way which does no more harm than the initial
handshake.

I don't think I've seen any proposals for an insecure method or
sending arbitrary attacker-controlled bytes early, and I wonder if
you're making an incorrect assumption that.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 26 Jul 2010 23:26:35 +0100

> server before sending content" requirement, then we're fine.

It seems a strange leap of logic, as it isn't the only way to mitigate
the attack risk and it has perceptible negative consequences.

Anyway I can't see how "client must validate server before sending
content" can be satisfied.  Clients are free to encode their content
in the subprotocol field, and JavaScript apps might actually resort to
that if it's the only way to hack around that side of the RT delay.
I'm not sure if the API gives any way for JS apps to retrieve anything
interesting sent on the other side of the handshake.

Note I have no problem with "client must validate server before
sending WebSocket encoded frames / dangerous frames / non-HTTP frames
(take your pick)" .

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 26 Jul 2010 23:32:47 +0100

Maciej Stachowiak wrote:
> I believe the TLS-based handshake (as proposed by Adam) is more
> robust than solutions involving nonces and challenge/response built
> into an HTTP-like handshake. The reason for this is that it's hard
> for the attacker to control the bytes sent, other than to cause the
> initiation of a TLS handshake, which is something that all network
> services are already exposed to by browsers. The attacker cannot get
> to the next-level protocol running over TLS unless the server
> explicitly opts in. With HTTP-based schemes, it takes much more care
> to avoid sending information as the "wrong" protocol.

Isn't the TLS handshake rather fragile and vulnerable, by virtue of
lots of services sitting behind TLS and using it only to get an
encrypted channel?  If you can open a connection and start TLS, say on
the ftps/imaps or even https port then it's basically unprotected
bytes again after the TLS stage, wrapped *inside* TLS, giving a false
sense of security.

Or is there some extra magic sent in the TLS setup which will be
consistently rejected by non-WebSocket TLS servers?

Thanks,
-- Jamie

---

From: Adam Barth <[email-masked]>
Date: Tue, 27 Jul 2010 00:39:20 +0200

> consistently rejected by non-WebSocket TLS servers?

Indeed.  That's the problem that the using the "next protocol" TLS
extension solves.  During the TLS handshake, the client and server
negotiate that they'd both like to start talking WebSockets after
they're done with the handshake.

Adam

---

From: Scott Ferguson <[email-masked]>
Date: Mon, 26 Jul 2010 17:09:37 -0700

> the attack risk and it has perceptible negative consequences.

Yes, which is why I thought it important to ask if the delay until 
validation is a hard requirement or not.

IMO, it may be worth spending a little time on proposals like Adam's to 
see if it's possible to reduce the handshake overhead below the server 
validation penalty.

> content" can be satisfied.  

Yes. I should have been more precise and said "before frame content".

> Clients are free to encode their content

-- Scott

-- Scott

---

From: Patrick McManus <[email-masked]>
Date: Sat, 31 Jul 2010 12:08:26 -0400

> On Wed, Jul 21, 2010 at 5:11 PM, Willy Tarreau <[email-masked]> wrote

+1 for me as well.

More specifically, I think the web sockets specification should define
the unique web socket protocol (shocking!). That protocol should NOT
bear a sometimes-it-is-convenient-to-look-like-http-without-being-http
look. Not being HTTP, that is just asking for confusion. It should of
course do the security checking that the list has been discussing.

If clients and servers would like to use that protocol directly - that
would be terrific and I doubt anyone here would be upset about that.

If clients and servers need to tunnel over HTTP in order to use
websockets, then use one of the established mechanisms for doing that
(e.g. Upgrade). It is fine to include a section in the websocket
specification on "upgrading to websockets over http" with implementation
advice, but that should be its scope. 

The notion of the WG endorsing a specification that sends
non-compliant-HTTP to HTTP ports is non sensical for the IETF. My
personal interpretation of the mailing list archive says that their is
consensus on that. I would hope that browser vendors would choose not to
ship such a scheme in anything except an experimental fashion.

It does cost 1 RTT to do an HTTP upgrade safely before allowing any
websocket or user supplied data to hit the wire. That is not a property
of websockets - that is a property of HTTP. Consider it the cost of
doing business if you would like to piggyback on http's port. honestly,
for the value of getting access to port 80 real estate - its a pretty
cheap price! If you use a different port then it can be avoided - the
cost and benefit are clearly defined.

personally, I'm not all that concerned about the 1st event latency for
websocket use cases. If they are no-wait instantaneous
events/status-dumps then they are well suited to http instead of
websockets. The latency of each additional event while the page is in
operation is much more important and is the websocket sweet spot. I
think this point of view is even more defensible if we include
multiplexing from day 1 - that will severely reduce the number of
handshakes.