Subject: [hybi] Masking for WS data content

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 18:54:20 +0200

Hello, Ian.

As the previous threads have been swamped in meta discussions, here is
a clear request "for comments" :)

The masking feature as used in the current spec, has, as one of the
subgoals, prevention of a family of attacks
where the attacker can cause mischief by control of actual
"data-on-the-wire" plaintext.

While this family of attacks may well exist, here is the counterargument:

1. HTTP POST currently has the ability to send binary data of arbitrary content.
2. HTTP POST is much more widely deployed than WS
3. HTTP POST headers can be faked in each WS message

Given the above, and that there is no reason to have WS provide "more"
security than HTTP,
since HTTP will be preferred medium in practice,

REQUEST: Is there a reason to not use fake POST-like headers in each WS message,
thus bringing the security to the same level as HTTP POST ?

This would preserve sendfile() as well as easy debug and make
implementations simpler.

If there is no such reason,
RECOMMEND: removal of masking feature and addition of fake HTTP-POST
headers to each
outgoing WS frame.
OR: make this security mechanism OPTIONAL.

--
 silvio

---

From: Dave Cridland <[email-masked]>
Date: Fri, 25 Feb 2011 17:09:31 +0000

> thus bringing the security to the same level as HTTP POST ?

The problem with this particular proposal is that a proxy is likely  
to then assume it really is POST requests, and handle accordingly -  
this may well include sending the requests via different TCP  
connections, amongst other things. There are ways of handling this,  
for example by including metadata in the body in order to handle  
ordering, and using "long response", pipelining, and multiple  
connections in order to keep latency low for both send and receive -  
all of these techniques have been deployed very successfully within  
the XMPP/BOSH framework documented as XEP-0124/XEP-0206.

But then you may as well not bother with WebSockets at all, and just  
use BOSH itself, which can be done perfectly well in existing  
Javascript anyway, with the inevitable cost of the multiple  
connections and POST header overhead.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 25 Feb 2011 18:12:37 +0100

> 3. HTTP POST headers can be faked in each WS message

4. HTTP POST only allows the server to talk once the client asks it to. The
   server is not able to send frames to the client when it wants to do so.

> since HTTP will be preferred medium in practice,

There's no reason to make WS more secure than HTTP however there are reasons
to ensure that once deployed, WS preserves browser security, so what is not
covered by current HTTP implementations must be covered by WS.

> thus bringing the security to the same level as HTTP POST ?

There's no "POST-like headers". Please be specific. You talk as someone who
doesn't know the HTTP protocol at all, which makes it very hard to understand
your concepts or how they would apply.

Regards,
Willy

---

From: John Tamplin <[email-masked]>
Date: Fri, 25 Feb 2011 12:13:19 -0500

> This would preserve sendfile() as well as easy debug and make

The overhead of including a fake POST request in every WebSocket frame
would be far larger than a 4-byte masking key and the overhead of
doing XOR.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 19:38:53 +0200

> would be far larger than a 4-byte masking key and the overhead of

... and the overhead of not being able to use sendfile()
... and the overhead of no HTTP-style debug capability
... and the overhead of having to pass over the data multiple times

Really, are you suggesting XOR'ing the entire data frame
has less impact than a short HTTP-POST header ?

--
 silvio

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 25 Feb 2011 18:44:49 +0100

> ... and the overhead of not being able to use sendfile()

This one was considered not critical since sendfile() is not commonly
used from the client to the server.

> ... and the overhead of no HTTP-style debug capability

That was the main reason why we finally opted for a per-frame mask instead
of a per-session+frame mask.

> ... and the overhead of having to pass over the data multiple times

Huh ?

> has less impact than a short HTTP-POST header ?

You're still comparing apples to oranges. A POST works only one way and
for one single message. WS allows any number of messages to flow in any
direction at any time.

Certainly if I want to remain connected to the server in a chat discussion
WS will be much more efficient than sending as many POST requests as I
expect to receive characters, words or lines !

Willy

---

From: John Tamplin <[email-masked]>
Date: Fri, 25 Feb 2011 12:52:58 -0500

> ... and the overhead of not being able to use sendfile()

Existing browsers do not use sendfile to send an in-memory string
supplied by JS.  Even if they were to add that in the future, the loss
of sendfile on the client was deemed an acceptable compromise.

> ... and the overhead of no HTTP-style debug capability

It is already a binary protocol, and hopefully be commonplace.

> ... and the overhead of having to pass over the data multiple times

It depends on how your code is structured whether you would need to do
so (for example, if you are converting UTF16 from JS to UTF8 you could
do the XOR at the same time), but even if you do then yes.

> has less impact than a short HTTP-POST header ?

Expected uses of WebSockets are likely to be small frames (see the
thread about compression from last May).  Adding a fake POST header of
dozens of bytes at a minimum would easily add a significant size
overhead to every frame.  Consider chat-like applications where you
are sending a few keystrokes at a time -- those currently can be sent
in 6+n bytes, compared to maybe 40+n.

Sizes are also extremely important for mobile applications, where the
low-power radio mode can only be used for packets of around 128-256
bytes (depending on carrier settings), and adding a large fixed
overhead to every frame (in addition to TCP/IP overhead until ROHC is
widely deployed) means very few frames can be sent without fully
powering up the radio (which also has latency issues aside from
battery issues).

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 19:55:18 +0200

>   server is not able to send frames to the client when it wants to do so.

If the server is a compliant WS endpoint, the server will ignore the POST
header and send the frames to the client when it wants to.

If the connection is made through an intermediate that doesn't allow server to
reply to client when it wants to do so, it is irrelevant whether POST or any
other header is used.

> covered by current HTTP implementations must be covered by WS.

Agreed. Point is that WS should be "as secure" as HTTP, but there is
no reason to
make it "more" secure, sacrificing useful features such as debug simplicity.

> your concepts or how they would apply.

"POST-like" in the meaning that they are intended to look like valid
POST headers
for consumption by intermediaries, but not by non-WS HTTP servers.

--
 silvio

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 19:58:37 +0200

> expect to receive characters, words or lines !

It seems you are talking about HTTP and not WS.

The discussion was about WS protocol, using POST headers just to make sure that
the security implications of binary data in the frame are same as for HTTP POST.

The actual communication is normal WS, the POST header is not supposed to even
be parsed by the WS endpoint.

--
 silvio

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 20:18:15 +0200

> It is already a binary protocol, and hopefully be commonplace.

Hopefully it will, but that doesn't remove the fact that plaintext protocols are
much easier to debug than encoded ones.
As an example, a binary data stream in WS message would be very hard to
debug, with or without WireShark decoder, while a simple netcat script
to strip the WS control bits, if rest of data is sent plaintext, is
very easy to write.

> in 6+n bytes, compared to maybe 40+n.

The tradeoff between considerable complexity in both normal operation and debug,
against slightly increased bandwidth. Look at HTTP: it's so prevalent
today because
it's so simple and easy to debug, while sacrificing bandwidth.

> powering up the radio (which also has latency issues aside from

It seems that  HTTP with similar or worse overhead does not have much
troubles in mobile space.

Let's try to save a resource that is actually scarce on current servers:
cpu time, memory, cache locality. Bandwidth at these scales is not scarce
by any definition.
When a reasonably average WS server can saturate a 100Mbit connection at
lower than 50% cpu usage, _then_, and only then should cpu time be
traded for bandwidth.

--
 silvio

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 25 Feb 2011 19:39:29 +0100

>"data-on-the-wire" plaintext.

We have masking because people wanted to initiate the protocol over port
80 where usually HTTP is spoken but'd not want to stick to HTTP framing.
However, there is no method to switch from the HTTP protocol to some
other protocol that is universally implemented, which can lead to a
situation where traffic that is meant to be Websocket communication is
interpreted as HTTP communication. To ensure such confusion cannot be
abused, traffic is masked with an unpredictable masking key.

No masking would mean you cannot use ports commonly used for HTTP or you
have to stick to HTTP framing or you have to tolerate some considerable
risk that someone will find a setup that is vulnerable to attack using
Websocket implementations.

>REQUEST: Is there a reason to not use fake POST-like headers in each WS
>message, thus bringing the security to the same level as HTTP POST ?

Making all communication conform to the HTTP protocol specifications is
indeed an alternative as far as security goes, but this would not play
very nicely with other HTTP uses (compare timeouts in request->response
use versus fully bidirectional uses, for instance) and would have higher
overhead than Websocket as it is being proposed at the moment.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 25 Feb 2011 19:54:49 +0100

> reply to client when it wants to do so, it is irrelevant whether POST or any

Please stop talking about that magical "header", once again there is no such
"header". There is a well-defined connection setup handshake which guarantees
that if it succeeds, you have a bidirectional connection between both ends,
this is called HTTP Upgrade. And it is irrelevant to the method (POST, GET,
...). Your POST method will not work because the server can't send anything
to the client before it gets the request. Also, many intermediaries will
buffer the request body as well as the response body.

Silvio, please take the time to look at what an HTTP request looks like,
especially how a message is defined (request/status line, headers, body).
This is very important and is the only way to understand why WS works and
why your POST-based proposals don't work.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 25 Feb 2011 19:59:03 +0100

> It seems you are talking about HTTP and not WS.

WS works on top of HTTP infrastructure. It is designed to be compatible with
HTTP-compliant clients, firewalls, proxies, caches, intercepting intermediaries,
load balancers, servers (dedicated or shared), etc... All the complexity that
you perceive comes from the fact that we're making use of the already deployed
infrastructure. Otherwise we could as well decide to work on a new port with a
very simple protocol and wonder why nobody adopts it.

So when I'm talking about WS, yes I'm also talking about HTTP and that's on
purpose, because a connection request must be able to pass through all the
chain described above, and we want to be able to safely detect dirty failures.

Willy

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 22:17:15 +0200

> indeed an alternative as far as security goes, but this would not play

You and Willy misunderstood.
The POST headers are not meant to be parsed by the server. They are added
when the server is already talking in WS protocol, not HTTP. Thus, these won't
actually mean anything to the server, and should be ignored.

The only reason to add them is to make sending any data content possible through
HTTP ports, with security implications _no worse_ than normal HTTP POST.

Again, the requests are _not_ HTTP requests. This is still WS, with a
"POST /ws HTTP/1.1\r\nHost: ws\r\n\rn" header attached in front.

This removes the need for masking, brings back debug and low
complexity and costs ~20 bytes overhead.

--
 silvio

---

From: John Tamplin <[email-masked]>
Date: Fri, 25 Feb 2011 15:23:14 -0500

> complexity and costs ~20 bytes overhead.

As I count your example, it is 31 bytes.  Also, I think you need a
Content-length header, and perhaps even a Content-type header, which raises
the overhead even further.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 25 Feb 2011 21:44:42 +0100

>complexity and costs ~20 bytes overhead.

You need 20 bytes for a Content-Length or Transfer-Encoding header alone
so the overhead would be quite a bit more than that and you would have
to show that there are no vulnerabilities in intermediaries that go off
only under certain pipelined request situations (I am assuming that you
allow the client to send a second request while the server has not yet
responded to the last one) with a server that can be controlled by the
attacker, and probably also that they do not get confused after seeing a
Upgrade but then seeing what looks like more HTTP requests. In addition
to explaining how this would be much better than simply using HTTP POST
requests as they are available today.

Do note that these alternatives have been studied long before this group
was chartered and they have been discussed at lengths in the group too,
and it is not what the group ended up with for one reason or another. If
you want more than putting your opinion on record, you have to come up
with a very detailed and well-researched argument with new information
for the issue to be re-opened within this working group. You also have
the option to define a different protocol in an Internet-Draft and run
it by interested parties outside the working group.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Silvio Ventres <[email-masked]>
Date: Fri, 25 Feb 2011 23:00:32 +0200

> so the overhead would be quite a bit more than that and you would have

For a caching intermediary, by HTTP/1.1, seeing a POST method should
be enough to stop trying to cache it, whether there are valid headers after
the first line or not. Indeed, "broken" UserAgents might use POST with
broken headers and still expect to _not_ be cached.
Whether the POST
Thus, based on the HTTP spec section 13.10, simply using "POST /ws HTTP/1.1" is
enough to bring WS security to the level of HTTP POST.

> Upgrade but then seeing what looks like more HTTP requests. In addition

If they were confused, than regular HTTP POST would have the same vulnerability.
As HTTP POST is much wider deployed, WS does not need to go beyond HTTP
level of security, especially sacrificing useful features such as debug and low
implementation complexity.

> requests as they are available today.

As available today, HTTP is a request-response protocol.
WS is a fully bidirectional communication protocol. Whether it adds static
POST-like headers on top of its frames to support broken proxies or not,
it remains bidirectional.

> was chartered and they have been discussed at lengths in the group too,

It seems as this specific alternative was never discussed in this group before.
Also, it seems that this group does not include enough people with actual
real-world web development and operational experience, for which losing debug
capability will easily multiply the development time many times.

--
 silvio

---

From: John Tamplin <[email-masked]>
Date: Fri, 25 Feb 2011 16:08:51 -0500

> enough to bring WS security to the level of HTTP POST.

No, because you have binary data in the POST (the WebSocket frame), so you
have to include a Content-length header to ensure that the intermediary
doesn't treat any WS payload data as an HTTP request (or else you have to
encode the WS payload, which will be at least as expensive as the XOR you
are trying to get rid of).

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 25 Feb 2011 22:37:46 +0100

> it remains bidirectional.

It is only for valid intermediaries but will fail on non-compliant ones
since they'll expect a response before processing a second request.

Willy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 25 Feb 2011 22:45:02 +0100

>broken headers and still expect to _not_ be cached.

Well, I would suggest you write down a protocol trace detailing which
bytes go over the wire and then show how an intermediary that does not
recognize the Upgrade mechanism will not pick up bytes controlled by an
attacker (who might also control the server) as HTTP request. That is
what masking is trying to prevent. You should find that you can put the
attacker controlled bytes only in either the pseudo-request header, and
that would require some form of escaping to handle CR LF bytes, or it
would have to go into the body of the pseudo-request, which requires a
Content-Length of Transfer-Encoding header.

Also note again that anything that's different from what is literally
possible today with the most widely deployed web browsers would have to
be shown reasonably risk-free, and while HTTP allows things like pipe-
lined requests, they are not in common use with the mainstream browsers
so there may well be security issues with them that would not surface
with masking until browsers make use of pipelined requests and similar
things your proposal would allow to be different from vanilla HTTP as
it is deployed with mainstream browsers.

Do keep in mind that the length encoding and frame flags are already
complicated to address various performance concerns, if there was no
problem with adding dozens of bytes of per-frame overhead, we'd have
a much simpler frame layout; it would very much unlikely that even if
you could address the security issues with your design that it would
gain considerable support in the working group. While you may value
plain text, others have other priorities, and you can't optimize for
all of them simultaneously.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Silvio Ventres <[email-masked]>
Date: Sat, 26 Feb 2011 00:56:02 +0200

> since they'll expect a response before processing a second request.

There is no problem with intermediaries that are not HTTP/1.1 compliant.
These intermediaries, being non-deterministic, are not the targets of
the WS protocol spec. The only thing important with these is that they
don't accidentally come to any harm by being used by WS clients.
This is done by adding the POST header, which makes it impossible
for WS clients to cause _more harm_ than regular HTTP POST requests.
That's all that needs to be done with non-conforming proxies. There's no
need to think whether they would forward the request or not, as, being
non compliant to HTTP/1.1, they would not be able to work with the WS
protocol anyway.

Please keep the problem definition in mind.
The problem was never "work with broken proxies" but instead "cause no
harm to broken proxies".
Indeed, being broken, those proxies cannot be relied upon to do anything
deterministic, so there is no way to ensure the protocol to work with them.

--
 silvio

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 26 Feb 2011 00:05:22 +0100

> don't accidentally come to any harm by being used by WS clients.

No this is not the only important thing. Another important thing is that
we ensure that non-working situations cause a quick failure that the client
can detect. Having a faulty request remain hanging is not an acceptable way
of handling the majority of the failure situations.

Willy

---

From: Silvio Ventres <[email-masked]>
Date: Sat, 26 Feb 2011 01:26:21 +0200

> of handling the majority of the failure situations.

A broken proxy is _broken_.
It's not deterministic. You cannot know or control what it may do
whatever you feed it.
Thus, however fast you might want to make it fail, you cannot force it
to happen.

--
 silvio

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 26 Feb 2011 00:49:28 +0100

> Thus, however fast you might want to make it fail, you cannot force it

Agreed, and that's why you can't claim that your discutable workaround
will fix every broken proxy. Also, you are deciding that you'd want many
deterministic proxies to randomly misbehave, and I don't find that
acceptable. For many months, WG participants have suggested minor
changes to help get the most possible deterministic behaviour because
this is what most of us are looking for. You can't get there and decide
that this is useless and let the broken situation fail the dirty way for
many users. That would definitely stop adoption.

Willy

---

From: Silvio Ventres <[email-masked]>
Date: Sat, 26 Feb 2011 01:55:14 +0200

> will fix every broken proxy. Also, you are deciding that you'd want many

The workaround doesn't claim to _fix_ anything.
It only brings the security level of WS, in regards to causing harm to
misbehaving
proxies, to the level of HTTP protocol using POST.
After taking care of that problem, there's nothing else to do with
_broken_ proxies.

> many users. That would definitely stop adoption.

You cannot claim that you support operation over broken software. The "broken"
part means that you cannot possibly know what is broken in it.
You can try not to cause harm to it, and that's what is done, by masking in the
current WS spec, and by POST-header attachment in the proposed solution.
In both cases, if the proxy is not broken, and is compliant with HTTP/1.1,
there's no need neither for the POST-header workaround nor for masking.

--
 silvio

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 26 Feb 2011 04:01:40 +0100

>proxies, to the level of HTTP protocol using POST.

There are only two possibilities here, either we abandon the Websockets
effort, in which case there are no new security issues, or we propose to
change something by continuing the Websockets effort, in which case we'd
have new security issues to deal with. There is no way to do something
that is not already possible without, for sure, introducing any new se-
curity issues.

If you want to continue discussing alternatives to masking, at the very
least you will have to make a proposal that can be implemented without
ambiguity whether an implementation conforms to your specification, and
without ambiguity whether there might be some security issue or not. So
far your suggestions are nowhere near that standard.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/