Subject: [hybi] Different server semantics of CONNECT

From: Greg Wilkins <[email-masked]>
Date: Thu, 2 Dec 2010 06:54:52 +0100

I'd just like to note some significant different semantics that will
result on the server from the use of CONNECT.

With HTTP, you can make the gross generalisation that the server
handles the connections while the application handles the requests.
Thus typically a HTTP server will accept an incoming connection
without any consultation with the application, which does not get the
chance to say: "I wont accept that connection".   Only when requests
arrive over that connection are they routed to the application, and
frequently a server will have more than one application running and
will use the host and URL to select the application.   The application
then handles the request.  It does not have the option of refusing the
request, but can respond with an error 404 or 500 etc. plus a hint to
the server that it should close the connection (normally setting
Connection header to close).  With this design, connections can carry
requests for different applications and thus there is natural support
for connection sharing.

In this environment, it made an incoming WebSocket upgrade handshake
something that is essentially an application level concern.   The
server would accept the HTTP connection like any other and then read a
HTTP request like any other, before consulting the host/url to
determine which "application" should handle the request.   Thus
application level code is given the opportunity to accept or reject
the incoming WS connection.  In Jetty, the application is also able to
associate application level objects with the handling of incoming
messages inside the scope of the application context.    On one level,
this application level semantic is very appealing, because it well
models the semantic intent of the WS API of opening a channel from
application client code to application server code.    However, the
issue with this model is that applications have never been good at,
nor should they be asked to, manage protocol level connections and the
resources associated with them.

If we change to CONNECT, then the semantics of the  server
significantly change.   CONNECT does not have a URL, so servers are
not going to be able to treat it like any other HTTP request and route
it through to an application.   The consideration of the CONNECT as a
WS handshake must be done by code outside of the scope of an
application and thus is becomes the servers concern if the connection
is accepted or not.  Of course the server handling of CONNECT can use
the URL from the WS specific headers to identify the application and
then to ask it if the connection should be accepted - but this will be
via new WS specific paths and not by simply routing a normal HTTP
request.  This makes it more disruptive to the server code - but
probably sensibly so.

I think this is a change for he good in the WS semantics.
Applications should not manage protocol connections - a best they
should manage logical channels over those connections.   Having the
server manage the WS connection means that extensions like MUX will be
much simpler to implement, as the protocol connection will not
logically terminate in one application or another.  It will terminate
in the server and the server will take responsibility of routing
individual messages from the connection to the application.  Initially
this will be one to one, but eventually I expect to see a connection
providing messages for multiple applications, just as we do in HTTP

So as much as I share Roy's concerns about CONNECT being potentially
misused, I think it is a step in the right direction semantically at
least.
But I think it is important to consider CONNECT really as a connect
and not as some syntax trick.   So while we might start with

      CONNECT server.example.com:80 HTTP/1.1
      Host: server.example.com:80

I can definitely see the migration path to a dedicated WS port  with
first changing to

      CONNECT server.example.com:pppp HTTP/1.1
      Host: server.example.com:pppp

and then eventually the browser just directly making the connection to port pppp

If that is where we are going, then the important thing to consider
for now, is not to put too much WS semantics into the headers of the
CONNECT request.   If they were instead put into a HELLO frame sent as
the first frame on the connection, then those semantics would work
equally well for direct connection or via CONNECT.  They also have a
natural migration to MUX as we can use an extended HELLO  to create
new channels

regards

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 2 Dec 2010 07:49:29 +0100

> requests for different applications and thus there is natural support

This is the same reason why I find it important to keep routing information
in the HTTP part : the infrastructure must be able to route the connection
request to the proper location without having to accept the WS handshake by
itself, otherwise it will pretend accepting it then the other end might
reject it. With a clean HTTP handshake, the handshake can be forwarded to
the point which can decide if WS is supported or not. Host+port is a minimum.

I agree that having a path looks better, however I don't know if it makes
sense to be able to support multiple WS connection points on a same host+port,
because I tend to consider that the application is defined by host+port and
that the path is used to select what application component is used. But I'm
not the best qualified to decide of the path importance!

Regards,
Willy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 1 Dec 2010 23:18:42 -0800

> and then eventually the browser just directly making the connection to port pppp

I don't think you understood my objection.  CONNECT is defined as a
method that is only applicable to proxies.  WS cannot change that definition.
The request target "server.example.com:80" above is only valid in HTTP/1.1
when sent to a proxy.  WS cannot change that definition either.  Both
are way outside the scope of hybi.  Is that clear?

Sending non-valid HTTP over the Internet's designated web port (80)
causes unanticipated results because there are a whole lot of
implementations out there that do weird, amazing, and occasionally
stupid things to the traffic on port 80.  Therefore, a protocol
in an IETF proposed standard that is expected to be sent over port 80
had better start off as HTTP-compliant (to allow compliant rejections).
Anything else will result in an arms race with the folks who need
to block entropy-generating apps.

What annoys me is that there is no reason to use CONNECT
for this use case.  Just define a new method, such as

   WEBSOCKET /application/path HTTP/1.1
   Host: server.example.com:pppp
   WS: nonce

(the request might include auth headers as well) and then wait for
a response of

   HTTP/1.1 200 OK
   WS: nonce
   Transfer-Encoding: chunked

   88
   whatever-opaqued-bytes-you-want-for-as-long-as-you-want ...

before the client begins sending its stream of non-HTTP socket
traffic.  To satisfy Adam's issues, that traffic can be opaqued
by the sending client via LF-escaping or encryption -- we could
even choose negotiate the kind of opaquing in the initial exchange.
Broken interception proxies may still fail to send the subsequent data,
just like they would for the CONNECT case, but they can't be
tricked into cache poisoning or host-based misdirection.

Note that this only applies to the fallback case of using HTTP
when a separate port doesn't work and no normal proxy has been
configured.  The "fast" websockets protocol could then use TCP,
SCTP, UDP, TLS, or whatever else one might want to specify,
independent of HTTP's constraints.

Every successfully deployed proprietary equivalent to websockets
has faced this same problem with firewall port blocking and
eventually adopted the same kind of fallback solution (e.g.,
Skype, Flash, etc.).

....Roy

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 2 Dec 2010 08:30:32 +0100

> are way outside the scope of hybi.  Is that clear?

Roy, I think I get your point. The presence of a CONNECT request on
a transparent proxy implies an explicit proxy connection for it, and
instead of forwarding it unmolested, it may connect to the destination
and only forward the payload, which is contrary to what is expected.

(...)
> even choose negotiate the kind of opaquing in the initial exchange.

I think that if we can solve all issues with a dedicated method, then
we don't need a dedicated method, because some of the issues are that
some proxies who won't know about that method will process it as any
unknown method (eg: GET) and consider what follows the end of the
request and response messages as a new request. But if we can ensure
that those data can't look like a new request (eg: by escaping LF),
then any method is OK, and the GET+upgrade is OK too.

Willy

---

From: Adam Barth <[email-masked]>
Date: Thu, 2 Dec 2010 00:18:57 -0800

>   whatever-opaqued-bytes-you-want-for-as-long-as-you-want ...

Interesting idea.  I wonder how intermediaries would handle additional
"HTTP requests" stuffed down the socket.  That's something we'd want
to test in the lab and in the wild before adopting this approach.

Adam

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 2 Dec 2010 09:20:57 +0100

> are way outside the scope of hybi.  Is that clear?

Roy,

definitely clear - and I've made very similar points against CONNECT
in the past.

I guess what I'm now suggesting is that a HTTP server that offers
Websocket on port 80 will be acting as a proxy to the real websocket
server (preferably available directly on a high port). So essentially
from a browsers point of view, there will be a configured proxy for WS
on port 80 of the target server.

>   whatever-opaqued-bytes-you-want-for-as-long-as-you-want ...

This would result in a one way only connection, with data only being
sent from server to client.   However, I wonder how intermediaries
would behave if they saw from the client:

    XX
    ws-frame
    XX
    ws-frame

and then from the server

   HTTP/1.1 200 OK
   WS: nonce

   XX
   ws-frame
   XX
   ws-frame

I wonder if  most intermediaries would allow for bidirectional
exchange of chunks on the connection?

My other concern with this approach is that caching proxies are under
no obligation to pass chunks through as they receive them.  For
example, they are entitled to buffer up and aggregate chunks and then
send the "final" content as a content-length encoded body.  Maybe that
can be prevented with suitable non-cacheable headers?

definitely worth evaluating

cheers

---

From: John Tamplin <[email-masked]>
Date: Thu, 2 Dec 2010 10:51:16 -0500

> exchange of chunks on the connection?

When I asked Adam about this (using POST instead and Content-type:
application/websocket), he said that since no clients use chunked
encoding of requests, it is likely to have similar problems to
GET+Upgrade in intermediaries.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Julian Reschke <[email-masked]>
Date: Thu, 02 Dec 2010 16:57:29 +0100

On 02.12.2010 16:51, John Tamplin wrote:
> ...
> ...

What do you mean by "no clients use chunked encoding in requests"?

I know for sure that chunked encoding in PUT and PROPPATCH is common in 
WebDAV.

Best regards, Julian

---

From: John Tamplin <[email-masked]>
Date: Thu, 2 Dec 2010 10:59:35 -0500

> not the best qualified to decide of the path importance!

Most websites host multiple applications, and differentiate them by
the path.  Having to have a single WebSocket server for all those
applications seems a non-starter to me, so you still have to send the
path somewhere.  CONNECT, as used by existing proxies, allows a full
URL in the CONNECT method so it can be included there.

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 2 Dec 2010 17:20:08 +0100

> URL in the CONNECT method so it can be included there.

Why is it a non-starter?  This is exactly how HTTP works - you have a
single protocol stack run by the server, with requests being farmed
out to one of the many applications deployed on the server.

Surely WS can work that way.

Surely WS when extended with MUX will have to work that way - you
could have multiple ws using widgets, frames, tabs or windows from the
one browser that could be accessing different applications on the same
server.  Ideally a TCP/IP connection could be shared by all these
client instances, but the logical connections over that TCP/IP would
be directed to different applications in the server.

If a server terminates a WS connection within the application, then a
MUX extension could mean that messages for appB are routed via appA -
that's a non starter for me as a failure of appA (or undeploy) will
break appB's connections

cheers

---

From: Brian <[email-masked]>
Date: Thu, 2 Dec 2010 16:07:31 -0800

It can be extremely valuable to have the path data available to a
reverse-proxy so that it can route the request to the actual server
that should receive the connection.  For example, I'm building an
interactive application where individual "rooms" are assigned to be
hosted on a particular websocket server for load balancing.  The
client is told which server id to connect to based on which room the
user wishes to join.

I'm using the Socket.IO library in a Node.JS-based server.  Socket.IO
can automatically handle fallbacks using other techniques over
standard HTTP, such as xhr-polling.  The exact same server routing
mechanism in HAProxy can handle every possible kind of connection.
Only when the request reaches the actual server does it decide whether
the request was websocket or standard HTTP -- HAProxy is none the
wiser.  It just turns into a bi-directional bit forwarder after the
first request.

The system uses a url format similar to
wss://www.mydomain.com/server2/websocket or
https://www.mydomain.com/server2/xhr-polling.  Both end up internally
routed by HAProxy to port 80 of server2.

I'd really rather not have to write my own WS-specific reverse-proxy
if I can avoid it when HAProxy is doing such a great job already.
Such a change would dramatically impact how my entire load-balancing
infrastructure and application architecture is built, just for lack of
a usable path.  It wouldn't be the end of the world, but it would be
really inconvenient and annoying to deal with such an omission.

This kind of routing is convenient to use with xhr because of the
same-origin policy, and it's nice that websocket can, at least
currently, fit into the same paradigm.

It could of course be useful for an HTTP reverse-proxy to be able to
route based on the host header and the path for other reasons as well,
such as delegating specific functionality to a particular server or
cluster of servers based on what is requested by the user.

Brian

On Thu, Dec 2, 2010 at 8:20 AM, Greg Wilkins <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 2 Dec 2010 16:28:54 -0800

On Dec 2, 2010, at 12:20 AM, Greg Wilkins wrote:
> sent from server to client.

Er, no, as I said below it -- after the client receives the header of
the 200 response containing the nonce, it can send anything it wants
on the connection provided that "anything" is prevented from looking
like an HTTP request.  There are many ways to prevent that.

>   However, I wonder how intermediaries

They would likely be stuck waiting for the client to complete its
first request, since the server does not need to respond earlier.
It just introduces another possible failure condition other than
the one we want to detect (WS support or not).

The point is that all the talk about "securing the handshake"
and ensuring that we are talking to a websocket server has
been conflated into a rather bizarre algorithm that breaks HTTP.

If the concern is that websocket requests on port 80 might be
intercepted as HTTP requests regardless of framing, then simply
making that impossible is the only solution (and the only thing
that Adam's paper does to solve that potential 0.0169% problem
is encryption of the data stream after the handshake is complete).
That's fine, but it has nothing to do with the handshake other
than the fact that the handshake should indicate how to do it
(assuming the how is allowed to vary in order to satisfy the
various use cases we already have).

If the concern is that the server might not speak websockets,
then including anything in the initial response that cannot
be mistaken as a non-websocket HTTP response is sufficient.
A single header field will suffice (please make it short).

Adam had another concern, which was that no user-provided data
be included in the clear for the initial request.  That is what
beget the whole invalid host thing.  There is no known use case
or vulnerability that justifies such a concern for the parts
of the request I gave above (i.e., the parts that came from a
URI are only sent over HTTP in URI-encoded form).

Finally, assuming we are no longer burdened by a non-HTTP
parsing algorithm (i.e., fixed headers, limited ordering,
and no extensions), then the above can also take advantage
of orthogonal HTTP concerns like authentication, identifying
the user agent, limiting the connection to a single hop, and
whatever else might be invented by someone in the future.

I should also note that this is assuming that we do not
want intermediaries to participate in websockets.  If we
wanted such a thing, then Upgrade would be the better solution
and we would need a uniform message-oriented set of semantics 
along the lines of HTTP but with bidirectionality (i.e., waka,
if I ever find time to publish it).

....Roy

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 02 Dec 2010 18:12:56 -0800

> is encryption of the data stream after the handshake is complete).

The threat model in Adam's paper assumes a malicious WebSocket server cooperating with the attacker. So the crypto is effective only if the crypto algorithm used can't be made to produce chosen ciphertext even if you know the key. The XOR masking in the compromise version of the proposal lacks this property - if you know the key (and with the WebSocket server at the other end telling you, you can), you just XOR your attack data with it.

I do not know if the earlier AES proposal has the property that it is hard to control the ciphertext even if you know the key. If so, that would be a reason to go back from XOR to AES.

That being said, empirically it seems that using CONNECT is also effective at preventing misinterpretation of WebSocket messages as HTTP request, even without any payload masking. That seems to be one of the key findings of the paper.

Regards,
Maciej

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 20:35:54 -0600

On Thu, Dec 2, 2010 at 8:12 PM, Maciej Stachowiak <[email-masked]> wrote:
> I do not know if the earlier AES proposal has the property that it is hard to control the ciphertext

That's not enough. Each message must be encrypted by a different
random key. With a new key for each new message, XOR probably is good
enough to prevent attacker from choosing cipher text.

>even if you know the key. If so, that would be a reason to go back from XOR to AES.
>

I also believe that's true. However I don't think the paper presented
convincing empirical evidence for that.

And even if it's true, the question is how big is the marginal
utility. Since minimally framed WS data is already very resistant to
HTTP parsers, CONNECT doesn't add much.

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 02 Dec 2010 19:23:53 -0800

On Dec 2, 2010, at 6:35 PM, Zhong Yu wrote:

> random key.

Each message encrypted with a different (browser-chosen) random key may protect against controlling the ciphertext, but do you know for a fact that streaming AES would not offer sufficient protection? I don't know enough about the details of the algorithm to know if it is easily reversible. I suppose you can always decrypt your ciphertext with the key, if any given octet sequence can be decrypted with any key.

> With a new key for each new message, XOR probably is good

There's two points to consider with this approach:

(1) The key would have to be included with each message sent, significantly increasing the overhead for small messages.

(2) The actual key used would need to still be partly based on info provided by the server, or else we reopen attacks using an HTTP client against a WebSocket server. You don't want an attacker with a browser-hosted client to be able to send valid frames without first having seen the server part of the handshake.

> HTTP parsers, CONNECT doesn't add much.

I think you are making assumptions here that are not backed up by the data in the paper, or any other data. They are simply your guesswork interpretations of the paper, and seemingly at odds with the interpretations of the authors.

(Perhaps Adam and his collaborators will release their code and/or raw data, and we won't have to wonder.)

Regards,
Maciej

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 2 Dec 2010 19:54:14 -0800

> The threat model in Adam's paper assumes a malicious WebSocket server cooperating with the attacker. So the crypto is effective only if the crypto algorithm used can't be made to produce chosen cip…

Right, XOR won't work (and is too expensive in any case).

The output of the websocket API is completely under the control
of the browser that implements the API.  Since the attack relies on
confusing intermediaries, it doesn't matter if the attacker receives
the data intact -- it only matters if the intermediaries (that know
nothing about websocket) can interpret the data by mistake.  Hence, it
should be sufficient to merely escape the LF characters in the data
(I don't know of any intercepts that treat CR as an EOL marker
during HTTP processing, though it too could be escaped).

> That being said, empirically it seems that using CONNECT is also effective at preventing misinterpretation of WebSocket messages as HTTP request, even without any payload masking. That seems to be…

Umm, no, that appears to be speculation in the paper.  The problem is
that they fix one problem by causing another one -- namely, sending
invalid CONNECT requests to a server is far more harmful than sending
valid GET, OPTIONS, or even new method requests.  The harm is caused
because CONNECT is an attack vector already, well known and guarded
against by servers.  They have no idea what it will cost folks on the
server end if any browser can be told by a third-party page to send CONNECT
to any server.  Furthermore, it serves no useful purpose given that all
of the attacks are neutralized by filtering the outgoing data.

....Roy

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 3 Dec 2010 11:57:38 +0800

On 2010-12-03 at 14:23:53, Maciej Stachowiak wrote:
> (1) The key would have to be included with each message sent,
> significantly increasing the overhead for small messages.

Not significant.  Depending on your personal definition of significant.  A few bits of entropy could be enough to reduce the chance that the attacker can guess the right combination.

> (2) The actual key used would need to still be partly based on info
> provided by the server, or else we reopen attacks using an HTTP client
> against a WebSocket server. You don't want an attacker with a browser-
> hosted client to be able to send valid frames without first having seen
> the server part of the handshake.

Take the old key, add the new bits, hash to get new mask.  Bootstrap the from the handshake.

I worry that you are heading into purely theoretical territory here though.  If the server is colluding with itself to generate a "ciphertext" that looks like HTTP, what are the chances that an intermediary is still looking for an HTTP request prologue?

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 3 Dec 2010 11:58:56 +0800

> during HTTP processing, though it too could be escaped).

Interesting idea.  You could just prohibit the sending of LF.  Or require that the most significant bit be set on every octet (no ASCII, no chance of it looking like HTTP).

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 2 Dec 2010 20:16:55 -0800

On Thu, Dec 2, 2010 at 7:58 PM, Thomson, Martin
<[email-masked]>wrote:

> Interesting idea.  You could just prohibit the sending of LF.  Or require
> that the most significant bit be set on every octet (no ASCII, no chance of
> it looking like HTTP).
>
>
I personally have no desire to get into escaping potentially large binary
blobs. Especially Maciej's earlier use case where he wanted to just
sendfile() something large [1], as now I'm essentially taking something and
making it variable length, and I have to figure out the length a-priori e.g.
count the number of characters to be escaped in my large blob before I can
start sending the fragment. I would much rather have a handshake that
ensures we are talking WebSockets, and then not rely on framing to save us.
Having to deal with xor is bad enough.

[1] http://www.ietf.org/mail-archive/web/hybi/current/msg02977.html

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 22:50:13 -0600

On Thu, Dec 2, 2010 at 9:23 PM, Maciej Stachowiak <[email-masked]> wrote:
> Each message encrypted with a different (browser-chosen) random key may protect against controlling the ciphertext, but do you know for a fact that streaming AES would not offer sufficient protection?

Positive. When plain text and cipher text are of same length, the
mapping must be one to one, encryption function and decryption
functions are reverse of each other. Attacker can choose cipher text C
by sending plain text P, where P=decrypt(C). We assume that the
attacker knows the decrypt function (and its key).
encrypt(P)=encrypt(decrypt(C))=C. The chosen cipher text C is now on
the wire.

> I think you are making assumptions here that are not backed up by the data in the paper, or any other data. They are simply your guesswork interpretations of the paper, and seemingly at odds with t…

I take offense of that. I actually read the paper and every number in
it. I have requested for missing details which are important to
understand what exactly were measured, and I have laid out my
arguments when I challenge the consistency of the data.

There are people who are making wishful statements and claim that the
paper presented empirical proof, without any citation. I comb through
the paper and cannot find relevant proof at all. I should start to
call them out, and they need to back up their claims.

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 23:14:13 -0600

> during HTTP processing, though it too could be escaped).

Java's BufferedReader.readLine() treat CR as EOL. And this method is
very popular, you'll find its usage in a lot of HTTP servers
implemented in Java.

I also agree with Ian that escaping is ugly and personally I would
even choose encryption over it.

In any case, whether we should de-http WS messages, and how should it
be done, is a quite isolated topic. I don't think we need to worry
about it now. Suffice to know that it can be added on anytime,
regardless how other parts of WS protocol are designed.

- Zhong Yu

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 00:21:27 -0500

> encrypt(P)=encrypt(decrypt(C))=C. The chosen cipher text C is now on

I don't believe that is true in general that encypt(decrypt(T)) =
decrypt(encypt(T)).  I don't know for certain with AES-CTR, since you
basically encrypt a counter and XOR it, and I was unable to find any
studies on the unforgeability (which is the property you are looking
for, the inability to produce plaintext that encrypts to a given
ciphertext given the key) of AES-CTR, but it was a requirement of the
AES selection process so I am pretty sure the core cipher does not
have the property you suggest.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 03 Dec 2010 14:36:29 +0900

Hello Ian,

> [1] http://www.ietf.org/mail-archive/web/hybi/current/msg02977.html

Xor would prevent sendfile, yes? Or not? What do you mean by "xor is bad 
enough"? What are the cases where it's bad for you, and how much would 
you ready to give up to avoid it?

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Zhong Yu <[email-masked]>
Date: Thu, 2 Dec 2010 23:42:12 -0600

> decrypt(encypt(T)).  I don't know for certain with AES-CTR, since you

It has to be true, as long as cipher texts are not longer than plain texts.

> basically encrypt a counter and XOR it, and I was unable to find any

We assume the attacker knows the encryption key and the counter value,
then he knows the masking bit stream.

To defeat the attacker, we must change the function(algo or key or
counter) as soon as the attacker knows the function (and its reverse).
That is, each message must use a different key/counter. The
key/counter is shipped in plain with the message to the receiver.

That increases the length of each encrypted message. Effectively, the
cipher text is now longer than the plain text. We no longer have
bijection functions.

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 2 Dec 2010 22:41:56 -0800

On Thu, Dec 2, 2010 at 9:36 PM, "Martin J. Dürst" <[email-masked]>wrote:

> Xor would prevent sendfile, yes? Or not?

yes

> What do you mean by "xor is bad enough"?

I mean I would much rather have a handshake that ensures we're talking with
a websocket server and not a confused intermediary, and then move on with
sending data back and forth unmolested, without requiring extra
computational cycles. Especially with a 20 byte key that in the best case
will get stored in multiple (3 on 64-bit systems) registers, but in reality
seems to introduce a more severe performance hit. A quick test with gcc
results in a >2x performance delta between using a 20-byte key and an 8-byte
key to xor 20mb of data 500 times over. (Granted, if I wrote it in assembly
I am pretty sure i could improve on this by using 3 registers and unrolling
the loop as opposed to a using a char[20] and a long long int for the 20/8
byte keys resp., but I doubt many people will do this.) I would rather not
pay any penalty at all though. Throughput is very important.

> What are the cases where it's bad for you, and how much would you ready to
> give up to avoid it?
>
> Regards,   Martin.
>
> --

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 07:52:41 +0100

Encrypting, XORing or escaping are all unnecessary

Framing should be sufficient to protect against this attack.  I have
described he framing defence as weak, but on reflection I don't think
it is.

We know at the very least that HTTP framing protects against this
attack, otherwise the attack would be possible using normal XHR and
sending a spoofed HTTP request as the message body.   I don't see
anybody saying that the framing defence of HTTP is weak and therefore
we must encrypt, XOR or escape all HTTP traffic.

So at the very least, we could wrap our WS streams in HTTP framing -
either as a HTTP frame per message, or one great big HTTP frame for
the entire stream, that would have intermediaries endlessly scanning
for the end of the request that never comes.

Ideally we can just come up with a dedicated WS framing that we test
against a large number of intermediaries to show that the WS frame is
sufficient to prevent intermediaries examining the payload.

regards

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 02:17:28 -0500

On Fri, Dec 3, 2010 at 1:52 AM, Greg Wilkins <[email-masked]> wrote:
> either as a HTTP frame per message,

In which case we have gained nothing over existing Comet techniques.

> or one great big HTTP frame for

If intermediaries actually implement client->server chunked encoding
and don't buffer chunks, then it might be viable to do that.

> sufficient to prevent intermediaries examining the payload.

It is impossible to prove that some intermediary won't fail
catastrophically by testing against a subset of them.  It is nicer to
prove that there is no possible way for an attack to influence the
intermediaries, though it may not be practical to do so.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 08:27:24 +0100

> intermediaries, though it may not be practical to do so.

It is impossible to prove that sending a CONNECT request before a
spoofed HTTP request is sufficient to prevent an intermediary "seeing"
the spoofed HTTP request.

Yet it is on the basis of a single empirical experiment with
undisclosed details that many have concluded that CONNECT is safe but
GET+Upgrade is not.

We can design framing that should be robust against HTTP parsing and
we can empirically test that design to give us confidence that it is
correct.   Adam has shown us that it is moderately easy to find
examples of vulnerable intermediaries, so we should be able to make a
similar experiments to test framing against those specific
intermediaries.

I'm working on my own experimental code to this effect... but I am
wondering about the ethics of deploying such experiments in
advertisements.  I'm not meaning to say that Adam acted unethically,
just that if his method becomes widespread then we could have all
sorts of experiments being conducted in our browsers without our
consent - or am I being naive here?? Probably it is already the case
that every browser is a hot bed of security experiments by both good
guys and bad guys.... let me go update my adblocker plugin :)

cheers

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 02:14:12 -0600

> I'm working on my own experimental code to this effect... but I am

That's great news. Would you publish the code?

> wondering about the ethics of deploying such experiments in

You open a socket to your own server, what's wrong with that? It's not
like you are causing havoc on intermediaries, at least not knowingly
and deliberately.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 10:36:08 +0100

> That's great news. Would you publish the code?

Here is the start that I've made

  https://github.com/gregw/probelet

Currently it is just an applet that makes a few HTTP requests back to
the server and then reports the results with another HTTP connection.
It has HTTP requests in there that may trigger a cache poisoning, but
their is nothing yet to check if the poisoning occurred. May need some
javascript magic or another applet to see if the cache has been
poisoned.

I'll try to find the time to update the tests to do a WS handshake
against the jetty server and to then send phantom HTTP requests within
valid WS frames and see if we can trigger cache poisoning.

Alternately, I might write a Jetty HTTP connector that will ignore WS
frames between HTTP requests.  We can then see if intermediaries will
pass on WS frames or not.

Feel free to contribute to this as I'm lacking in time.

cheers

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 01:55:55 -0800

> undisclosed details that many have concluded that CONNECT is safe but

Our paper provided fairly detailed descriptions of the experiments we
performed.
Certainly, they were in line with the descriptions I've seen in other
empirical
papers.

What details would you like disclosed?

> correct.

I don't see that this is at all obvious. Rather, it's a heuristic defense
that
is very sensitive to the behavior of the proxy with respect to the framing.

-Ekr

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 04:37:07 -0600

> Certainly, they were in line with the descriptions I've seen in other

It's unclear what exact bytes are sent in the POST/Upgrade/CONNECT
experiments. I hope this is not unreasonable to ask, our analysis of
what's going on depends on this very detail.

- Zhong Yu

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 12:27:41 +0100

> is very sensitive to the behavior of the proxy with respect to the framing.

As is the belief that a CONNECT method will prevent intermediaries
from wrongly interpreting phantom HTTP requests sent after the
CONNECT.

I'm just trying to get people to be even handed here.  You can't
reject GET+Upgrade because it is impossible to prove that framing
makes it safe, yet rely on a single empirical experiment failing to
find any intermediaries that are vulnerable to CONNECT as proof the
CONNECT is never vulnerable.

If we are going to rely on empirical "proof" then there is very good
"proof" that framing is sufficient, because there are no reported
cases of intermediaries being vulnerable to phantom HTTP requests sent
in the content of an XHR.    Thus it is obvious that there is some
type of framing at least that is sufficient because we have an
absolutely massive same size that indicates so.  However it is an open
question if WS framing can be made as good as HTTP framing and I think
it would be wrong to assume an answer either way.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 3 Dec 2010 12:39:35 +0100

Eric,

to put this another way, the paper produced the following results:

  POST  handshake  + non-framed-date   15 vulnerable
  GET-Upgrade + Non-framed-data             8 vulnerable
  CONNECT + non-framed data                   0 vulnerable

>From this you have concluded POST and GET-Upgrade are unsafe, but
CONNECT is safe.

I've suggested that the experiment be repeated with websocket framed
data and I expect that the results would then be

  POST  handshake  +framed data     0 vulnerable
  GET-Upgrade + framed-data            0 vulnerable
  CONNECT +  framed data                0 vulnerable

You are suggesting that we should not run this test, because the 0
vulnerabilities for GET-Upgrade+framed-data would not prove that it is
safe.

Correct!

but the zero discovered vulnerabilities for CONNECT also do not prove
that it is safe.

Note that I'm not arguing here against trying CONNECT - I'm just
saying lets not apply different standards of proof to the various
proposals.  If such proof is insufficient to accept GET+Upgrade, why
is it sufficient to accept for CONNECT?

regards

---

From: "Pat McManus @Mozilla" <[email-masked]>
Date: Fri, 03 Dec 2010 08:52:25 -0500

> described he framing defence as weak, but on reflection I don't think

I believe it is very weak.

> sending a spoofed HTTP request as the message body.

Expecting an HTTP stack to be protected by HTTP message length framing
is reasonable and does indeed work as far as I can tell. Expecting an
HTTP stack to be protected by Websocket framing placed between the HTTP
messages is not reasonable.

The HTTP standard itself adds confusion on this front. CRLFs in between
requests are clearly not allowed by the BNF in the document, but it also
says that in the name of robustness a server SHOULD ignore CRLFs when
expecting a request line. 

It is not much of a leap of faith to see a proxy implementor, seeking
maximum interoperability and robustness and (mis?) guided by 'be liberal
in what you receive', ignoring things other than CRLF when looking for a
request line. The standard has already made an exception to the BNF for
robustness and she is making this change for reasons of robustness,
right? 

Firefox, for better or for worse, has done exactly this kind of thing
under some circumstances when looking for response headers. When looking
for the next response it will scan for "HTTP" tossing away anything else
along the way. I expect to see similar things in other implementations
on both the request and response side from time to time.

We can all have opinions on whether or not such a thing is apropos but
the more important issue is whether or not it is plausible to exist in
the field. I certainly think it is.

For that reason a handshake that isn't based on compliance with the
heretofore irrelevant Upgrade header is in order.

-- 
http://www.getfirefox.com/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 03 Dec 2010 18:29:23 +0100

>have the property you suggest.

Well, going from draft-abarth-websocket-handshake-00 'The client-to-
server key is the intial 16 bytes of the HMAC-SHA1 of the UTF-8 string
"363A6078-74D2-4C0B-8CBC-1E6A36E83442" keyed with the concatenation of
the client-nonce and the server-nonce. ... All subsequent bytes sent
from the user agent to the server are encrypted using AES-128-CTR keyed
with the client-to-server key.'

That would be (ignoring some details like making sure the server nonce
is guranteed to be different from the client nonce):

  #!perl -w
  use strict;
  use warnings;
  use Digest::SHA qw/hmac_sha1/;
  use Crypt::Rijndael;
  
  my $sv = "363A6078-74D2-4C0B-8CBC-1E6A36E83442";
  my $k1 = join '', map { pack('v', int(rand(2**16))) } 0..7;
  my $k2 = join '', map { pack('v', int(rand(2**16))) } 0..7;;

  my $client_to_server_key =
    substr hmac_sha1($sv, $k1 . $k2), 0, 16;
  
  my $crypt = Crypt::Rijndael->new( $client_to_server_key,
    Crypt::Rijndael::MODE_CTR() );

  my $plaintext = $crypt->encrypt("0123456789abcdef");

  print $crypt->encrypt($plaintext);

As you say, you simply XOR with the encrypted counter value; the
attacker just has to XOR the data he wants to send with the encrypted
counter value, the browser then XORs with the encrypted counter value
a second time, which restores the data the attacker wants to send.
And the attacker knows the encrypted counter value if he knows the
keys, which he does if he controls the client-side script and the
server-side component the client performs the handshake with. So the
code above prints "0123456789abcdef", the ciphertext chosen by the
attacker.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 09:48:56 -0800

> From this you have concluded POST and GET-Upgrade are unsafe, but

I wouldn't say we concluded that. Rather, I would say it's the best we know
how
to do and that the alternatives are demonstrably flawed.

> vulnerabilities for GET-Upgrade+framed-data would not prove that it is

I don't recall saying that we shouldn't run this test. If I did, I apologize
since I
didn't mean to. Rather, I would say that I think that the issue of whether
framing
is really being successful against attack is hard to be concrete about,
since the
exact behavior of the proxies is complicated and it might take a lot of work
to
figure out how to resynchronize any particular proxy. By contrast, the
behaviors we're measuring in our research (routing on the host header and
caching) are relatively concrete.

> but the zero discovered vulnerabilities for CONNECT also do not prove

Nor did I say otherwise. I said it was the best we knew how to do in terms
of the
handshake and that if we go with a handshake that is demonstrably weak, then
we
are relying primarily on the security of the framing, which I'm not that
confident
in, particularly if we don't mask.

-Ekr

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 13:02:25 -0500

On Fri, Dec 3, 2010 at 12:48 PM, Eric Rescorla <[email-masked]> wrote:

> handshake and that if we go with a handshake that is demonstrably weak,
> then we

I think some of the discussions here have shown that masking that is
effective against an attacker controlling both the client payload and the
WebSocket server (ie, preventing the attack from sending data that encrypts
to a chosen ciphertext on the wire that might confused intermediaries) is at
best more complicated and expensive than either the XOR or the AES-CTR
schemes that have been proposed.

If we get much more complicated than those schemes, I am not sure what the
benefit is over just encrypting everything for real instead of just using it
for masking.  When that was brought up before, there seemed to be
significant objections.

So, I am not currently convinced that masking the payload data is a viable
option.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 3 Dec 2010 10:10:13 -0800

On Thu, Dec 2, 2010 at 6:12 PM, Maciej Stachowiak <[email-masked]> wrote:

> The threat model in Adam's paper assumes a malicious WebSocket server
> cooperating with the attacker. So the crypto is effective only if the crypto
> algorithm used can't be made to produce chosen ciphertext even if you know
> the key. The XOR masking in the compromise version of the proposal lacks
> this property - if you know the key (and with the WebSocket server at the
> other end telling you, you can), you just XOR your attack data with it.
>

That is true IFF the xor mask the server receives is the same as the one the
client will next use. If you ensure that isn't the case (e.g. by varying the
xor mask per write), then the client has no way to produce the bytes it
wants on the wire for *certain.

*depending on the size of the xor mask, it is possible to generate a request
that goes through all variations of the xor mask and thus produce at least
one copy of what you want.

>
> I do not know if the earlier AES proposal has the property that it is hard
> to control the ciphertext even if you know the key. If so, that would be a
> reason to go back from XOR to AES.
>

AES, etc are definitely more difficult to predict and thus come up with data
that will provide for the bits on the wire an attacker would want. AES and
other similar algorithms are, however, significantly more expensive in all
ways (computation, complexity, storage).

-=R

>
> That being said, empirically it seems that using CONNECT is also effective
> at preventing misinterpretation of WebSocket messages as HTTP request, even
> without any payload masking. That seems to be one of the key findings of the
> paper.
>
> Regards,
> Maciej
>

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 13:15:54 -0500

On Fri, Dec 3, 2010 at 1:10 PM, Roberto Peon <[email-masked]> wrote:

> That is true IFF the xor mask the server receives is the same as the one
> the client will next use. If you ensure that isn't the case (e.g. by varying
> the xor mask per write), then the client has no way to produce the bytes it
> *depending on the size of the xor mask, it is possible to generate a
> request that goes through all variations of the xor mask and thus produce at
> least one copy of what you want.
>

Right, but if we start sending a per-frame mask value and make it large
enough to prevent the attacker from including all variations of their
message, then what have we gained over just using TLS?  We spent a lot of
cycles and bytes in each frame and still the payload data is vulnerable to
snooping - we might as well just encrypt everything and be done with it.

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 10:16:38 -0800

> schemes that have been proposed.

I'm not actually sure that's correct.

I believe that the AES-CTR scheme described in our paper (random
per-connection
AES key; random per-write() nonce) is secure against an attacker who
controls
both sides. Or do you just mean those in drafts?

It's an open question whether there is a simpler scheme.

-Ekr

---

From: Roberto Peon <[email-masked]>
Date: Fri, 3 Dec 2010 10:22:15 -0800

> snooping - we might as well just encrypt everything and be done with it.

If you make the frames smaller (fragmentation) and/or vary the xor mask on a
known byte amount it becomes more difficult.

If we're happy with encryption, why not. But that would definitely slow down
sendfile(). I don't think we care about snooping here. If we did we'd just
put the whole thing into the TLS context. If we're running  on a separate
port, we could certainly do that. It does have significant problems with
deployment at schools, etc. So again, I believe that we don't care about
snooping-- merely preventing an attacker from doing bad things to other
people.
-=R

---

From: Maciej Stachowiak <[email-masked]>
Date: Fri, 03 Dec 2010 10:53:19 -0800

On Dec 3, 2010, at 10:15 AM, John Tamplin wrote:

> That is true IFF the xor mask the server receives is the same as the one the client will next use. If you ensure that isn't the case (e.g. by varying the xor mask per write), then the client has no way to produce the bytes it wants on the wire for *certain.
> 
> *depending on the size of the xor mask, it is possible to generate a request that goes through all variations of the xor mask and thus produce at least one copy of what you want.
> 
> Right, but if we start sending a per-frame mask value and make it large enough to prevent the attacker from including all variations of their message, then what have we gained over just using TLS?  We spent a lot of cycles and bytes in each frame and still the payload data is vulnerable to snooping - we might as well just encrypt everything and be done with it.

It should make us start thinking seriously about the TLS-only option.

However, there would be some advantages over TLS-only:

- Doesn't suffer performance/memory problems of some existing TLS implementations.
- Doesn't require getting a cert to play (or else us designing an approach that can work without an assigned cert, without compromising real TLS.
- Probably still less overhead per minimum size message.
- Potentially allows filtering/monitoring by WebSocket-aware intermediaries.

Regards,
Maciej

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 14:00:02 -0500

On Fri, Dec 3, 2010 at 1:53 PM, Maciej Stachowiak <[email-masked]> wrote:

> - Doesn't require getting a cert to play (or else us designing an approach
> that can work without an assigned cert, without compromising real TLS.
>

I would think ws would mean the cert's credentials aren't checked (so it
gives no protection again MITM attacks, but still protects against passive
snooping) while wss would mean that they are checked.

> - Probably still less overhead per minimum size message.

True, I am sure we could get less than 20 bytes (IIRC, the minimum overhead
for TLS records).  However, there is a benefit to using existing
infrastructure rather than reinventing the wheel if we are going to do
encryption anyway.

> - Potentially allows filtering/monitoring by WebSocket-aware

I don't know how to allow that without allowing attackers to eavesdrop as
well.  My feeling is that over time, more and more services are going to be
available only over encrypted channels.

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 13:38:11 -0600

TLS won't help us here. With TLS, attacker can still arrange chosen
cipher texts on the wire.

Remember the threat model we describe here are not typical in
cryptology discussions. We say the attacker sits at both ends of
communication. There is no such threat model in cryptology.

We are discussing this overblown situation, because of the fear that
simple framing bytes are not sufficient to convince any http parsers
that this is not a http stream. This fear is yet to be justified by
data.

- Zhong Yu

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 11:43:45 -0800

> cipher texts on the wire.

That's not necessarily correct. For instance with CBC mode the attacker
has no control over the IV and hence no control over the plaintext.

> that this is not a http stream. This fear is yet to be justified by

Nor has the claim that they are sufficient. I generally prefer to err on the
side
of caution when definitive data is not available.

-Ekr

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 13:45:45 -0600

>  GET-Upgrade + Non-framed-data             8 vulnerable

I know Adam said that data isn't framed in the Upgrade experiment. I
don't buy it. If this is what they sent in the Upgrade experiment:

GET /resource HTTP/1.1
Upgrade: WebSocket
...
<CRLF>
GET /script.php/<random> HTTP/1.1
Host: target.com

i.e. there is no non-http bytes between handshake and spoof requests,
then this is very similar to the POST experiment. The only major
difference is the Upgrade header.

Yet in POST experiment, 1376 firewall circumvention attacks succeeded,
and in Upgrade experiment, only 1 succeeded. The two experiments are
done at the same time, over the same intermediaries, same parsers. How
can this be?

- Zhong

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 13:52:30 -0600

> has no control over the IV and hence no control over the plaintext.

IV must be sent to the receiver, who is the attacker. If IV is reused
for next message, attacker can choose cipher text for the next
message.

If IV changes for each message, it will work, exactly like what we
discussed before.

---

From: John Tamplin <[email-masked]>
Date: Fri, 3 Dec 2010 14:59:58 -0500

> communication. There is no such threat model in cryptology.

That isn't quite accurate -- here the attacker has total control over the
bytes on the wire from the server, but only control over payload bytes (and
perhaps some influence over non-payload bytes) on the client.  Ie, they
didn't get to install a different browser, so things the browser enforces
they have no control over (such as using a new masking key for each frame,
by the time the server gets it, it is too late for the attacker to use it,
though if we ever expose a streaming API to JS that wouldn't be the case).

> that this is not a http stream. This fear is yet to be justified by

As has been mentioned, we already know there are HTTP parsers out there that
ignore some level of garbage bytes between HTTP requests on the same
connection, so it does not seem overblown to be concerned about their
behavior in the presence of attacker-controlled data after a valid HTTP
request.

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 12:09:34 -0800

> If IV changes for each message, it will work, exactly like what we

Yes, this is how TLS 1.1 works.

-Ekr

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 14:12:47 -0600

> though if we ever expose a streaming API to JS that wouldn't be the case).

correct. browser must chose a new key for each message. encrypted
payload must be larger than plain payload, there's no way around it.

> As has been mentioned, we already know there are HTTP parsers out there that

The question is how many. I speculate it is very small. Majority of
http components *need* to know the request line, and the request
method. Framing bytes pollute the method, rending these components
unable to make sense of the stream.

If a component doesn't care about method, it has to forward the whole
thing to the next hop, and eventually somebody must try to extract the
method, and fail. In Adam's paper, there seems to be one case this
happens in Upgrade/firewall-circumvention experiment, when the
interceptor only checks for the "Host:" line and forwards all the
bytes blindly to the server. This is counted as one successful attack.
However, the apache server cannot make sense of these bytes, and it
most likely returned 400 Bad Request, it shouldn't have been counted
as success.

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 14:24:58 -0600

> Yes, this is how TLS 1.1 works.

only if we put each WS message in a separate record?

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 12:33:56 -0800

> only if we put each WS message in a separate record?

No. Each TLS record is encrypted with a separate IV (again, in CBC mode) so
it doesn't matter
how the writes are presented to the TLS layer.

-Ekr

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 04 Dec 2010 00:15:37 +0100

>both sides. Or do you just mean those in drafts?

The proposal in the paper suggests "preventing the browser application
from directly controlling any of the bytes on the wire by encrypting the
traffic using keys which are established as part of the handshake." and
Adam's announcement said "We recommend that the working group adopt the
CONNECT-based handshake described in draft-abarth-websocket-handshake"
and both versions of draft-abarth-websocket-handshake do not have a per-
message nonce. So at this point nobody seems to be proposing that.

If we did do this then any attack that depends on an attacker making the
browser send something over an established Websocket connection becomes
impossible independently of how the connection was established, since he
can only make it send data of the browser's choosing.

(There is an error btw in draft-abarth-websocket-handshake-01, the keys
are truncated to 16 byte but then it's assumed they are 20 bytes long.)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Zhong Yu <[email-masked]>
Date: Fri, 3 Dec 2010 17:27:46 -0600

> can only make it send data of the browser's choosing.

Agreed. By completely obfuscating payload, CONNECT becomes irrelevant.

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 3 Dec 2010 16:39:23 -0800

> message nonce. So at this point nobody seems to be proposing that.

Well, I'm not sure it's productive to debate what "proposed" means.

I would prefer that the WG adopt something more like what's in the paper.
However, we recognize that for some of the reasons indicated by people
on these threads that it encrypting the payload as described in our paper
may be too much to accept, so I'm probably prepared to compromise on
that point.

> can only make it send data of the browser's choosing.

Well, that's not entirely true. Bandwidth consumption, etc. attacks still
remain possible. And if we're not doing that, as I understand people prefer,
then the security of the handshake remains important.

-Ekr

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 04 Dec 2010 02:01:33 +0100

>may be too much to accept, so I'm probably prepared to compromise on

Right. My purpose was to make sure people understand that your paper
discusses two types of encryption, per-frame and per-connection, and
the recommendation in the paper and in the announcement to the list is
only for the per-connection encryption. So simply following that re-
commendation would not protect against the "chosen ciphertext" attack,
even though the paper discusses how to do that.

>then the security of the handshake remains important.

I don't know what the et cetera are, but I would expect browsers to use
the same measures to prevent abusive use of bandwidth for Websockets
that they also use for all the other methods to utilize bandwidth by
possibly malicious scripts, so that's not something we need to consider
here, besides briefly mentioning it in the Security Considerations.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 4 Dec 2010 08:52:21 +0100

> done at the same time, over the same intermediaries, same parsers. How

Zong,

well it's hard to know for sure unless Adam & Eric disclose.

I did ask if they framed the second request and Adam said no.
It's true that it is not clear if the 8 random bytes are sent between
the requests.

It is certainly true that they did not send a hello frame with known
unacceptable bytes.

Is it true that the experiment was conducted over the same intermediaries etc?
If the experiments were conducted by ad placements, then it would be
quasi random where the adverts were placed.  Actually it may be worse
than that, as the advert system may track data about the advertisement
and alter it's algorithm for where to display the advert over time,
target pages and client source.   There could be large systematic
variables in the experiment.

I really don't want to rain on Adam & Erics parade, as I think their
work is really interesting and well worth doing.  It is just that
without full disclosure of methods and data, I think it is impossible
to understand their findings or fully support their conclusions.

cheers

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 4 Dec 2010 06:07:08 -0800

> well it's hard to know for sure unless Adam & Eric disclose.

Greg,

> Is it true that the experiment was conducted over the same intermediaries

The Flash and Java experiments were run in the same browsers, though even
then there is a fair amount of variation in which browsers you actually get
results from. This is all described in Section III(B)(2).

I'll have to check with Adam, but I believe the other handshakes were run
on the same browsers as well, because the number of impressions is
identical (54,534). However, as with the previous experiment, you don't
get the exact same of successful (i.e., completed execution) tests.

> variables in the experiment.

It's not clear to me why you think this would create a systematic error.
It's not like the ad network knows the results of the campaign; it just
gives you impressions.

> to understand their findings or fully support their conclusions.

As I said earlier when you said that the paper didn't have details, I
believe the
level of detail in this paper is consistent with the level of detail in
other experimental
papers and describes the steps necessary to replicate the experiment. But
of course like almost all papers this was written at the last minute and so
we
might have missed something.

What details is it you would like to know about that you think aren't
in the paper?

-Ekr

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 4 Dec 2010 16:46:41 +0100

> It's not like the ad network knows the results of the campaign; it just

I don't know what ad service you used, but many will attempt to place
an ad on a relevant page. So if your ad was for something technical,
it may get place on pages that are more likely to be viewed by firefox
behind a company firewall. If it was for flower arrangements, it would
more likely be displayed on IE behind home routers.   More
importantly, so add services will "learn" as they go and change where
they place an add, so some experiments run after others may see
different environments.    Eitherway - it's just an idea that might
explain some variation in the absolute numbers - but would not detract
significantly from the findings of the paper.

> What details is it you would like to know about that you think aren't

I'd like to see the actual bytes sent on the wire by the client and
server.   Specifically, I'd like to know if you sent the -76 random
bytes after the GET+Upgrade request.  If possible, it would be good to
know the value of the random bytes sent for the runs that succeeded in
poisoning the cache and for the runs where POST poisoned the cache,
but GET+upgrade did not.

cheers

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 4 Dec 2010 12:20:24 -0800

> poisoning the cache and for the runs where POST poisoned the cache,

There were no random bytes.

The message was essentially as described in III(B)(1)

-Ekr

---

From: Zhong Yu <[email-masked]>
Date: Sat, 4 Dec 2010 14:50:19 -0600

> The message was essentially as described in III(B)(1)

So in the POST experiment, the bytes are

    POST /path/of/attackers/choice HTTP/1.1
    Host: host-of-attackers-choice.com
    Sec-WebSocket-Key: <connection-key>

In 1376 cases the 2nd request was routed to target.com, presumably
because some interceptors parsed it as an HTTP request, and routed it
based on Host.

In the Upgrade experiment, the bytes are

    GET /path/of/attackers/choice HTTP/1.1
    Host: host-of-attackers-choice.com
    Connection: Upgrade
    Sec-WebSocket-Key: <connection-key>
    Upgrade: WebSocket

In only 1 case the 2nd request was routed to target.com. This
experiment is apparently done in the same ad display as the POST
experiment, and the bytes passed over same intermediaries.

Don't you find that odd? How do you explain the difference?

- Zhong

---

From: Zhong Yu <[email-masked]>
Date: Sat, 4 Dec 2010 15:39:47 -0600

>From paper: "47,741 (96:9%) reported that no intermediaries were
confused when sending the spoofed HTTP request"

What's the expected behavior here? The 2nd request is transmitted to
host-of-attackers-choice.com verbatim, right?

"There were 97 of 49;218 impressions (0:2%) where the spoofed request
was routed by IP"

The 2nd request is also transmitted to host-of-attackers-choice.com
verbatim, right? How does this differ from the 1st case?

- Zhong

---

From: Maciej Stachowiak <[email-masked]>
Date: Sat, 04 Dec 2010 14:29:40 -0800

On Dec 4, 2010, at 6:07 AM, Eric Rescorla wrote:

> As I said earlier when you said that the paper didn't have details, I believe the
> level of detail in this paper is consistent with the level of detail in other experimental
> of course like almost all papers this was written at the last minute and so we

Seems like the following additional pieces of data could be useful, even though they are apparently more than is expected of experimental infosec papers:

(1) Enough detail to determine the exact bytes exchanged on the wire for each handshake tested.
(2) Source code for the program used to do the experiment (makes #1 to some extent redundant).
(3) Raw data dumps of the network exchanges that were classified as successful or unsuccessful attacks.
(4) Statistical analysis, e.g. are various differences between the handshake statistically significant given the size of your data set? (I would find a Bayesian statistical analysis even more informative but that is more than you can expect of any scientific paper.)

Although I find the results of the paper believable, the above info would both aid in interpreting the results, and would provide enough info to check the methodology or replicate the experiment for those who have doubts. I think it would be pretty hard to replicate based on just what is in the paper. I just re-read it, and I can't find enough detail on what was actually exchanged in the handshake experiments to attempt to reproduce the results.

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 5 Dec 2010 09:46:36 +0100

Zhong,

when you say: "the 2nd request was routed to target.com, presumably
> based on Host.",   I don't think that is the attack vector we are worried about.

I'm sure some of the intermediaries did route to target.com, but that
is fine as their cache would then be filled with content from
target.com

The attach vector is intermediaries that don't actually route the
request, but use the connection they already have  - so the request is
sent to the attackers server and the response is generated by the
attacker.  The dumb thing the intermediaries then do is use the host
header as part of their cache key, so the content from the attacker
server is placed into the cache in a way that makes it available to
new normal connections to target.com

Note that intermediaries that are scanning for HTTP content will
probably have to consider the Connection header, so adding it does not
surprise me that it changes this experiment.    What it does show is
that these intermediaries can be very sensitive to small changes in
the bytes sent - which can both be good (when trying to trip them up)
and bad (when trying to "proved" you'll always trip them up).

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 5 Dec 2010 09:51:55 +0100

On 4 December 2010 23:29, Maciej Stachowiak <[email-masked]> wrote:
> Seems like the following additional pieces of data could be useful, even
> though they are apparently more than is expected of experimental infosec
> papers:

Guys - I hope nobody thinks that by asking for more information I've
been implying that the paper should have contained it or that
Adam/Eric are under any compulsion (moral or otherwise) to disclose.

This is an open list and perhaps not the best forum to disclose full
details of cache poisoning attacks, so restraint and consideration of
disclosure is entirely understandable.

cheers

---

From: Maciej Stachowiak <[email-masked]>
Date: Sun, 05 Dec 2010 02:04:22 -0800

> disclosure is entirely understandable.

The paper has all the info that black hats would need to exploit Flash sockets or Java sockets already (I presume Adam and Eric contacted the affected vendors). I believe any missing info is not to avoid disclosing an attack (the paper already does that). I think it is missing because, as Adam and Eric mentioned, the norm in this particular academic field does not require such a level of detail. I don't believe there is a security reason to withhold this info, or at least, the authors have not said so.

However, the peper as it stands doesn't have enough info (afaict) to reproduce the handshake results. And it doesn't seem to have enough detail to evaluate exactly what was done, in ways that are relevant to the handshake choices. I think it would be better science to include the info I suggested, as well as more helpful to the WG.

I do not personally doubt the results of the paper, but I don't feel like I am in a good position to support it without knowing exactly what the experiment was.

Regards,
Maciej

---

From: Zhong Yu <[email-masked]>
Date: Sun, 5 Dec 2010 13:54:28 -0600

> is fine as their cache would then be filled with content from

Greg, first we need to understand the data better before we can depend
on them. I'm arguing that there are glaring inconsistencies in the
data that beg explanation.

--
The question of what "interceptors" can legally do is in itself quite
interesting. If the connection is known to be a HTTP connection, then
both "routing based on Host" and "caching based on Host" are *not
guilty* of violating HTTP spec.

Since it's so easy for an outsiders to initiate a socket from inside,
interceptors cannot assume any connection to be a HTTP connection. If
an interceptor should allow arbitrary TCP connections, then it shall
not temper with the connection and the data flow in any way, no matter
how much the data flow look like HTTP. If an interceptor should only
allow HTTP connections, it must examine the connection extremely
carefully, abort the connection as soon as the data does not seem to
come from a typical browser. Host-IP mismatch is a such a sign.
--

- Zhong Yu

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Tue, 7 Dec 2010 00:19:30 +0000

I would like to second Roy's observation that redefining HTTP semantics is a non-starter. The main goal should be to settle on a handshake that works and has a high probability of surviving the IETF process which basically means that it has to follow HTTP semantics. I haven't seen any persuading arguments that would justify this WG to go beyond its charter and attempt to change HTTP directly. Traditionally, arguments citing interop problems with particular existing implementations is not sufficient for a spec to change, especially not when it is as widespread as HTTP. I don't think this in any way precludes the WG from coming up with a reasonable, lightweight yet robust handshake so I would encourage the discussion to continue but also to constrain itself to the boundaries put forward.

Thanks,

Henrik

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roy T. Fielding
Sent: Wednesday, December 01, 2010 11:19 PM
To: Greg Wilkins
Cc: Hybi
Subject: Re: [hybi] Different server semantics of CONNECT

> and then eventually the browser just directly making the connection to 

I don't think you understood my objection.  CONNECT is defined as a method that is only applicable to proxies.  WS cannot change that definition.
The request target "server.example.com:80" above is only valid in HTTP/1.1 when sent to a proxy.  WS cannot change that definition either.  Both are way outside the scope of hybi.  Is that clear?

Sending non-valid HTTP over the Internet's designated web port (80) causes unanticipated results because there are a whole lot of implementations out there that do weird, amazing, and occasionally stupid things to the traffic on port 80.  Therefore, a protocol in an IETF proposed standard that is expected to be sent over port 80 had better start off as HTTP-compliant (to allow compliant rejections).
Anything else will result in an arms race with the folks who need to block entropy-generating apps.

What annoys me is that there is no reason to use CONNECT for this use case.  Just define a new method, such as

(the request might include auth headers as well) and then wait for a response of

   HTTP/1.1 200 OK
   WS: nonce

   88

before the client begins sending its stream of non-HTTP socket traffic.  To satisfy Adam's issues, that traffic can be opaqued by the sending client via LF-escaping or encryption -- we could even choose negotiate the kind of opaquing in the initial exchange.
Broken interception proxies may still fail to send the subsequent data, just like they would for the CONNECT case, but they can't be tricked into cache poisoning or host-based misdirection.

Note that this only applies to the fallback case of using HTTP when a separate port doesn't work and no normal proxy has been configured.  The "fast" websockets protocol could then use TCP, SCTP, UDP, TLS, or whatever else one might want to specify, independent of HTTP's constraints.

Every successfully deployed proprietary equivalent to websockets has faced this same problem with firewall port blocking and eventually adopted the same kind of fallback solution (e.g., Skype, Flash, etc.).

....Roy

---

From: John Tamplin <[email-masked]>
Date: Mon, 6 Dec 2010 19:47:56 -0500

On Mon, Dec 6, 2010 at 7:19 PM, Henrik Frystyk Nielsen
<[email-masked]> wrote:
> I would like to second Roy's observation that redefining HTTP semantics is a non-starter.
> The main goal should be to settle on a handshake that works and has a high probability of
> surviving the IETF process which basically means that it has to follow HTTP semantics. I
> haven't seen any persuading arguments that would justify this WG to go beyond its
> charter and attempt to change HTTP directly. Traditionally, arguments citing interop
> problems with particular existing implementations is not sufficient for a spec to change,
> especially not when it is as widespread as HTTP. I don't think this in any way precludes the
> WG from coming up with a reasonable, lightweight yet robust handshake so I would
> encourage the discussion to continue but also to constrain itself to the boundaries put
> forward.

First, I am not sure that the CONNECT method is well-enough specified
that anything could be considered changing it.  RFC2616 only has this
to say:

> 9.9 CONNECT
>
> This specification reserves the method name CONNECT for use with a proxy that can
> dynamically switch to being a tunnel (e.g. SSL tunneling [44]).

It doesn't even talk at all about what is acceptable in the request
line, headers, responses, etc.  The only draft which ever made any
attempt to specify it (as far as I am aware) -
http://tools.ietf.org/id/draft-luotonen-web-proxy-tunneling-01.txt -
expired in 1999 (!), so it seems to be stretching the facts to call
that an active spec.

Second, I think it is arguing over semantics to say that the proposed
use of CONNECT does not fit this definition.  The 14 mentions of
"proxy" in the HTTP spec don't really shed any light on whether an
HTTP server acting as a gateway to WebSocket applications could be
considered a proxy, but certainly seems reasonable to consider it so.
Even if not, I would argue that refining the terminology in 9.9 to
clarify "proxy" should not be considered changing the HTTP spec, when
it is so woefully underspecified at present.  Clearly, the intent of
WebSocket is to setup a message-oriented tunnel between the client and
the server, so it seems that part fits.

It seems that those so fundamentally opposed to using CONNECT are
setting up a no-win situation.  If we can't use GET+Upgrade because
real-world implementations never bothered to implement it because it
wasn't actually used, and we can't use CONNECT because it is deemed to
be in violation of the HTTP spec, then where does that leave us?  We
can use a dedicated port, in which case WebSockets will not be usable
in the near future for anyone behind a configured proxy (ie, most
corporate environments and some home ones), or we can use some other
hack which has its own problems like POST with chunked encoding in
both directions.

Do you just want to kill WebSockets so we can stick with Comet/etc?
Is that any less abusive of existing HTTP infrastructure?

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 07 Dec 2010 01:58:10 +0100

>expired in 1999 (!), so it seems to be stretching the facts to call

RFC 2616 is updated by the proposed standard RFC 2817 which defines the
method (imperfectly, but nevertheless). If the draft you mention was the
only specification for it, that would be all the more reason to be very
sceptical about relying on this method for assumed security properties.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 7 Dec 2010 08:56:02 +0100

On 7 December 2010 01:47, John Tamplin <[email-masked]> wrote:
> corporate environments and some home ones), ....

John,

I do mostly agree with you that definition of acceptable use of
CONNECT and the definition of "proxy" is being a little strictly
applied.

However, I don't agree being pushed to use a dedicated port will make
websocket less usable.    Consider the following experiment from the
Chrome team:

  http://www.ietf.org/mail-archive/web/tls/current/msg05593.html

whose results (from a small self selected sample) were:

  "The experiment tries to perform a trival WebSockets transaction over
   HTTP, over HTTP on a random, high numbered port (61985) and over
   HTTPS. This experiment is run via Chrome for users who have opted in.
   These are normal requests in the Chrome network stack and so will use
   any configured proxies.

   Excluding users that couldn't even do basic a HTTP transaction over
   the transport layer (6.37%, 10.15% and 7.91%, respectively), the
   success rates are:
       HTTP (port 80)      67%
       HTTP (port 61985)   86%
       HTTPS (port 443)    95%

   This results in overall sucess rates of 63%, 77% and 87%, respectively."

Thus a dedicated high port is going to have a success rate of 70-80%
I don't think that either CONNECT or GET+Upgrade handshakes are
claiming any significantly higher success rates.

Let's say we got permission to go crazy with the cheeze whiz and
change HTTP in any way that we liked, we are still looking at > 10%
that cannot establish a websocket connection over port 80.   Even port
443 is far from perfect.

All our proposed solutions suffer from connectivity issues - and I
don't think that it is the fault of strict application of the HTTP
specification.

We are beating ourselves up here saying: "your solutions is not
perfect" - "no your solution is not perfect" etc.   If we grind this
out to some kind of rough consensus on either connect or upgrade, we
are still going to have something that is not provably secure and will
fail in >10-20% of cases.

A dedicated port is going to give us
  + the same order of magnitude of success.
  + a much clearer security story.
  + a simpler argument to intermediary venders to support WS
  + the option of using a CONNECT handshake on port 80 to connect to it.

cheers