Subject: [hybi] CML

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 23 Aug 2010 11:27:37 +0800

CML, which I presume signifies "complete message length", has appeared in some of the later framing proposals.  Can anyone provide me with a succinct summary of the reasons for its inclusion?

---

From: John Tamplin <[email-masked]>
Date: Sun, 22 Aug 2010 23:32:19 -0400

On Sun, Aug 22, 2010 at 11:27 PM, Thomson, Martin <[email-masked]
> wrote:

> CML, which I presume signifies "complete message length", has appeared in
> some of the later framing proposals.  Can anyone provide me with a succinct
> summary of the reasons for its inclusion?
>

There were objections to fragmentation without it, as the receiver would
then have to realloc/copy the buffer it will hand off the client code
without it.  Since fragmentation support is a requirement, even if a sender
writes a message in one large frame, an intermediary might choose to break
it up into multiple frames.  With CML, the  fragmenter can still indicate to
the ultimate receiver the size of the original message.

It is optional so that it is only included when it is useful and available,
and may be an estimate that is updated (which facilitates a UI for reporting
progress).

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 23 Aug 2010 12:00:31 +0800

> There were objections to fragmentation without it... 

Where do you expect to see CML included?  The first frame, every frame, the last frame, or some other pattern?

Presumably, much of the benefit is lost if the complete length is only reported on the last frame - that information could be trivially gained by doing a little arithmetic (and I hear that computers are good at that ;).  That leads me to conclude that it’s most useful on the first frame.  Or at least as soon as possible.

If you look at chunked encoding as an analogy, a common use case is where the length is unknown at the start, or streaming of data.  This doesn't help there.  I guess that a receiver will have to make some compromises (in performance, memory usage, or otherwise) if length is not known.  A receiver that is sufficiently constrained probably will want to limit the total message size anyway.

ISTM that the most of the other constraints will come from the structure of the API.  If you need to pass a single array, you might suffer.  Otherwise, a simple abstraction (e.g. Java's InputStream or Reader classes, maybe ByteBuffer depending on paradigm) can help you avoid needing to re-alloc().

If the receiver is so constrained that they can't deal with multiple frames, how able are they to use a BigInteger effectively, which would be necessary to hold CML?  Speaking from the Java perspective, this would be relatively easy, though not exactly efficient.

Overall, I'm not seeing a strong case for including CML.  It seems relatively harmless, but harmlessness is not really great motivation.

--Martin

p.s. I still struggle to identify what an intermediary is able to do with this protocol.

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 00:21:24 -0400

On Mon, Aug 23, 2010 at 12:00 AM, Thomson, Martin <[email-masked]

> last frame, or some other pattern?

Where the fragmentation is introduced by an intermediary or by a sender with
fixed-size but large content, I would expect it to be only on the first
frame and it would be completely accurate.  If a sender is fragmenting
dynamically generated content, I would expect an estimate might be included
on the first frame, and then an accurate count might be included once when
it is first known, if it is known before the last frame.  The estimate could
conceivably be updated several times if a more accurate estimate became
available.  In the worst case, the receiver still would have to realloc/copy
the buffer the same number of times if it had no CML estimates, but an
argument was made that it could be better and that it enabled UI affordances
like progress meters.

> Presumably, much of the benefit is lost if the complete length is only
> reported on the last frame - that information could be trivially gained by
> doing a little arithmetic (and I hear that computers are good at that ;).
>  That leads me to conclude that it’s most useful on the first frame.  Or at
> least as soon as possible.
>

Correct.

> ISTM that the most of the other constraints will come from the structure of
> the API.  If you need to pass a single array, you might suffer.  Otherwise,
> a simple abstraction (e.g. Java's InputStream or Reader classes, maybe
> ByteBuffer depending on paradigm) can help you avoid needing to re-alloc().
>

Remember that the current JS API is message based, where it receives a
single string containing the entire message.  It seems likely browsers are
going to allocate an entire buffer for the incoming message (granted some of
them will want that to be in UTF16 characters, which won't be tied to the
number of UTF8 bytes - there was discussion of perhaps adding that as an
extension), filling it as it reads fragments, and then handing it off to
client code once the last frame is received.

> If the receiver is so constrained that they can't deal with multiple
> frames, how able are they to use a BigInteger effectively, which would be
> necessary to hold CML?  Speaking from the Java perspective, this would be
> relatively easy, though not exactly efficient.
>

A 63-bit integer is a Java long, no BigInteger required (the high bit was
specifically required to be 0 to support Java's lack of an unsigned 64-bit
type).

> Overall, I'm not seeing a strong case for including CML.  It seems
> relatively harmless, but harmlessness is not really great motivation.
>

Primarily I suggested it as a way to avoid the apparently endless arguments
we were having about fragmentation.

> p.s. I still struggle to identify what an intermediary is able to do with

Initially, I think of proxies as the likely intermediary, and I see two
roles for them:

   - enforce corporate policies, such as restricting which sites can be
   contacted, sniff content for malware or corporate IP being leaked, etc
   - aggregate connections - a MUX extension would be helpful here to reduce
   the number of outbound connections when many internal users are hitting the
   same sites

Also, an intermediary might be used as a front end to a server farm, which
receives the connection, aggregates them, and load-balances to backend
servers.

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 23 Aug 2010 13:02:15 +0800

John Tamplin writes:
> Remember that the current JS API is message based, where it receives a single string containing the entire message.  It seems likely browsers are going to allocate an entire buffer for the incoming message (granted some of them will want that to be in UTF16 characters, which won't be tied to the number of UTF8 bytes - there was discussion of perhaps adding that as an extension), filling it as it reads fragments, and then handing it off to client code once the last frame is received.

'twasn't my memory that was the problem, but the assumption that reallocation of memory was necessary.  If the message arrives in pieces, you have two choices:
  A. re-assemble those pieces into the expected form
  B. provide an abstraction that makes it look like a single piece

This holds true irrespective of the nature of the API, though there might be some implementation constraints that make one or other harder.

A is a fairly unsophisticated approach, it's simple and might be perfectly good if you don't want to spend too much effort on implementation, want to expose a simple array to users.  A is even better if you have a frame size limit.  B follows the old maxim: every problem can be solved by adding another layer of indirection.  B might be more efficient if the implementation needs to be more flexible.

> Primarily I suggested it as a way to avoid the apparently endless arguments we were having about fragmentation.

Sadly, I don't think that it necessarily addresses those concerns; nor do I believe that those concerns are especially well grounded.

--Martin

p.s. Gmail and Outlook don't appear to be getting on very well.

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 23 Aug 2010 13:10:44 +0800

John Tamplin writes:
> Initially, I think of proxies as the likely intermediary, and I see two roles for them:
> • enforce corporate policies, such as restricting which sites can be contacted, sniff content for malware or corporate IP being leaked, etc
> • aggregate connections - a MUX extension would be helpful here to reduce the number of outbound connections when many internal users are hitting the same sites
> Also, an intermediary might be used as a front end to a server farm, which receives the connection, aggregates them, and load-balances to backend servers.

Aggregation of connections is only going to be possible if there is no state accumulation associated with a connection, or some way to identify/associate state for channels within a multiplexed stream.  The current design doesn't do a lot to facilitate any of this.  

It's a non-trivial problem.  These all sound nice, but the difference between theory and practice here is fairly large.  

I'll give an example: how would you, as an intermediary, decide to multiplex if you had an open WS stream to ws://example.com/a and someone requested a stream to ws://example.com/b ?

--Martin

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 01:21:36 -0400

On Mon, Aug 23, 2010 at 1:10 AM, Thomson, Martin
<[email-masked]>wrote:

> Aggregation of connections is only going to be possible if there is no
> state accumulation associated with a connection, or some way to
> identify/associate state for channels within a multiplexed stream.  The
> current design doesn't do a lot to facilitate any of this.
>

The current design punted all extensions for future discussion so we could
finally reach agreement on the framing.  I believe since the proposal
requires that intermediaries do nothing with any frame that has any reserved
bits set or uses a reserved opcode unless they understand the extension
negotiation, that we will be in no way restricted from implementing whatever
extensions we need.  I would expect that discussions on ironing out the
details of extensions would take place as soon as we say the framing is
done, and I would rather not muddy the waters discussing that before framing
is done.

It's a non-trivial problem.  These all sound nice, but the difference
> between theory and practice here is fairly large.
>
> I'll give an example: how would you, as an intermediary, decide to
> multiplex if you had an open WS stream to ws://example.com/a and someone
> requested a stream to ws://example.com/b ?
>

If they both go to the same endpoint and that endpoint negotiated the MUX
extension when /a was opened, I would expect to be able to MUX them over the
same TCP connection.  If there turns out to be some example where that
wouldn't be true, then we can have a way of negotiating that as well,
whether part of the original handshake or in MUX control frames.

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 23 Aug 2010 13:26:22 +0800

It was a rhetorical question, but I was really looking for a less handwavey answer.

From: John Tamplin [mailto:[email-masked]]
Sent: Monday, 23 August 2010 3:22 PM
To: Thomson, Martin
Cc: [email-masked]
Subject: Re: Intermediaries (was: CML)

On Mon, Aug 23, 2010 at 1:10 AM, Thomson, Martin <[email-masked]<mailto:[email-masked]>> wrote:
Aggregation of connections is only going to be possible if there is no state accumulation associated with a connection, or some way to identify/associate state for channels within a multiplexed stream.  The current design doesn't do a lot to facilitate any of this.

The current design punted all extensions for future discussion so we could finally reach agreement on the framing.  I believe since the proposal requires that intermediaries do nothing with any frame that has any reserved bits set or uses a reserved opcode unless they understand the extension negotiation, that we will be in no way restricted from implementing whatever extensions we need.  I would expect that discussions on ironing out the details of extensions would take place as soon as we say the framing is done, and I would rather not muddy the waters discussing that before framing is done.

I'll give an example: how would you, as an intermediary, decide to multiplex if you had an open WS stream to ws://example.com/a<http://example.com/a> and someone requested a stream to ws://example.com/b<http://example.com/b> ?

If they both go to the same endpoint and that endpoint negotiated the MUX extension when /a was opened, I would expect to be able to MUX them over the same TCP connection.  If there turns out to be some example where that wouldn't be true, then we can have a way of negotiating that as well, whether part of the original handshake or in MUX control frames.

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 01:27:24 -0400

>  It was a rhetorical question, but I was really looking for a less

The point is we haven't nailed down how it should work (or even how
extensions should be represented), but I remain confident that we have
sufficient hooks that we can implement what we need to.

---

From: Brian <[email-masked]>
Date: Mon, 23 Aug 2010 00:38:00 -0700

>> requested a stream to ws://example.com/b ?
In my infrastructure that I'm working on building, I'm using HAProxy with
layer-7 inspection specifically to forward requests for different paths to
different physical servers.  I specifically -don't- want multiple paths at
the same hostname to be muxed on the same connection.  If you want two
different endpoints, open two distinct TCP connections and be done with it.
 If you want a single stream, design your server and subprotocol to
specifically handle all the use cases.

Brian

---

From: Patrick McManus <[email-masked]>
Date: Mon, 23 Aug 2010 08:23:13 -0400

On Mon, 2010-08-23 at 00:21 -0400, John Tamplin wrote:

> Also, an intermediary might be used as a front end to a server farm,
> which receives the connection, aggregates them, and load-balances to
> backend servers.

+1 on connection management.

If WS is over TLS, this server side intermediary is going to want to
terminate the TLS off the server as well. That makes it necessary to
have some kind of non-tls format for it to forward the stream on with.
(HTTPS/HTTP works well here imo.)

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 08:24:10 -0400

>> single string containing the entire message.  It seems likely browsers
>> are going to allocate an entire buffer for the incoming message (granted
>> some of them will want that to be in UTF16 characters, which won't be
>> tied to the number of UTF8 bytes - there was discussion of perhaps
>> adding that as an extension), filling it as it reads fragments, and then
>> handing it off to client code once the last frame is received.
>
> 'twasn't my memory that was the problem, but the assumption that
> reallocation of memory was necessary.  If the message arrives in pieces,
> you have two choices:
> This holds true irrespective of the nature of the API, though there might
> be some implementation constraints that make one or other harder.
>
> A is a fairly unsophisticated approach, it's simple and might be perfectly
> good if you don't want to spend too much effort on implementation, want to
> expose a simple array to users.  A is even better if you have a frame size
> limit.  B follows the old maxim: every problem can be solved by adding
> another layer of indirection.  B might be more efficient if the
> implementation needs to be more flexible.
>
>> Primarily I suggested it as a way to avoid the apparently endless
>> arguments we were having about fragmentation.
>
> Sadly, I don't think that it necessarily addresses those concerns; nor do
> I believe that those concerns are especially well grounded.

The above taken together with Thomson's prior post:

http://www.ietf.org/mail-archive/web/hybi/current/msg03525.html

I agree with Thomson's points.  The CML is useless, and perhaps harmful.

CML is not a generalized abstraction of what it attempts to do.

CML abstraction should be in the data not in frame header. For example, if
the sender wants to signal the size of array, then put it in the data of
the first frame sent.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 09:48:10 -0400

> (HTTPS/HTTP works well here imo.)

Can we please stop conflating the protocol stack layers?  Here is another
example:

http://www.ietf.org/mail-archive/web/hybi/current/msg03551.html

And yet more examples of wanting to conflate TLS with WebSockets base
protocol, which goes against the principle of letting users (market)
decide:

http://www.ietf.org/mail-archive/web/hybi/current/msg03552.html

It is sub-optimal design, it is often harmful in unforseen ways, and it is
nearly _always_ unnecessary conflation (there is another way to do it).

Tangential to my main point, I personally wouldn't feel that secure about
losing encryption inside the LAN or even inside the server or even inside
the database server (end-to-end security please!):

https://bugzilla.mozilla.org/show_bug.cgi?id=588704#c43

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 10:28:48 -0400

> specifically handle all the use cases.

Why wouldn't you want your proxy to receive multiplexed connections from the
clients to reduce load, and then you can split them out to the other side
how you like?  Plus, if you don't want multiplexing, simple have your proxy
fail to negotiate it.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 10:50:09 -0400

And when we return the CML bit to reserved bits, please make it the MSB
next to LenLen, so that if ever we need to support 128-bit sizes, it will
be easy to do so.

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Patrick McManus <[email-masked]>
Date: Mon, 23 Aug 2010 11:02:13 -0400

On Mon, 2010-08-23 at 09:48 -0400, Shelby Moore wrote:

> Can we please stop conflating the protocol stack layers?  

nope, because layers are a weak abstraction and they do not wholly
reflect the real world requirements of infrastructure operators.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 11:15:08 -0400

> reflect the real world requirements of infrastructure operators.

I disagree they must be weak.  That seems like an excuse to do bad design.

Build your own protocol on top of the base protocol to signal to your
internal LAN to drop the TLS.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 11:18:52 -0400

> internal LAN to drop the TLS.

Also the reason you are wanting to drop the TLS inside your LAN is
probably because we are conflating TLS with WebSockets, not for its
security, because we think it will be a more successful in the market
place:

And we should not be making that conflation.  Let the market decide.  TLS
is for security.  People use it because they expect you to keep the data
secure.  Now you are trying to change what TLS is used for, and that is
what is causing the conflation you want to do above.

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 11:27:18 -0400

> the sender wants to signal the size of array, then put it in the data of

The CML is not primarily intended for the sender (if the sender knows the
length up front, it is probably sending it in one message) -- it is intended
so an intermediary that fragments a single-frame message can still let the
receiver get the benefit of knowing the buffer size required up front.  The
other uses were added on because someone asked for them and it was free to
do so.

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 11:38:10 -0400

> what is causing the conflation you want to do above.

The data is decrypted inside the receiving computer anyway, and is passed
unencrypted between software components.  Why should that be any different
if my "server" the external user sees is in fact a cluster of machines each
running part of the protocol stack?

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 11:43:07 -0400

> other uses were added on because someone asked for them and it was free to

You still didn't get my or Thomson's point.

The application data structures should not be in the frame header.  It
isn't necessary and isn't desired and doesn't help and is conflation.

If the sending application puts the sizes of the data structures in the
data, then the intermediary doesn't have to know anything about the data
structures. Don't burden the intermediaries with something they have
nothing to do with.  The base protocol is for handling frames and frames
only.  Refragmenting frames has nothing to with the data structures inside
the frames.  Don't conflate it, that is INCORRECT DESIGN.

Just like TCP doesn't know anything about WebSockets, SMTP, SSH, HTTP,
HTTPS, FTP, etc...

Stop the conflation please. It is a fundamental concept of good design.
Any good network designer knows you don't conflate the layers of the
network stack. I know that and I am not even an expert in networks.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 11:54:18 -0400

> running part of the protocol stack?

1) What you do inside your LAN is your own business, but there is no
justification for forcing that on the standard that is used every where
outside your LAN.

2) I happen to expect you to keep my data encrypted always, even when you
store it.  I know computers are perfected yet in that regard, but they
need to be:

https://bugzilla.mozilla.org/show_bug.cgi?id=588704#c42
https://bugzilla.mozilla.org/show_bug.cgi?id=588704#c43

But regardless of whether the user's expectation of security with TLS is
met by your company, nevertheless, TLS is not a routing protocol that we
need to conflate into our framing header for WebSockets.  TLS is a way to
achieve encryption.  If your particular implementation needs to signal
something, then signal it on top of the base protocol, not force the
entire world to carry your reserved bit around as unused payload.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 08:54:33 -0700

On Mon, Aug 23, 2010 at 8:38 AM, John Tamplin <[email-masked]> wrote:

> On Mon, Aug 23, 2010 at 11:18 AM, Shelby Moore <[email-masked]>wrote:
>

Which is indeed a very conventional configuration.

-Ekr

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 11:57:17 -0400

> isn't necessary and isn't desired and doesn't help and is conflation.

This isn't a application data structure.

> the frames.  Don't conflate it, that is INCORRECT DESIGN.

Why should the upper-level app, which has nothing to do with buffer
allocation inside the sender and receiver, include a message length for the
receiver to interpret?  How can the receiver know where the app put it?  Why
should it be included on a single-frame message when you already know the
total length of the message?

> network stack. I know that and I am not even an expert in networks.

Yes, and you are suggesting an application-layer component provides the
length to a transport-layer component, which seems a pretty clear violation.

Think about what is actually going on here.  The WS server has a block of
text it wants to send, and it calls ws.sendMessage(binaryArray).  The
sender's WebSocket layer will then take that and, assuming the size is less
than the negotiated maximum length, it will write a header and the text in a
single frame.  There is no need to include CML, since the frame length is
the message length.  Along the way, an intermediary wants to fragment it.
 It knows the complete message length, which is the frame length of the
original, so when it fragments the message it provides it to the receiver.
 The browser now receives this first frame.  Ultimately, it wants to pass an
object containing the message to JS code, and it needs to allocate a buffer
to receive the data.  How big does it allocate?  Without CML, it doesn't
know and must guess.  Note that if there were no intermediary introducing
fragmentation, it would have had the buffer length required after reading
the header of the first frame.  Note that it is not the application on the
receiving end that consumes the CML, but rather then WS framing level, which
is why the CML is included in the framing rather than the application data.

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 12:01:23 -0400

> justification for forcing that on the standard that is used every where

I thought you were the one arguing that non-TLS should be an option as well?
 So why are you objecting to the connection between the frontend and backend
not being in TLS?

> store it.  I know computers are perfected yet in that regard, but they

This isn't about storing it - it is about layering software.  I receive a
frame that is encapsulated in TLS -- I remove the TLS wrapping (ie, decrypt
it), and pass it to the next layer.  If that next layer happens to be on a
different machine in a cluster, I don't see why I should have to re-encrypt
it to send it there rather than if that next layer happens to be on the same
machine.

> entire world to carry your reserved bit around as unused payload.

I am really at a loss to understand what you are arguing at this point.  You
seem to be arguing about a CML bit while discussing TLS, and they aren't
related in the slightest

---

From: Patrick McManus <[email-masked]>
Date: Mon, 23 Aug 2010 12:03:34 -0400

On Mon, 2010-08-23 at 11:18 -0400, Shelby Moore wrote:
> Now you are trying to change what TLS is used for, and that is

I have noted that large installations use RSA offloaders for valid price
performance reasons and it is worth considering what happens when that
pattern re-applies itself to a websockets-over-tls deployment. I didn't
advocate using TLS in any particular pattern, I simply suggested we
consider the implications of a forseeable implementation action.

I would personally appreciate it if you took a breath or three in
between postings so there might be enough proverbial oxygen in the room
to hear from more diverse voices/perspectives in a less frenzied
environment.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 12:05:58 -0400

> Which is indeed a very conventional configuration.

It doesn't matter, even if it was used by the entire world every where, it
still does not justify conflating.

Essentially what you are asking for WebSockets' base protocol to carry
around an identifier of the protocol being used underneath it.  Do you
know how many more bits we are going to need to reserve for that? And who
is going to run the repository of registrations for new uses of
WebSockets's base protocol?

Let WebSockets' base protocol be orthogonal to the protocol(s) that we use
to transport it on top of.  AND THAT IS the convention way to do correct
network layer design.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 23 Aug 2010 09:20:27 -0700

> I disagree they must be weak.  That seems like an excuse to do bad design.

You can achieve good or bad design either way.
The layers are not inviolable nor are they sacred. They exist as a useful
abstraction. If their utility is less than that of a layer conflation, so be
it.

It would have been extremely nice, for instance, if we could send some data
with the TCP handshake. That would potentially be a layer violation, but
nonetheless would allow for the users to be able to use their applications
with one fewer RTT-- an extremely desirable outcome.

-=R

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 09:24:24 -0700

On Mon, Aug 23, 2010 at 9:05 AM, Shelby Moore <[email-masked]> wrote
> still does not justify conflating.

[...]

> to transport it on top of.  AND THAT IS the convention way to do correct

Honestly, I have no idea what argument you're making.

The conventional way to disambiguate protocols which run at layer N+1 is to
have
an indicator in layer N. For instance, SMTP, HTTP, and SSH run over TCP and
are disambiguated by an indicator in the TCP layer, namely port.

What is being suggested here is simply say that WebSockets and ordinary
non-WS HTTP would be disambiguated by an indicator at a lower layer, and
that furthermore we should commit to that lower layer be TLS, at which point
NPN or something like it becomes the relevant indicator. This seems to me
to be an utterly conventional design. Note that if we suddenly decided that
we wanted to support WS over some non-TLS protocol (e.g., HTTP over
SSH), then what would be required would be have an indicator in that layer.

> WebSockets's base protocol?

I think you are misunderstanding the proposal. The TLS layer would simply
indicate that what was in use was WebSockets. Any upper-level protocols
would
be indicated in the WS layer. Again, this is quite a conventional design;
IP indicates that it is carrying TCP; TCP indicates that it is carrying
HTTP,
etc.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 09:26:12 -0700

> with one fewer RTT-- an extremely desirable outcome.

Not only was this a good idea, it's actually been done:
http://www.rfc-archive.org/getrfc.php?rfc=1644

Unfortunately, for a variety of reasons which aren't relevant here, it
didn't take off.

-Ekr

---

From: Roberto Peon <[email-masked]>
Date: Mon, 23 Aug 2010 09:32:06 -0700

On Mon, Aug 23, 2010 at 9:26 AM, Eric Rescorla <[email-masked]> wrote:

>> On Mon, Aug 23, 2010 at 8:15 AM, Shelby Moore <[email-masked]>wrote:
>>
>> It would have been extremely nice, for instance, if we could send some
>> data with the TCP handshake. That would potentially be a layer violation,
>> but nonetheless would allow for the users to be able to use their
>> applications with one fewer RTT-- an extremely desirable outcome.
>>
>

Thanks for the reference-- I already knew about t/tcp :)
For that matter, I'd really love to spend time on it to see if the blocking
issues could be mitigated. Unfortunately deployment of such as t/tcp would
take a long time and parts of which can be (for better and worse) emulated
by having the layer above store data about the connection (e.g. application
protocol stores data on behalf of the transport at the end of a session
which is reconstituted for a new session at the transport layer). Thus I
spend my time elsewhere for now!

-=R

---

From: "Brian Smith" <[email-masked]>
Date: Mon, 23 Aug 2010 11:46:29 -0500

The protocol should be designed to be optimized for any particular web browser memory allocation strategy. There is so many layers and each is potentially going to do its own fragmenting/coalescing: at the TCP/IP level, in the IP(v6) compression layer, in the operating system network buffer manager, in the operating system’s VMM, in the TLS stack (including an extra layer for compression), in the browser’s networking code (including possibly an extra layer for compression again), in the browser’s JavaScript buffer/string handling, etc. As long as the protocol is not designed maliciously, it should be straightforward to find a memory allocation strategy that adds virtually no overhead.

Plus, like somebody else pointed out, a lot of the buffer management stuff can be abstracted out at the application API level like boost::buffer_iterator and the old SGI Rope class do for C++. 

I think some basic problems have to be solved before it makes sense to try to resolve issues CML. For example, if WebSockets is to be a TLS-only protocol, then there are no intermediaries fragmenting messages according to their own whims. In the case of a multi-server configuration, the way messages are transferred between the servers is an implementation detail; the wire format used for framing between those servers could be completely different than the standard WebSockets wire format (e.g. just “real” TCP/IP), and even the format of the application data could be different (e.g. using efficient binary encodings instead of data that was originally sent JSON-encoded). It isn’t (necessarily) WebSockets all the way down.

Also, I joined the list late. Was there already a discussion about the merits of optimizing framing below the TCP/IP MTU? In particular, what is the motivation for optimizing the overhead of a 128 byte message when even a 512-byte message will still only require a single packet? If the idea is that many frames could be stuffed into a single TCP/IP packet when multiplexing is used, then it is probably better to instead directly address the overhead of that multiplexing. Perhaps techniques similar to the ones used in modern SQL databases (e.g. Oracle) for compression of data within a given block could be used: create a frame header format that can describe multiple frames much more compactly than it can describe frames individually, and send that “multi-header” ahead of all the frames it describes. But, such complexity should first have a clear concrete motivation.

Regards,

Brian

The browser now receives this first frame.  Ultimately, it wants to pass an object containing the message to JS code, and it needs to allocate a buffer to receive the data.  How big does it allocate?  Without CML, it doesn't know and must guess.  Note that if there were no intermediary introducing fragmentation, it would have had the buffer length required after reading the header of the first frame.  Note that it is not the application on the receiving end that consumes the CML, but rather then WS framing level, which is why the CML is included in the framing rather than the application data.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 17:55:25 +0100

On Mon Aug 23 17:46:29 2010, Brian Smith wrote:
> Also, I joined the list late. Was there already a discussion about  
> the merits of optimizing framing below the TCP/IP MTU? In  
> particular, what is the motivation for optimizing the overhead of a  
> 128 byte message when even a 512-byte message will still only  
> require a single packet?

Makes a lot of difference to mobile cases, particularly 3G. A  
low-traffic connection using <128 octet packets (including TCP/IP  
overhead) will use less than half the power of a high-bandwidth one  
based on radio state alone.

This threshold is under operator control, so in principle a  
particular mobile network could choose something other than 128  
octets, but I've not seen that case. Maybe some proper mobile people  
might comment.

Either way, at 128 octets and below, every byte counts.

Above that level, I agree it's generally a case of packets rather  
than octets.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 12:57:06 -0400

On Mon, Aug 23, 2010 at 12:46 PM, Brian Smith <[email-masked]> wrote:

> The protocol should be designed to be optimized for any particular web
> browser memory allocation strategy. There is so many layers and each is
> potentially going to do its own fragmenting/coalescing: at the TCP/IP level,
> in the IP(v6) compression layer, in the operating system network buffer
> manager, in the operating system’s VMM, in the TLS stack (including an extra
> layer for compression), in the browser’s networking code (including possibly
> an extra layer for compression again), in the browser’s JavaScript
> buffer/string handling, etc. As long as the protocol is not designed
> maliciously, it should be straightforward to find a memory allocation
> strategy that adds virtually no overhead.
>
>
>
> Plus, like somebody else pointed out, a lot of the buffer management stuff
> can be abstracted out at the application API level like
> boost::buffer_iterator and the old SGI Rope class do for C++.
>
>
The JS API, in its current form, eliminates many of those options.  It was
also representatives of browser makers who were arguing against
fragmentation because of receiver buffer management issues, and adding CML
removed those objections.

> I think some basic problems have to be solved before it makes sense to try
> to resolve issues CML. For example, if WebSockets is to be a TLS-only
> protocol, then there are no intermediaries fragmenting messages according to
> their own whims. In the case of a multi-server configuration, the way
> messages are transferred between the servers is an implementation detail;
> the wire format used for framing between those servers could be completely
> different than the standard WebSockets wire format (e.g. just “real”
> TCP/IP), and even the format of the application data could be different
> (e.g. using efficient binary encodings instead of data that was originally
> sent JSON-encoded). It isn’t (necessarily) WebSockets all the way down.
>
>
If WebSockets over TLS eliminates the possibility of corporate proxies to
inspect and aggregate content, then I think it shouldn't be the only option.

> Also, I joined the list late. Was there already a discussion about the
> merits of optimizing framing below the TCP/IP MTU? In particular, what is
> the motivation for optimizing the overhead of a 128 byte message when even a
> 512-byte message will still only require a single packet? If the idea is
> that many frames could be stuffed into a single TCP/IP packet when
> multiplexing is used, then it is probably better to instead directly address
> the overhead of that multiplexing. Perhaps techniques similar to the ones
> used in modern SQL databases (e.g. Oracle) for compression of data within a
> given block could be used: create a frame header format that can describe
> multiple frames much more compactly than it can describe frames
> individually, and send that “multi-header” ahead of all the frames it
> describes. But, such complexity should first have a clear concrete
> motivation.
>

One concern is mobile networks, such as 3G where the radio can send small
frames in a low-power FACH mode which supports around 128-256 bytes
(depending on the carrier's configuration) including all overhead.  Further,
the only traces of real data that have been presented show that 98%+ of
frames will fit under 126 bytes after compression is supported, and it makes
sense to optimize for the most frequent case when it is that high a
percentage.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 13:17:03 -0400

> This isn't a application data structure.

If it is not, then you do not need to worry about it.

The point is that size of the "original" frame is both ambiguious and
meaningless.  What is meaningful is the way the receiver intends to store
the data.  And I said 'data' not 'frames'.

> receiver to interpret?

That is wrong assumption.  The receiver will allocate two kinds of buffers:

1) Frames (separate malloc for each fragmented frame).
2) Data structures

The receiving application does not care if the data structures span frame
fragments.  Are you telling me that for example a Javascript array is a
contiguous malloc'ed block?  I think it is not.

The WebSockets implementation has to provide the means for that.  The
scripting amatuer programmer layer will never be aware it is going on.

>  How can the receiver know where the app put it?

See above.

> Why

I don't understand the question.

> length to a transport-layer component, which seems a pretty clear

No I am not suggesting that.  The transport-layer does not need to know
the length of the application layer data structures.  And the
transport-layer does not need to know the original size of the original
frame, because it is meaningless and useless and undefineable any way.

> to receive the data.  How big does it allocate?

You are conflating layers.

The sending layer will serialize the Javascript data structure, perhaps
into JSON (even if it was already a JSON string being transported), and
then the receiving layer will de-serialize it.  The de-serializer code
needs to know how to hop from frame fragment to next one.

The JS coder never sees that.  That de-serializer is a low level operation
inside the browser code.

>  Without CML, it doesn't

Read above.  You are not handling it correctly.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 13:30:05 -0400

> The JS coder never sees that.  That de-serializer is a low level operation

And thus your current statistics on mean frames sizes could end up being
too low.

Maybe the serializer will be more efficient than JSON.

But I think it is really wrong to try to squeeze 2 - 4% of blood with the
"Option #2 - 7/16/63-bit" at the possible disaster having most frame sizes
end up greater than 126 once we get all the conflation out of the design.

Must better to add the extra byte and choose either "Option 15/63-bit" or
"Option 8/16/32/64-bit" so as to get some margin of error and safety.

Also safety against my alleged game theory disaster below, regardless
whether you consider it likely or not, it just isn't good engineering to
not allow a margin of error for safety:

http://www.ietf.org/mail-archive/web/hybi/current/msg03578.html

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 13:58:37 -0400

> I thought you were the one arguing that non-TLS should be an option as

Correct I am. I have not changed.  We should support both HTTP and TLS.

Tangentially, I am also still praying to geek god of divine intervention,
that somehow this list will see the wisdom of WebSockets over any other
port with recommended STUN tunneling to get through roughly 65% of the
NATs (roughly same success as for HTTP, with both to rise in success as
they gain support and use).  It might just gain momentum and make us all
super proud of our dramatic effect on the world.

>  So why are you objecting to the connection between the frontend and

I am not objecting technically.  You can do what you want.  Just don't
force a CML bit to accomplish it.

I am only objecting that if I was a user of your services and if you are
going to promise to secure my data and you don't keep it encrypted, then I
am going to call your company a liar, because no company can be sure a
hacker won't get inside.  Google in China is an example.  But I am not
being too hard on you for that, because none of us are employing
end-to-end security yet.  I wish we would.  There are many barriers, such
as the operating systems, etc.  Unfortunately everyone thinks the firewall
is security.  But any way, that is not worth debating here.  Lets stay on
the technical debate about conflation.

See below...

> it to send it there rather than if that next layer happens to be on the

Same reply as above.

> seem to be arguing about a CML bit while discussing TLS, and they aren't

I had originally replied to Patrick McManus, who was replying to John
Tamplin, who was arguing that we needed the CML bit in order to inplement
the TLS issue we are debating now:

http://www.ietf.org/mail-archive/web/hybi/current/msg03553.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03530.html

Or was I mistaken about what they were arguing for?

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 14:05:49 -0400

> to hear from more diverse voices/perspectives in a less frenzied

I have no technical objection to what you are describing above, as long as
you are not using that to argue for a CML bit.  I had originally replied
to your original reply, where you had replied to John Tomplin, who was
debating Thomas about the CML bit.

Are you arguing for a CML bit with that?  If not, then I drop all
technical objection.

I am simply replying to the messages.  I am a volunteer here, doing my
best to help.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 14:16:29 -0400

> abstraction. If their utility is less than that of a layer conflation, so

Incorrect:

http://www.ietf.org/mail-archive/web/hybi/current/msg03583.html

Maybe because I have a lot of experience with application programming,
that is why I was able to see this.  You networking guys are trying to do
too much in one layer.  Let the application layer handle the
defragmentation, because in reality, applications never assume contiguious
memory for data structures.  They use linked lists, hash tables, etc.. 
You guys were thinking about sendig one contiguous binary blob and
thinking you then needed to receive the same.  That is the wrong way to
think about it.  The application must serialize its data structures and
then deserialize them again.  That is the way transport is always
accomplished.

> data

Actually that is wrong. It has been proven again and again in computer
science, that such "optimizations" end up as massive accumulated spaghetti
breakage.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 14:45:33 -0400

> Honestly, I have no idea what argument you're making.

I am arguing against the CML bit only.

below...

> are disambiguated by an indicator in the TCP layer, namely port.

I was arguing against N-1.

I thought you were going to put the indicator for the lower layer (TLS) in
the higher layer (WS), which is of course wrong.  If not, then I have no
technical objection.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 14:50:07 -0400

> issues could be mitigated.

Of course not.  You can't conflate and expect to not have intractable
disasters:

http://www.ietf.org/mail-archive/web/hybi/current/msg03589.html

> Unfortunately deployment of such as t/tcp would

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 11:55:28 -0700

> http://www.ietf.org/mail-archive/web/hybi/current/msg03589.html

As far as I can tell, the sorts of issues you are raising have nothing to
do with why T/TCP failed. Rather, it was the customary coordination
problems:

(1) Clients didn't know if they could safely send it so they didn't.
(2) Servers couldn't depend on clients having it so they didn't deploy.
(3) Intermediaries blocked it and there wasn't any real incentive to unblock
     it because people didn't really use it (see #1 and #2).

Now, it's true that there are some security issues around T/TCP but they're
not *that* hard to ameliorate, if you had an otherwise workable protocol.

-Ekr

---

From: Roberto Peon <[email-masked]>
Date: Mon, 23 Aug 2010 11:56:33 -0700

On Mon, Aug 23, 2010 at 11:55 AM, Eric Rescorla <[email-masked]> wrote:

> On Mon, Aug 23, 2010 at 11:50 AM, Shelby Moore <[email-masked]>wrote:
>

Which is precisely why I'd love to be able to spend time on it :)
-=R

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 11:57:02 -0700

> the higher layer (WS), which is of course wrong.  If not, then I have no

I don't believe that anyone was in fact arguing for that.

However, in point of fact, putting an indicator in a higher layer that
points to a lower layer (e.g., channel binding) is actually quite common
and accepted practice as well.

-Ekr

---

From: Roberto Peon <[email-masked]>
Date: Mon, 23 Aug 2010 12:11:15 -0700

> > abstraction. If their utility is less than that of a layer conflation, so

"Incorrect"? This isn't a matter of fact, it is one of opinion and it is
not, as far as I know provable any more than is any other philosophical
matter.
I'm not an ideologue, and so I like to chase the most pragmatic solution.

> that is why I was able to see this.  You networking guys are trying to do

"why I was able to see this" implies that others are not able to. This may
or may not be what you meant, but it does help to ruffle feathers and does
not help you forward your opinions.

> Actually that is wrong.

> It has been proven again and again in computer

 At times overall system complexity can be *reduced* by layer violation. I'm
hoping that we're generally chasing a reduction of total complexity!
In the case of these layers, we end up with work-arounds that do increase
overall complexity.

I won't respond further on this tangent on this thread.
-=R

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 15:38:17 -0400

[snip]

> sense to optimize for the most frequent case when it is that high a

The optimization of frame header size for 126 threshold is only 1 byte
less (2 - 4%), so it doesn't help the above, and has other risk factors
which make it not worth it (whether you believe those are risks or not, we
should still give safety margin for unknown risks):

http://www.ietf.org/mail-archive/web/hybi/current/msg03597.html

Many people have said we should have a larger static size for the frame
size field, and they have expressed that we shouldn't be so overly focused
on "optimizing" something so much that we don't allow some safety margin.

Some apparently think the 126 threshold costs nothing, but we don't know
that for sure.  One analysis is that if frames end up greater than 126,
then with "Option #2 7/16/63-bit", then that is only 2/128 = 1.6% increase
in load.  However, why risk it, when it only costs us 2 - 4% to add a
byte?  And "Option 8/16/32/64-bit" has a higher CPU load, so it seems to
me that "Option 15/63-bit" hits the sweet spot in balancing efficiency and
unknown risk.  It also is more efficient for the > 126 case.

---

From: gustav <[email-masked]>
Date: Mon, 23 Aug 2010 21:44:13 +0200

> unknown risk.  It also is more efficient for the > 126 case.
You convinced me at least, i want to change my vote to 15/63 =).

regards
 gustav trede

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 21:05:24 +0100

On Mon Aug 23 20:38:17 2010, Shelby Moore wrote:
> > frames will fit under 126 bytes after compression is supported,  
> and it
> > makes
The only use-case proposed where the precise octet-count of the  
overall packet is significant is the one John outlines above.  
Therefore, it makes perfect sense that this is the only case where we  
actually care about an octet enough to optimize for it.

> that for sure.

This is FUD. You need to support this kind of handwaving with  
evidence.

>   One analysis is that if frames end up greater than 126,
> byte?

Because it's quite likely to cost more - it can cost double the  
power, for starters, if it happens to cause a handset to have to  
raise to DCH, as well as a not insignificant time to transition, as  
it requires radio traffic to do so. Transitions cost about 2.5  
seconds from either IDLE or FACH, and during this time the handset  
will be drawing 380mA (on my handset), and it'll remain drawing that  
power for 8 seconds *after* the packet has been received. That's over  
10 useless seconds, drawing power at a rate which would empty the  
battery in 3 hours.

If you want to know whether that's a significant cost, I suggest  
asking John what percentage of packets in his samples would be  
triggered - I know from XMPP that every octet counts at that level.

>   And "Option 8/16/32/64-bit" has a higher CPU load, 

I think John clearly demonstrated that it won't.

> so it seems to

Where no use-case has been presented in which it matters.

Dave.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:06:04 -0400

> You convinced me at least, i want to change my vote to 15/63 =).

Forgot to mention that "Option 15/63-bit" has also less CPU load than
"Option #2 7/16/63-bit" as well.

And again I repeat it is more efficient for the > 126 case.

Afaics so far, the only big risk of using "Option 15/63-bit" versus
"Option #2 7/16/63-bit", is at the extreme of 1 byte frame size, it uses
33% more network bandwidth.  But how likely is that, given most
applications will serialize data structures, meaning a 1 byte payload is
impossible.  And that is not counting the TCP/IP payload too.  So in
reality the maximum risk is probably about 10% more network bandwidth at
the extreme, but that won't even be the mean.

The problem with the "Option #0 - 8/63-bit" and "Option #2B - 8/16/63-bit"
is they eliminate 1 reserved bit and only have 2 reverse bits.  That is
not enough reserved bits if we are going to allow many extension
protocols.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 21:10:10 +0100

> not help you forward your opinions.
It also implies that none of us have any application programming  
experience, nor experience of higher-level network protocol  
engineering.

Dave.

---

From: gustav <[email-masked]>
Date: Mon, 23 Aug 2010 22:13:03 +0200

On 23 August 2010 22:05, Dave Cridland <[email-masked]> wrote:

>>  The only use-case proposed where the precise octet-count of the overall
> packet is significant is the one John outlines above. Therefore, it makes
> perfect sense that this is the only case where we actually care about an
> octet enough to optimize for it.
>
>
> This is FUD. You need to support this kind of handwaving with evidence.
>
>
> Because it's quite likely to cost more - it can cost double the power, for
> starters, if it happens to cause a handset to have to raise to DCH, as well
> as a not insignificant time to transition, as it requires radio traffic to
> do so. Transitions cost about 2.5 seconds from either IDLE or FACH, and
> during this time the handset will be drawing 380mA (on my handset), and
> it'll remain drawing that power for 8 seconds *after* the packet has been
> received. That's over 10 useless seconds, drawing power at a rate which
> would empty the battery in 3 hours.
>
> If you want to know whether that's a significant cost, I suggest asking
> John what percentage of packets in his samples would be triggered - I know
> from XMPP that every octet counts at that level.
>
>
> Dave ,that is a rather arrogant way of expressing your point o view,
you seem to run out of valid arguments when your resorting to personal
attacks.

regards
 gustav trede

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:20:34 -0400

> actually care about an octet enough to optimize for it.

Do you care about data serialization and scripting?  That is a use case
that may alter John's statistics about frame sizes.  Some games are
hand-optimized, but Javascript will serialize its data structures in a
generalized format.

> This is FUD. You need to support this kind of handwaving with

"we don't know for sure" is FUD?

I already presented use cases that may not have been included in John's
statistics (?) before you wrote this sir, so please retract the false
accusation.

> power,

1 extra byte can't affect the thresholding John mentioned, and 2 - 10%
increase in load can't double the power:

http://www.ietf.org/mail-archive/web/hybi/current/msg03601.html

> for starters, if it happens to cause a handset to have to

That has nothing to do with adding 1 byte or 2 - 10% in load.

> I think John clearly demonstrated that it won't.

To the contrary, he clearly demonstrated that it will:

http://www.ietf.org/mail-archive/web/hybi/current/msg03597.html
(see the bottom of the post)

> Where no use-case has been presented in which it matters.

Potential use cases have been outlined, and there is a negligible cost of
adding the safety margin.

I think you are confused.  Or have I missed something?

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:23:59 -0400

> not *that* hard to ameliorate, if you had an otherwise workable protocol.

I do not want to spend the time to argue this.  But my point remains that
conflation is sub-optimal design, and there is always a way to eliminate
it.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:35:17 -0400

>> > The conventional way to disambiguate protocols which run at layer N+1
>> is
>> > to
>> > have
>> > an indicator in layer N. For instance, SMTP, HTTP, and SSH run over
>> TCP
>> > and

That conflation is done for a very specific optimization because of the
special dual-end requirements of cryptography:

http://www.rfc-editor.org/rfc/rfc5056.txt

It is not a general design goal to emulate.  It is done above because the
desire to accelerate security outweighs the desire to be orthogonal to the
possible layers that could be placed underneath the higher layer.

We have no such special need driven by cryptography, because our layer
sits on top of TLS and TLS can bind with IPsec for us.

Rather our overriding need is simplicity and orthogonality, because we are
sitting just below web applications.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:44:30 -0400

>> > abstraction. If their utility is less than that of a layer conflation,
>> so
>> > be
>> > it.
>>
>> Incorrect:
>>
>

below...

> In the case of these layers, we end up with work-arounds that do increase

That is possible only if you have limited the features too.

It isn't appropriate in most general use cases.  If you have a very narrow
goal and your protocol will never be cross-composed in ways that you did
not anticipate, then you can make gains in speed and/or apparently
complexity.

But these are temporal illusions, because they actually increase your
complexity some where else.  Limiting features is like outlawing sex, you
can abide for a while, but really hard to do it forever.

It will be justified very, very rarely in optimal design.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 13:45:00 -0700

> http://www.rfc-editor.org/rfc/rfc5056.txt

Since I'm the one who referenced channel bindings, I think it's safe to
say I know how to find it.

> possible layers that could be placed underneath the higher layer.

Huh?

I'm fairly familiar with the design forces that drove channel bindings
(and the concepts that preceded it) and they were precisely the desire
to cross layers, namely to have confidentiality and integrity provided
at a generic layer but to have the application layer handle authentication
rather than extending the lower-level authentication concepts all the way
upward.
As far as I know, "accelerating security" wasn't a design goal at all,
unless
what you mean by that is "getting security deployed". Rather,
the design goal was to have a generic layer where possible but to glue it
to a specific layer where necessary.

> sits on top of TLS and TLS can bind with IPsec for us.

Well, I wasn't saying that we needed to do channel bindings but rather that
there were situations where it was appropriate for layer N+1 to explicitly
reference layer N, using channel bindings as an example. You seem to
be arguing that that's a special case, but I'm not convinced.

> sitting just below web applications.

I don't see how this follows at all.

-Ekr

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 16:58:52 -0400

I don't really care if you choose 7/16/63-bit or 15/63-bit.  I was just
trying to help analyze.  I originally voted for the 7/16/63-bit, then I
was listening to what everyone else wrote against it, so I realized maybe
they had a point. And then I realized that given we _MUST_ have a maximum
frame size, that the 126 threshold does scare me a bit. Maybe the is FUD,
but it is not FUD to say we don't know for sure our real world frame size
stats.

In any case, if the 2 - 10% load risk of 12/63-bit scares you (also FUD?),
I want to mention that this whole problem with batteries was solved 100
years ago by Tesla!  But the power transmission companies don't want to
allow transmission of electricity via airwaves, because it affects their
monopoly. Why do you think JPMorgan funded Tesla?

I don't like to design for marginal edge case arguments.  I like to leave
some room for error on either side of the argument.  I was trying to pick
the choice with the least chance of blowing up in our face.

I respect the wishes of the list.

Now I have been doing nothing but this list for 3 days.  I need to go get
some real work done.

I hope I have helped and will be appreciated.  But if not, so be it.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 22:01:16 +0100

On Mon Aug 23 21:20:34 2010, Shelby Moore wrote:
> >> > (depending on the carrier's configuration) including all  
> overhead.
> >> > Further,
> >> > sense to optimize for the most frequent case when it is that  
> high
> >> a
> >> > percentage.
> >>
> >> which make it not worth it (whether you believe those are risks  
> or
> >> not, we
I do indeed care about both, but that doesn't alter the argument...  
It's already been demonstrated that small packet sizes are achievable  
in web apps, so arguing that it cannot be the case without evidence  
seems a very weak argument.

Are you arguing that if some people might be unable to optimize their  
application messages, nobody should be allowed to do so?

> >> Some apparently think the 126 threshold costs nothing, but we  
> don't
> >> know
> >> that for sure.
> >
Yes.

It's a vague expression of "there might be something wrong we haven't  
thought of, and you'll never be able to prove there isn't". That  
statement is true, but it's equally true of anything. You need to  
demonstrate that there is a valid issue affecting this design in  
particular, or at the least provide evidence of a common class of  
issue that applies to similar designs.

> statistics (?) before you wrote this sir, so please retract the  
Please retract your accusation that mine is false.

See, we can both play this game, but it gets us no further.

> >> in load.  However, why risk it, when it only costs us 2 - 4% to  
> add
> >> a
> >> byte?
> >
One extra octet in a packet can indeed cause the threshold to be  
exceeded. In my own XMPP-related testing, 

> That has nothing to do with adding 1 byte or 2 - 10% in load.
It has nothing to do with percentage increase - nobody's actually  
demonstrated a case where that matters aside from aesthetically - but  
it has everything to do with optimizing tightly for very small packet  
sizes, since it's a concrete use-case where 

> (see the bottom of the post)
No, he clearly states "if you have a variable number of bytes for the  
length, there will be branches regardless". He also included code  
which shows one branch.

> I think you are confused.  Or have I missed something?

Yes.

You see, whilst there is a small overhead for a 63 or 64 bit length  
field width, in anything other than the mobile space, the overhead  
doesn't matter at all.

Within the mobile space, there is a small overhead for transmission  
of additional octets - and it is tiny - and no overhead for reception.

In terms of bandwidth, due to packetization, again, this additional  
overhead is not an issue.

The only case where it really matters is in order to keep packets  
below the FACH mobile threadhold.

Dave.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 17:31:41 -0400

> say I know how to find it.

I didn't send you a private message.

>> possible layers that could be placed underneath the higher layer.

I was giving an example of the specific use case, there are others.

> (and the concepts that preceded it)

Probably much more than me too.

> and they were precisely the desire

Abstractly, it was a choice to insure+restrict functionality at the lower
layer which could provide some benefit to the higher layer, but thus
limiting the features that the higher layer could achieve if it were not
constrained to a set of lower layer choices:

http://www.ietf.org/mail-archive/web/hybi/current/msg03607.html
(see bottom)

Such conflation will actually lead to increased complexity later.

What would have been wiser would be to have an Upgrade feature in the base
protocol, then first handshake on the base and then Upgrade to bind the
higher layer.  In that way, full orthogonality would have been retained. 
Users could cross-match as they desired.

below...

> be arguing that that's a special case, but I'm not convinced.

Actually the channel binding choice was a poor one.  I just told you above
a better way to do it. Sorry I wasn't there in 2007.

> I don't see how this follows at all.

You don't see that non-conflation is always superior?

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 22:41:21 +0100

> but it is not FUD to say we don't know for sure our real world  
Sure we don't. But we do have some statistics, and we also have data  
from other realtime protocols, like XMPP, if we want it. (It's harder  
to get, but I could run some off - we could also ask Google and/or  
jabber.org).

The issue is that in mobile, there's a concrete case for keeping  
small packets very small, to maximize the potential payload size. In  
most cases, a few extra octets in the packet really doesn't make much  
difference.

> monopoly. Why do you think JPMorgan funded Tesla?
I shouldn't, but it's a nice digression:

- NASA gave a $900k prize for power transmission, won in 2009, by  
transmitting 1kW over 1km, but this uses directional lasers, so not  
really an option.
- Intel powered a light bulb wirelessly in 2008, but this was close  
range and lost 25% of the power - it's not *much* more interesting  
than the inductive charging pads about now.
- Nokia have recently been charging mobile phones using ambient  
cell-tower energy, at the rate of roughly 5mW.
- None of these organizations have been prevented in experimenting by  
the power companies.

I'm pretty sure that if Tesla had figured something out, the power  
companies would have simply paid the royalties and done it. As it  
happens, the Intel experiment was what he was doing. Similar work has  
been demonstrated by Sony, amongst others, to power TVs wirelessly,  
and I think that's where that technology will lead, if they can make  
it reasonably efficient - in this day and age, I think a 25% hit is  
too much.

Nokia's work is also interesting - it might at least mean that  
standby times are vastly extended, and they might refine the  
technology to mean that even with an IDLE 3G session, the battery  
could remain at a steady-state. (My standby appears to take 10-15mW,  
thus it'd be currently extended by netween 33 and 50 percent given  
the current tech, as I understand it).

But there's nothing suggesting that the cell towers will charge a  
battery on an actively used handset unless you're either within a  
meter or two, or else it has a laser tracking you. Neither appeals  
much to me.

> I don't like to design for marginal edge case arguments.

Mobile isn't marginal or edge-case - it could *very* easily be the  
driving force behind WebSocket adoption. All you need is Nokia,  
Apple, and Android to deploy WebSocket capable phones, and for it to  
make a significant difference to mobile users, and it'll happen.

All three companies are represented here, and if we can sneak some  
webapps under the FACH->DCH threshold, we'll make a huge difference  
to mobile users, who very quickly learn what drains their battery and  
what doesn't, and adapt their habits accordingly.

Dave.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 17:48:11 -0400

Huh?

This may affect everyone who uses a scripting language like Javascript. 
We won't know until we implement serialization and try it.

> thought of, and you'll never be able to prove there isn't".

I did not use the word 'never'.

> That

I did. JAVASCRIPT.

I did not implement and test yet.  Did anyone?

> See, we can both play this game, but it gets us no further.

I did not falsely accuse you.

> exceeded. In my own XMPP-related testing,

Math helps here.

If the mean is 24 - 50 as John's use case suggests, then adding 1 byte
won't cross the threshold of 126+ for cell phone power.

And you are arguing for John's use case fiercely above.

> sizes, since it's a concrete use-case where

Where what?

Does 2 - 10% change anything?

What is wrong with having a 15-bit value with lowest CPU load?

It seems you are zealously focused on one measured use-case, with reckless
abandon of Murphy's Law.

Did you know that statistic are useless when you are not measuring every
possible signal that could occur?

It is called aliasing error. And it applies when your sample is skewed and
even when you don't know it is skewed.

What is wrong with a little bit of safety, in exchange for 2 - 10%
penality on network bandwidth, but with positives of drastic reduction in
CPU load and even reduction in network bandwidth for the 126+ case, and
even possibly lower network bandwidth overall due to CPU load issues we
can not predict entirely.

But as I said I don't care which choice the list chooses:

http://www.ietf.org/mail-archive/web/hybi/current/msg03609.html

I was just trying to help illuminate our choices.

> length, there will be branches regardless". He also included code

And "Option 8/16/32/64-bit" has a variable number of bytes. It says LenLen
in the header specification.

> below the FACH mobile threadhold.

Which I refuted above.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 17:52:13 -0400

> experience, nor experience of higher-level network protocol

If you wish.

It was more of a question, "why was I able to see this", since it was so
crucial to the design, you would think someone had already realized you
need serialization.

I think I was challenging people to be more open-minded.

So now you need to cut me down to size?

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 17:59:52 -0400

> won't cross the threshold of 126+ for cell phone power.

Plus 52 bytes TCP/IP overhead, plus 15-35 bytes for TLS if we go that route,
and suddenly we are quite close to that threshold.  Throw in some extension
data we are likely to add, and I do think every byte matters for small
frames.

You argue on the one hand that a byte doesn't matter for small frames, yet
you argue that a byte does matter for larger ones, and that a single compare
instruction is more important than either.  Given the fact that you didn't
even realize that any of the options would involve branches, I am not sure
why you are arguing so hard for what seems rather insignificant.

I really don't think it matters much among most of the options presented --
sure one option or another will be slightly better for different use cases.
 I think we should pick either of the two leading candidates (7/16/63 or
8/16/32/64) and call for consensus.  I prefer the former for saving a byte
in the small frame case, but the latter is also nice since it has extra
reserved bits for more expansion capability, so whichever is fine with me.

> in the header specification.

All of the options except the v00 one of a fixed 64-bit length have a
variable number of length bytes.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 15:07:37 -0700

> I didn't send you a private message.

 OK... I generally assume people can use google.

> I was giving an example of the specific use case, there are others.

Once again, I have no idea what you're talking about. As far as I can
make sense of your first message, you were arguing that:

" It is done above because the
desire to accelerate security outweighs the desire to be orthogonal to the
possible layers that could be placed underneath the higher layer."

To the best of my knowledge this was not one of the major motivations
that went into Channel Bindings.

> Users could cross-match as they desired.

To the extent to which I understand this suggestion, I don't at all agree
that
it's better. Let's take the specific case of IMAP with SASL over TLS. IMAP
has an upgrade, a  way to say "negotiate TLS over this channel" (STARTTLS).
Once the TLS handshake has completed, you resume IMAP over the TLS
channel. However, the problem is that while the client and server already
share an authentication method (namely, a password, GSS credentials, etc.)
they which they want to use for authentication, if they just do that without
checking the server cert there is a MITM attack. Channel bindings block that
attack.

The two major design options here are:
(1) Embed a complete traffic security system in the higher level protocol.
(2) Use something like channel bindings.

If you have some other proposal, I'd certainly be interested in hearing it;
this hardly the last time we'll need to solve this kind of problem. I don't
find the description above very clear, so perhaps you could provide some
sort of ladder diagram to show how you would address it.

Within the space of the two alternatives I mentioned above,
(2) has huge advantages, namely that it lets us leverage a tool like TLS
for every protocol instead of reinventing the wheel every time. Obviously
there are cases where it would be a worse choice, but there is a large
envelope of cases where it is not.

> a better way to do it. Sorry I wasn't there in 2007.

What you said above isn't clear enough for me to form any clear opinion on
whether
in fact have a "better way". Regardless, I think this message is a good
illustration
of why you're getting negative reactions from people like Dave Cridland,
Adam Barth,
and myself. The pieces of technology we're discussing have received quite a
lot
of attention from smart people and they've thought a lot about the best way
to
do it. It's certainly possible that you have a better set of ideas, but for
you to
handwave the way you're doing here and then take such a frankly dismissive
tone towards those efforts is, to put it bluntly, rude.

> You don't see that non-conflation is always superior?

Obviously not. Otherwise I wouldn't be arguing with you.

-Ekr

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 18:08:16 -0400

> to get, but I could run some off - we could also ask Google and/or

But we don't have enough data with a complete serialization in amateur
scripting environments, which will also run on cell phones.

Maybe the CPU load factor will end up being more important, and 15/63-bit
has a lower CPU load.

Too much handwaving on all sides, and not even hard data.

> small packets very small, to maximize the potential payload size.

Agreed.

> In

Thanks for admitting that.

> - None of these organizations have been prevented in experimenting by

Apparently Tesla used VLF?  He tapped into the natural resonance of the
earth to gain efficiency?  Wait I hear David Rothschild knocking at my
door...oh sh8t... :D

> companies would have simply paid the royalties and done it.

Once you let that cat out the bag, power becomes free.  The control of
power distribution is central to the control of fiat and center of power
on earth.

> As it

It is too high frequency.  Inefficiency increases with frequency.  It is
as if they want to do it wrong on purpose?

> the current tech, as I understand it).

I think we can do better than that if we can get his work that was
destroyed in the fire.

> meter or two, or else it has a laser tracking you. Neither appeals

Maybe so, but I have my doubts.  It is all about resonance in the ether. 
The resistance drops to 0.

> driving force behind WebSocket adoption.

I wasn't referring to mobile.  I agree.

I mean the frame rates measured by John.  I don't think those are
necessarily representative of what will happen.  I see WebSockets used for
low quality interactive video perhaps or other things we haven't thought
of yet.

But my point still applies, which is if the frame sizes get tiny, then the
CPU load factor becomes prominent.  So if we choose 7/16/63-bit and the
implementation decides to set maximum size at 126 to avoid the CPU load of
the conditionals, then it forces a refragmentation of the large messages
to 126 byte frames.  May never happen, but i will be laughing at all of
you if you choose it and it does happen.

> All you need is Nokia,

It will happen.

> what doesn't, and adapt their habits accordingly.

I was not arguing not to.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 23 Aug 2010 23:20:58 +0100

> low quality interactive video perhaps or other things we haven't  
In general, it's best to work with the data you have to construct  
your argument, rather than construct your argument then discount data  
which doesn't fit. At least then, you have a good chance of meeting  
some known use-cases.

> CPU load factor becomes prominent. 

It's not clear that's the case. As the payload size decreases, the  
processing of the packets will increase WRT the processing of the  
payloads, this much is true, but it's not clear that WebSocket  
headers would be significant over other factors in packet processing,  
let alone dominate, and in particular:

>  So if we choose 7/16/63-bit and the

.. this would obviously be a false economy, as it would increase the  
number of headers to avoid one single conditional jump which would be  
undetectable noise. I'm apparently more confident than you in the  
sanity of stack implementors.

Dave.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 18:26:11 -0400

> data we are likely to add, and I do think every byte matters for small

So then 1 byte is not that critical if you are that close to threshold,
you already toasted.

Developers are going to have to do special design for mobile, and really
get their frame sizes down.  Our 1 byte won't matter.

> you argue that a byte does matter for larger ones,

I did not argue that a byte matters for larger ones, I simply mentioned
all the characteristics of each option, so readers can compare.

I said the 15/63-bit saves one byte from 126+.  Those are still small and
under your mobile limit!

> and that a single
> compare
> instruction is more important than either.

Maybe it could be if the frame sizes are super tiny and sent in high
volume.  I don't know everyone's hardware out there.

>  Given the fact that you didn't
> even realize that any of the options would involve branches,

Huh?

I realized the options had conditionals.

I didn't spend any time thinking about the LenLen, I noticed late last
night before I went to sleep and then I was focused this morning on other
issues.  I let you raise that one, as I specifically asked for help
comparing that one in my first email this morning when I woke up.

If you stoop to attacking the person instead of the facts, then our
ability to get useful work done will degrade.

> I am not sure

I am not arguing so hard.  I already wrote that I don't care which one we
choose.  I am presenting all the options in order to try to end the logjam
on concensus.  And trying to not have a bias and help us.  Why are you
arguing so hard?  Don't answer.  Lets get back focused on the facts.

>
> I really don't think it matters much among most of the options presented

Probably not.

> 8/16/32/64) and call for consensus.

8/16/32/64 has too high of CPU load, uses an extra byte.  The only
redeeming advantage is that it gives us more extension bits.

If we need more extension bits then we could use 7/16/63 with an extra
byte, it would be more efficient.

So I really can't understand who is voting for 8/16/32/64??? I wish they
would present some logic so we can understand them.  I think you instead
conduct a non-anonymous poll here on this list with the 3 top choices.

We need to know who thinks what and why.

>  I prefer the former for saving a byte

See above.

> variable number of length bytes.

Not if maximum size is set.  You keep forgetting that.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 18:47:20 -0400

>> >> > On Mon, Aug 23, 2010 at 11:45 AM, Shelby Moore
>> <[email-masked]>
>> >> > wrote:
>> >>
>> >> desire to accelerate security outweighs the desire to be orthogonal
>> to
>> >> the

Below I was describing what I am talking about.

> has an upgrade, a  way to say "negotiate TLS over this channel"

No. I meant you need the Upgrade in TLS.  In the lower layer.

After you connect and handshake in TLS, then you Upgrade to IMAP.  That is
so much better and not conflated.  Then any one can mix and match without
needing the feature in the higher layer.

It is what we are going to do in HTTP, Uprade to WebSockets.

> checking the server cert there is a MITM attack.

If you already have a secure channel, then how can upgrading to IMAP ever
be insecure?

> Channel bindings block

Maybe I don't understand the problem sufficiently well, but the above is
my suggestion.

If the above is wrong, I can take some more time to study the issues.

> envelope of cases where it is not.

If 1 and 2 are the only choices, then agreed that 2 is superior.

> and myself.

Okay that was a poor choice of words on my part, but I am getting a little
bit flustered at getting attacked continuously.  I have done a pretty good
job of retaining my cool.  I am also tired.  Put a lot into this for 3
days.  And apparently at least I did one important thing for the CML.  So
that is good.  Won't feel bad when I quit to get back to real work.  I can
stay here long, so I have to work like mad for the little time I have
here.

> The pieces of technology we're discussing have received quite
> a
> lot
> do it.

No doubt there are many smarter people here than me.  What is frustrating
me is the closed-mindedness with respect to input.  Thomas was
ignored/brushed aside on CML issue, until I carried the torch on his
points.  What if I hadn't been here?

> It's certainly possible that you have a better set of ideas, but

Not my ideas.  I am just implementing the overriding doctine to not
conflate ever.  It is like a religion for me.  I want people to be free. 
I want to be open-minded to all possibilities.  I am not ruled by a small
dataset.  If that is handwaving, then I am guilty as charged.

I will make mistakes, and that is okay.

> for

Okay but I doubt that is something that I can convince you of.  I already
did one on the CML.  That was my lucky feat.  I don't know if I can do
another big one. I am just average guy.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 23 Aug 2010 16:08:04 -0700

> needing the feature in the higher layer.

That may well be desirable for other reasons but it doesn't solve the
problem at hand,
which is avoiding separation/MITM attacks between the application and TLS.

> If you already have a secure channel, then how can upgrading to IMAP ever

What you have in this case (or at least you would like to have) is a TLS
connection
to a counterparty with an unverified certificate. You can all that TLS
connection "secure"
if you like, but what it really is confidential and integrity protected but
anonymous.

You want to bootstrap up your other credential in order to validate the
server without
having to check his certificate.

[...]

> Maybe I don't understand the problem sufficiently well, but the above is

I fear it doesn't work, for the reasons I indicated above.

> bit flustered at getting attacked continuously.

I'm sure you perceive yourself as being attacked continuously, but that's
not
how I see it. I would encourage you to go back and reread the messages you
have written with a clear eye and ask yourself whether an unbiased observer
would think you were being clear and polite.

> points.  What if I hadn't been here?

And here we go again. Shockingly, people don't like to be called
closed-minded.

-Ekr

---

From: John Tamplin <[email-masked]>
Date: Mon, 23 Aug 2010 19:08:20 -0400

> arguing so hard?  Don't answer.  Lets get back focused on the facts.

With you posting continuously, frequently multiple times in the same thread
before anyone can answer, there is no time for others to reply.

> redeeming advantage is that it gives us more extension bits.

Too high a CPU load?  Where did you measure it?  How many ns is too many?
 What is the proper trade off between an extra instruction processing a
header and extra bytes on the wire?

> conduct a non-anonymous poll here on this list with the 3 top choices.

Well, you can see people who presented the idea and people that gave +1
before I created the Moderator poll.

I prefer 8/16/32/63 to 15/63 because if I am going to spend the extra byte
on the most common frames, I would like to get something for it.  Future
extensibility has value, and if we have 8 reserved bits then we could also
leave one for private-use, which would better allow experimentation with
future extensions.

I believe you are the only one suggesting that you wouldn't bother to check
the incoming length values if a shorter maximum frame size were negotiated.
 If you do check, then you have the same conditions.  There are lots of
attacks that are present because differing levels of validation -- for
example, a typical HTTP Request Smuggling attack is to have two
content-length headers.  One entity uses one of them, and another uses the
other, which allows something that would have been filtered through to the
second.  This is a real problem, and you have consistently ignored it when
it has been brought up.  So, I think this spec should require that
everything be validated.

You keep repeating the same argument and people keep giving the same
responses.  This isn't progressing anywhere, and saying the same thing again
louder isn't going to move it anywhere.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 19:09:40 -0400

> which doesn't fit. At least then, you have a good chance of meeting

At least 2 others agree that real-world data may be larger:

http://www.ietf.org/mail-archive/web/hybi/current/msg03618.html

> let alone dominate, and in particular:

You've got the logic wrong.  As payload decreases, the proportion of the
header CPU load increases WRT to total CPU load.

> sanity of stack implementors.

Incorrect.

It would not increase the headers if the frames were tiny size already.

My game theory logic is correct.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 20:11:24 -0400

> if you like, but what it really is confidential and integrity protected

The higher layer needs to interact with the lower layer in terms of
guaranteeing that the ends points don't change before the ends can be
verified by the higher layer?

In any case, the way to unconflate, is that the functionality that is
needed from the lower layer, should be abstracted so that any lower layer
with that functionality can be bound with the higher layer.

I am a bit sleepless right now (5am here been awake whole night since
yesterday at 10am, we are going on 3+ days like that), so I will need to
familiarize myself with the details more, then maybe I will be able to
have a more specific idea for that specific case. Or maybe I will discover
that channel binding is that abstraction.  I don't have too much time to
spend on this, but maybe I will convince myself it is worth trying.

> I fear it doesn't work, for the reasons I indicated above.

The only way there is not an abstraction, is if the design of the existing
things being used were conflated in their design.  Well to be fair, any
Turning complete machine is going to have conflation.  Only a pure Lambda
calculus can get you away from it.  So you are correct that eventually you
have to conflate.  I should have admitted that.  Goal is to minimize it
and localize it (but that will just push it into other places).

So I do relent that perfect avoidance of conflation is impossible.  But I
really don't like to hear that it reduces complexity.  If possible we
should strive to localized where side-effects propogate.  See the Haskell
State-monad:

http://copute.com/dev/docs/Copute/ref/Functional_Programming_Essence.html#OOP

(that is my site)

>> >> > Well, I wasn't saying that we needed to do channel bindings but
>> rather
>> >> > that
>> > What you said above isn't clear enough for me to form any clear
>> opinion
>> on
>> > whether
>> > in fact have a "better way".

I may be able to find a clever abstraction.  I may not.  I need to study
the issues more to see what was the existing conflation being worked with.
 I need more time to study it.

>>> Regardless, I think this message is a
>> good
>> > illustration
> I'm sure you perceive yourself as being attacked continuously,

Not too much, but I did get a couple of private emails that were basically
telling me that I don't have free speech.

I also got a couple of "bravo!" messages.

But I did feel that feeling I get when I know people are not understanding
me.

> but that's

Oh I do that and that motivates me to write more, because I realize I did
not make many issues clear.  I cringe when I read something I wrote and it
doesn't convey what I was thinking. Happens too often, no matter how hard
I try not to miscommunicate.

Polite is really hard to quantify, because at the extremes it means
express no strong opinion that goes against any other person listening.

I felt really strongly about the CML, because that was one that was so
clear to me.  So I pushed really hard on it.  Because I had told myself
that today was my last day in this list, unless the list agreed to allow
my tunnel+port.  I gave myself this day to get that impossible goal done,
and I failed.  So now I have to leave with my tail between my legs.  My
fault for trying.  I should know better.

I am also still frustrated from learning yesterday that the Linux people
think that Windows people should become like them.  I want to see
engineers who are humble and respect that users don't want to.  I guess I
feel this big pressure that I will have to go off and roll my own solution
for the users, and I was really hoping that just for once I could get it
done through a group.

I am frustrated that every "secure" website in the world is a lie.That
makes me feel really depressed. And it seems we are not even trying to fix
it. There is so much we need to do. It can't happen fast enough. I am
impatient.

>> > of attention from smart people and they've thought a lot about the
>> best
>> > way
>> > to
>> > do it.
>>
>>

==========Nothing important follows==========
I don't like being called many things that I was called, especially in
private from 2 or 3 people in this list yesterday and back when I was
discussing security in the tunneling+port.  But I did try to listen to
their frank feedback and the first thing I did this morning was try to
write a more coherent summary of my prior points.

I still don't know why Thomas was ignored and brushed aside.  He made his
point several times:

http://www.ietf.org/mail-archive/web/hybi/current/msg03538.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03535.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03525.html

Well actually now I know, and I must apologize.  I just read those posts
again, and in fact Thomas did not present the solution in the same way I
did.  He punted.  It was so obvious to me that I guess I was reading that
he had made it obvious, but now I see he didn't.

Okay I must you tell you that I am very mild autistic (blind in one eye
too), and I only score 90th percentile on verbal tests, except for
comprehension where I am in high 90sth and my IQ is very narrowly focus in
"visual mathematical conceptual" realm (and maybe hard to measure), what
ever that is.  So it means I guess that I see things and I don't realize
that i am thinking them and that I am not communicating them.

Thus I struggle with disconnection between my ability to think and
communicate effectively.  When I try to write what I am thinking, it reads
to me as if it makes 100% sense, but then when I read it the next day, it
is not coherent. So you can see I am very poorly equipped for the academic
or political realm.

So I am sorry okay.  I am doing the best i can with this condition. I am
actually very nice person and many people like me in person.  I don't know
why i can not do well in written forums, I guess it accentuates it.

Maybe it is just that I am impatient and trying to do too much too fast. 
Or maybe it is just that I am really a jerk and I don't realize it, but if
so then people would hate me in person, but that is not the case. It must
be somewhat personality based, so I am not denying that.

Or maybe this is just the way forums are.

Any way, sorry.

I am frustrated lately that we can't do more to let browser clients talk
to each other directly.  And I have a really big reason.  It has to do
with our ability to survive a global meltdown coming. I think the servers
are going to be controlled by the cancer.  So I feel an extra sense of
pressure lately. We don't have much time.

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 20:47:50 -0400

> before anyone can answer, there is no time for others to reply.

I am merely replying to your emails.  And maybe 2 - 3x per day, I had to
reply to my own email to clarify an omission.

For example, I had to reply to you 3 more times than necessary because you
wouldn't grasp a very simple concept:

=================================
http://www.ietf.org/mail-archive/web/hybi/current/msg03517.html (1st time)
http://www.ietf.org/mail-archive/web/hybi/current/msg03545.html (2nd time)
http://www.ietf.org/mail-archive/web/hybi/current/msg03578.html (3rd time)

"If the receiver expects to be attacked, then checking the size field
guarantees nothing."

And then finally I got tired of repeating myself on 4th time:

> Not checking it means a sender that ignored the maximum frame size could
> get
> the receiver out of sync with framing boundaries.  Experience has shown
> many
> attacks to follow from such problems.

It doesn't matter if you check the size, the size may be a lie too. I
can't believe I had to say that 4 times already. Really you can't see
that?
=========================

> Too high a CPU load?

Okay I should rephrase that.  It has probably up to an order-of-magnitude
more load than 15/63-bit.

>  Where did you measure it?

I can see that order-of-magnitude the pseudo-code, can't you?

>  How many ns is too many?

That is the wrong question to ask. What matters are ratios. Think it out.

> header and extra bytes on the wire?

I choose 2 - 10% over a 1000% ratio, if that 1000% ratio is dangerous in
any use case.

> before I created the Moderator poll.

Okay I will go back and see if I can deduce their logic from their posts. 
I doubt it.

To have concensus, we need to know their logic.

Blind concensus is a punt.

> on the most common frames, I would like to get something for it.

But I already told you before, you can just add an extra byte to 7/16/63
and get more reserved bits and less CPU load than 8/16/32/64.  (don't
forget maximum size again please when reading the prior sentence)

>  Future

See above.

> the incoming length values if a shorter maximum frame size were

Do I have to repeat this for the 5th time?

Why should I if the maximum size is set accordingly?

> it has been brought up.  So, I think this spec should require that

That is nonsense, because what ever the intermediary passes through, that
is what the next end will see.  You are conflating an HTTP issue which is
unrelated.

> louder isn't going to move it anywhere.

But I am correct and you are wrong, so who is causing the unnecessary
dialogue?

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 21:27:23 -0400

>> And here we go again. Shockingly, people don't like to be called

What does it say about a person if they get offended when I say the list
has a state of closed-mindedness?  If someone says something critical
about the list, I would either agree or disagree in my mind.  But I would
not take it personally.  I am not the list.  The only way I would take it
personally is if the criticism was correct, I was the one who did it, and
I assumed it was directed at me personally instead of at the list.  But in
that case I should not get offended, because it was correct.  If it was
incorrect, I can blow it off, because it was directed at the list and
everyone knows it is incorrect (unless I assume the list is stupider than
me).  See there is no way I can get offended, because I think the list is
stupid, then the criticism is correct about the list.

You know what, you can never win with conflating yourself with a list
identity.  Keep your own independence.  I did not say any specific person
was close-minded (although I may have a few in mind, and they already know
because I told them in private).

So I just caught you in a pyschology trap. I didn't intend to, but my
logic works so much faster than the other parts of my mind, it figures it
all out before I do.  It is like it is on auto-pilot.

I do better sometimes with helping and also this shows I have done a lot
of thinking about how to achieve optimum "referential transparency"
(non-conflation):

http://hackage.haskell.org/trac/ghc/ticket/3630

---

From: "Shelby Moore" <[email-masked]>
Date: Mon, 23 Aug 2010 22:32:28 -0400

[snip]

> I am a bit sleepless right now

[snip]

> things being used were conflated in their design.

Okay I tried to figure this out before I sleep (I'm half delirious), and I
think I got the solution.

Actually I was very close with the Upgrade idea, and then the further
comments above.

Afaiu, what channel binding is doing is compositing lower layer that
encrypts the channel (that may or may not have its own authentication)
with a higher layer that has its own authentication. The desire is to use
the higher layer's authentication with the lower layer's encryption.

The ideal solution is that the authentication should be in its own layer
that sits between the two aforementioned layers.  The reason you were not
able to do this, is because those higher layers (e.g. IMAP, IPSEC) are
already conflated with authentication, so you have no way to efficiently
separate out the authentication.

In the ideal way, one would first connect with say TLS, then Upgrade to
the Authentication protocol, which would interact with the encrypt layer
in a standard API, and then the higher layer would interact with the
Authentication layer through another standard API.

In that way, the bindings are generalized to work with any protocol that
supports the standard API.  Mix-n-match composition is fully generalized.

Whereas, for channel binding, a new binding has to written for each
permutation of mix-n-match.

The protocols selected would be done through Upgrades from each lower
layer.  In that way, the higher layers are never bound to a lower layer by
name, only through a standard "nameless" API.  In other words, the API
does not reveal to the higher layer what is the identity of the protocol
in the lower layer.

I stand by my earlier assertion that never is conflation required, except
by prior conflation.

I stand by my strong admonishment not the conflate.  It is fundamental to
optimal design.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 24 Aug 2010 09:17:42 +0100

Forgive me, I know this is going way off topic, really.

> Authentication layer through another standard API.

FWIW, this is more or less how it happens, just with an ordering  
change.

One first connects with the application protocol (and there are  
several we might choose, such as XMPP). Next, a layer insertion  
happens, inserting TLS. Next, a second layer insertion occurs,  
inserting SASL. The SASL layer may communicate with the TLS layer for  
authentication or channel binding - both are abstract concepts, and  
could be done with an abstract API. Finally the application layer  
continues.

If you accept the conceit of layer insertion - and why not - then you  
can use the existing layer model just fine. Adding layer insertion  
also helps conceptually with compression techniques like XEP-0138 or  
RFC 4978, which insert a compression layer into the stack.

However, it's important to recognise the distinction between a model,  
which is a method for people to understand and discuss particular  
areas of the overall functionality, and the reality.

In reality, the authentication provided by TLS, if any, is mediated  
by the application - which controls authorization, and therefore  
needs to ascertain whether the TLS credentials (typically an X.509  
certificate) can be used to authorize the session. Similarly, SASL  
communicates back to the application protocol to translate from a  
username to whatever the application deals in (which can be similar  
or wildly different - for XMPP, a Jid; for LDAP, a DN). So as long as  
you don't look too closely, the layer model holds, but in many areas,  
the boundaries are blurred.

There's two important things to remember, though:

1) In abstract terms, the layers exist, and are reusable. Thus  
generic TLS and SASL libraries can, and do, exist, and applications  
can use them.

2) But the layer model is just a model; it is there for humans, and  
doesn't produce any inherently better overall solution. By limiting  
the amount of knowledge required at each layer, though, it allows us  
to work around human limitations and achieve good solutions via  
combining specialized expertise. So I get to treat TLS as a  
"magically secure" thing for exchanging certificates and "doing  
security", and TCP is "just a pipe".

If anything, it's the human expertise that really follows the layer  
model - as an application protocol guy, every time I need to know  
something new about TLS or TCP in order to properly code a client or  
server, that's the kind of layer violation that I really care about.

Of course, the worst layer violation is the one where we force the  
user to have to have significant understanding of some portion of the  
stack. (We do this far too often, by trying to explain X.509 strong  
authentication to users in Firefox, and trying to explain the  
intricacies of unauthenticated addressing in internet mail).
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Dave Cridland <[email-masked]>
Date: Tue, 24 Aug 2010 09:19:29 +0100

> header CPU load increases WRT to total CPU load.

That's precisely what I said, just rephrased.

What I additionally said was that it wasn't clear that in such a  
case, the WebSocket header CPU load would dominate.

Let's suppose that total CPU load is, to make up some equations, the  
sum of T (TCP/IP), S (TLS), W (WebSocket), and P (Payload).

It is absolutely true that as the size of the payload decreases, P  
will reduce to near-zero, and the other three will dominate.

However, it does not follow that W will dominate T and S, even if S  
is zero (ie, no TLS), and even if T is reduced by piggybacking  
multiple frames in a single TCP packet.

Dave.

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 09:32:11 -0400

[snip]

> client code once the last frame is received.

Yeah so you already have to write glue ('transistion') code any way!

http://www.ietf.org/mail-archive/web/hybi/current/msg03582.html

> The JS API, in its current form, eliminates many of those options.
> It was also representatives of browser makers who were arguing against
> fragmentation because of receiver buffer management issues, and adding
> CML removed those objections.

My response (please click and read):

http://www.ietf.org/mail-archive/web/hybi/current/msg03634.html

I will expound on the above link.

It is beyond stupidity to force contiguity requirement from poor design
out on to the entire network!  Cripes, what are you thinking?

Am I not clear?

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 09:52:18 -0400

> Forgive me, I know this is going way off topic, really.

The debate is very relevant I think. And this thread is only about Layer
models, so if any one isn't interested, they can ignore.

> If you accept the conceit of layer insertion - and why not -

That is infinitely worse than an ordering change, because the higher layer
now chooses the lower layer, i.e. fundamentally conflated.

The higher layer should know nothing about the lower layer, that isn't
abstracted through a standard, layer-agnostic API.

> then you

If it had been correctly, your standard API for the transport (where the
TLS gets inserted via Upgraded from TCP) should accomodate and filter,
e.g. compression.

> areas of the overall functionality, and the reality.

That is an excuse to not follow the optimum design model.  Not wise.

> In reality, the authentication provided by TLS,

Authentication should never have been conflated into TLS layer.

> if any, is mediated
> certificate) can be used to authorize the session.

Authorization should never have been included in a layer that does
anything but authorization.  The layers should interact via standard,
layer-agnostic APIs.

> Similarly, SASL
> 1) In abstract terms, the layers exist, and are reusable.

Through ad hoc APIs that are not consistent between each composited
permutation.

Spaghetti.

I will not read further below...that is enough already.

> Thus

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 10:18:33 -0400

> multiple frames in a single TCP packet.

More conflation.

It depends on the ratio of the W frame size to the T packet size, which is
why I had said "when the frame size gets tiny".

And remember that T is highly optimized over decades of industry
investment, probably running in an ASIC, with clever algorithms given
decades to optimize.

Also remember an internal highly reliable fiber network could be running
with some huge customized TCP packet size, and that could be re-assembling
(re-fragmenting inversion) the packets incoming to optimize to the maximum
frame size of the next hop.

Do not conflate design by assuming your lower layer costs can't be
optimized away.

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 10:37:39 -0400

> [snip]
>
>>> ISTM that the most of the other constraints will come from the
>>> structure
>>> of
>> are
>> going to allocate an entire buffer for the incoming message (granted
>> some
>> of

Also, afaics there is a very simple solution to this for the JS API.

At the application layer, JS interface layer needs to add a size field at
the start of the string!

The frame transport layer can have a standard API for ReAssembleSize(). 
After JS interface layer reads the size field, it can call
ReAssembleSize().  Thus, typically only a small (negligible) quantity of
memory will have to be copied.

And if JS wants to eliminate that negligible inefficiency, then they can
write their API to accept strings which are non-contiguous in memory.

I hope I am convincing you all with my examples of unconflated design,
that there is always an optimum way to not conflate:

http://www.ietf.org/mail-archive/web/hybi/current/msg03636.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03628.html

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 24 Aug 2010 17:21:58 +0200 (CEST)

On Tue, 24 Aug 2010, Shelby Moore wrote:

(I changed subject to make it clear I've switched to meta-mode to just briefly 
mention what I think has become a problem on this list.)

> I hope I am convincing you all with my examples of unconflated design, that 
> there is always an optimum way to not conflate:

I think you've primarily flooded the list making it impossible for casual 
readers to follow along and your posting of more than 100 mails on this list 
in less than 7 days also makes it very hard for people to join the discussion 
and participate.

If there's silence and no opposition anymore in a subject now, it may very 
well be due to pure mail exhaustion rather than a silent agreement.

I think it is a good and sensible mail etiquette to only respond once or twice 
or so in a mail thread and then give everyone else 8-12 hours to reply before 
you return and post again in a subject.

-- 

  / daniel.haxx.se

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 11:52:23 -0400

> I think you've primarily flooded the list

Every single post I made has added information or clarified a mistake.

You are arguing for me to not work hard to help add information.

> making it impossible for casual
> readers to follow along

Impossible?

That is your problem if you don't have enough time to keep up with the
fast pace of the list.

You expect me to slow down my rate of help to least common denominator of
reading speed and allocated time.  That is communism.

> and your posting of more than 100 mails on this
> list

It doesn't stop you from posting.  How dare you use the list as an
authoratarian force to control how much information a contributor can add
to the knowledge base of the WG.

How dare you criticize my hard work with that illogical nonsense.  You
should be ashamed of yourself.  See why below...

> you return and post again in a subject.

The argument you are making for interactive dialog is utter nonsense, due
to the reality of timezones.

I woke up in the morning, and I answered all the emails from the prior
day.  Maybe you haven't noticed, but I am a rotate schedule from the rest
of you in USA and Europe.

And what is wrong with answering all the points that were made during my
absense?  I did wait 8 hours, I was sleeping.

And you were one of the people emailing me in private telling me that I
don't have my free speech rights.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 24 Aug 2010 18:09:23 +0200

> well be due to pure mail exhaustion rather than a silent agreement.

confirmed!

Willy

---

From: "Anne van Kesteren" <[email-masked]>
Date: Tue, 24 Aug 2010 20:49:11 +0200

On Tue, 24 Aug 2010 17:21:58 +0200, Daniel Stenberg <[email-masked]> wrote:
> I think you've primarily flooded the list making it impossible for  
> casual readers to follow along and your posting of more than 100 mails  
> on this list in less than 7 days also makes it very hard for people to  
> join the discussion and participate.

I certainly stopped contributing because of that. Also:

http://www.glazman.org/weblog/newarchive/2003_01_05_glazblogarc.html#s87008479
http://ln.hixie.ch/?start=1041735552&count=1
http://ln.hixie.ch/?start=1132620717&count=1

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 15:00:59 -0400

> http://ln.hixie.ch/?start=1132620717&count=1

Drag up something from 7 years ago.

Build your political case, that is what you excel at.

In meantime, I am focused on engineering.

IETF = Internet ENGINEERING Task Force

Wasting our time on petty bickering.

If you stopped contributing, you have no one to blame but yourself.

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 15:03:19 -0400

> If you stopped contributing, you have no one to blame but yourself.

Also about the 2003, where is XBL today?  I was correct.  It conflated the
style with the execution layer.  And it failed in market place.

Thanks for reminding me that I was correct in 2003 also.

---

From: gustav <[email-masked]>
Date: Tue, 24 Aug 2010 21:07:46 +0200

On 24 August 2010 21:00, Shelby Moore <[email-masked]> wrote:

> You got nothing better to use as argument in your quest to make websocket
better ?.
But i can (almost) understand that,  Shelby is using technical sound
reasoning based on decades of experience, it might be frustrating for some
that just cant compete.

regards
  gustav trede

---

From: John Tamplin <[email-masked]>
Date: Tue, 24 Aug 2010 15:10:24 -0400

> Drag up something from 7 years ago.

Not having been involved in those discussions, a brief perusal indicates
your tactics haven't changed and the same tactics are likely to get the same
result.

I have tried to keep responding to you because there are nuggets of useful
comments buried inside mounds of unrelated things (such as insurance is
socialism, electric companies are suppressing Tesla's perfect solution,
etc), personal attacks, insults, bravado, etc.  But I am running out of
patience and I suspect many here already have.

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 24 Aug 2010 15:12:59 -0400

> patience and I suspect many here already have.

You called me "absurd" before I ever used the word "illogical" in reply to
you.

You started it the animosity, and you keep pushing and pushing.

The reason is because you all want me gone.

So be it.  I am gone.

Last email to the list.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 24 Aug 2010 21:34:43 +0200

> Wasting our time on petty bickering.

that's what you've been doing for at least one week of mixed useful ideas
and childish thoughts that are unrelated to the discussions here. Your mind
is so much twisted that half of your mails are replies to yourself. Please
start to *think* before posting and let other people understand what you
mean and extract the 80% useless bullshit that is wasting our time.

> If you stopped contributing, you have no one to blame but yourself.

You make me think about the guys who constantly make noise to harrass their
neighbours and then claim that they have never prevented them from making
as much noise too and that they went away on their own purpose.

Please be less selfish, respect the people on this list and the time they
spend working on something you'll use later. Welcome if you want to
contribute, but please stop harrassing contributors to get them out of
your way.

Willy

---

From: gustav <[email-masked]>
Date: Tue, 24 Aug 2010 21:35:57 +0200

On 24 August 2010 21:12, Shelby Moore <[email-masked]> wrote:

> Great job guys, hope your satisfied !.
I'm leaving the list too,  this cirkus has no place for me.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 24 Aug 2010 21:37:55 +0200

On 24.08.2010 21:03, Shelby Moore wrote:
> ...

ENOUGH.

And this applies to all people participating in these exchanges.

At this time, I believe the editor should make a concrete spec proposal. 
I'm sure all the data is there and all arguments have been exchanged.

Not everybody needs to be happy with it, but it needs to work. That's all.

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 24 Aug 2010 21:35:41 +0200

> The reason is because you all want me gone.

Thank you.

---

From: Brian McKelvey <[email-masked]>
Date: Tue, 24 Aug 2010 14:34:02 -0700

> 
> If there's silence and no opposition anymore in a subject now, it may very well be due to pure mail exhaustion rather than a silent agreement.
> 
> I think it is a good and sensible mail etiquette to only respond once or twice or so in a mail thread and then give everyone else 8-12 hours to reply before you return and post again in a subject.
> 

Seconded!  The volume of mail on this list has become exhausting and I cam barely keep up with it, even devoting a couple hours a day.  Filtering out the useful dialog takes a lot of effort and I've been afraid that we'll lose many well-qualified people because they just don't have the time available to keep up with it.

Brian

---

From: "Shelby Moore" <[email-masked]>
Date: Wed, 25 Aug 2010 05:38:21 -0400

>> If there's silence and no opposition anymore in a subject now, it may
>> very well be due to pure mail exhaustion rather than a silent agreement.
>>
>> I think it is a good and sensible mail etiquette to only respond once or
>> twice or so in a mail thread and then give everyone else 8-12 hours to
>> reply before you return and post again in a subject.
>>
>
> Seconded!  The volume of mail on this list has become exhausting and I cam
> barely keep up with it, even devoting a couple hours a day.  Filtering out
> the useful dialog takes a lot of effort and I've been afraid that we'll
> lose many well-qualified people because they just don't have the time
> available to keep up with it.

I have decided to refute this illogic.

Brian, it helps if you apply science when making a post to this list.

1) MEASURE: it is not factually correct for you to blame me for the email
overload, when in fact it is the repeated illogic by others:

http://www.ietf.org/mail-archive/web/hybi/current/msg03685.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03626.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03637.html
etc...

2) THEOREMS: anyone who expects a public forum to have a very S/N ratio,
is not very intelligient. Signaling theory tell us that when there is a
lot of noise amongst the brains of the population, there is only two
choices, reduce discussion or increase it. The S/N ratio will not increase
in the latter case. The latter is a degenerate case, and it has a well
known scientific name 'Communism':

http://www.ietf.org/mail-archive/web/hybi/current/msg03627.html

3) FALSEFIABILITY: if your allegation is not falsefiable, then it is not
science. I assert that your allegation is not.

4) PEER REVIEW: I have explained in this WG, that hacks that have been
used to create other standards, had non-conflated solutions, i.e.
channel-binding is really a bad direction and will eventually lead to
breakage. I know it may hurt the egos of some people here, but don't shoot
the messenger when you can't shoot the facts/truth:

---

From: "Shelby Moore" <[email-masked]>
Date: Wed, 25 Aug 2010 06:02:41 -0400

> http://www.ietf.org/mail-archive/web/hybi/current/msg03627.html

Futhermore, I suggest that the problems that have been encountered with
low S/N ratio in this list (which was evident before I arrived, due to the
interminable debate on issues), is due to following the wrong process.

The standards process should not be able creating new technology, because
a large group separated by email (and having other day jobs), is
inherently inefficient at such a task, and will make egregarious mistakes.

The correct standardization process is for successful protocols already
proven in the market be adopted as standards, possibly merging the best
features of several.

The problem we have is that if we throw this back to the vendors to
compete on in the market, it may be very difficult to get
interoperability.

However, I assert that we have already presented enough ideas in this WG
for the browser vendors to go implement their preferred flavor and then
let the server programmers support the version they like the most. Then we
come back here and identify who won in the market.

Competition brings out the best.  Communism brings out failure.

below...

> http://www.ietf.org/mail-archive/web/hybi/current/msg03636.html

Add this link:

P.S. <joke>given I am self-diagnosed very mild austistic savant, then I
don't think the IETF wants a lawsuit for discrimination against the
handicapped.</joke>

---

From: "Shelby Moore" <[email-masked]>
Date: Wed, 25 Aug 2010 06:24:47 -0400

> That may be so, but it helps to realise that it hurts the cause and in
> yourself when there is too much of harsh personal attacks involved.
> People might never be able to understand your point of view, or agree on
> it,
> but you can ensure they never will by attacking too harshly,
> then the focus shifts from the issue at hand, towards personal behaviour.

I don't disagree, but remember who made the first attack after I joined
this list:

http://www.ietf.org/mail-archive/web/hybi/current/msg03322.html

And now Firefox/Mozilla has agreed I was correct:

https://bugzilla.mozilla.org/show_bug.cgi?id=588704#c47

Competition reveals who was correct.  Communism hides truth.

Btw, I did not count how many times that John used words similar to
"absurd" in his replies to me, before I finally replied with such a type
of language. If necessary, I can go back into the archives and build a
case. But I assert this petty bickering over style of communication is
nonsense, because it will always happen amongst groups of humans.
Assigning blame is for people who have nothing better to do with their
time.

---

From: S Moonesamy <[email-masked]>
Date: Wed, 25 Aug 2010 03:25:40 -0700

Hello,
At 02:38 AM 8/25/2010, Shelby Moore wrote:
>known scientific name 'Communism'

The above topic is outside the scope of the HyBi Charter.  Please do 
not use the HyBi mailing list for discussions that are outside the 
scope of HyBi Charter.

Regards,
S. Moonesamy
HyBi WG Secretary

---

From: "Shelby Moore" <[email-masked]>
Date: Wed, 25 Aug 2010 06:36:15 -0400

> not use the HyBi mailing list for discussions that are outside the

I did not start this thread.

The thread is unfactual ad hominen attack against me.

The above is the way to defend myself scientifically.

---

From: S Moonesamy <[email-masked]>
Date: Wed, 25 Aug 2010 03:47:29 -0700

Hi Shelby,
>The above is the way to defend myself scientifically.

Could we please end this thread?

Thank you,
S. Moonesamy
HyBi WG Secretary

---

From: "Shelby Moore" <[email-masked]>
Date: Wed, 25 Aug 2010 06:57:39 -0400

> Could we please end this thread?

+1

Thank you!!!

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Aug 2010 03:12:03 +0100

John Tamplin wrote:
>    merits of optimizing framing below the TCP/IP MTU? In particular, what
>    is the motivation for optimizing the overhead of a 128 byte message
>    when even a 512-byte message will still only require a single packet?

Yes, there have been such discussions, but minimising packet counts
hasn't had much traction as an explicit concept.  It is, of course, an
implicit goal of both multiplexing and keepalive aggregation.  In the
case of multiplexing, it isn't the message size, so much as having
multiple application instances share packets when they are
simultaneously active.  In the case of keepalives, it's pretty obvious.

>    If the idea is that many frames could be stuffed into a single TCP/IP
>    packet when multiplexing is used, then it is probably better to
>    instead directly address the overhead of that multiplexing.

Effective multiplexing needs more than has been discussed recently.
It needs flow control, low-latency channel setup, keepalive
aggregation, opportunistic negotiation and and a few other things to
work.  Just squashing lots of messages into a single TCP/IP for
dispatch at the other end - applications can do that themselves
easily, and almost certainly will.  That isn't the goal of
multiplexing.

>    techniques similar to the ones used in modern SQL databases (e.g.
>    Oracle) for compression of data within a given block could be used:
>    create a frame header format that can describe multiple frames much
>    more compactly than it can describe frames individually, and send that
>    "multi-header" ahead of all the frames it describes. But, such
>    complexity should first have a clear concrete motivation.

By the time you get to micro-optimising like that, it's probably
simpler and just as effective to use generic byte compression, such as
LZO or deflate, on each batch of messages/frames as it becomes
available, or even on the raw byte stream.  That way nobody has to
standardise another (expletive-deleted) protocol :-)

>    One concern is mobile networks, such as 3G where the radio can send
>    small frames in a low-power FACH mode which supports around 128-256
>    bytes (depending on the carrier's configuration) including all
>    overhead. Â Further, the only traces of real data that have been
>    presented show that 98%+ of frames will fit under 126 bytes after
>    compression is supported, and it makes sense to optimize for the most
>    frequent case when it is that high a percentage.

It would be great to have a bit more detail about mobile networks.

For me, one of the goals of multiplexing is to improve the latency,
throughput and power consumption on mobile and similar networks.  It
is also a motivation to devise minimal-round-trip negotiation/setup
methods, as 3G latency varies enormously and can be quite large, even
when the bandwidth is being good.

With a mobile handset, it is feasible to force all WebSocket
connections to a single proxy on the other side of the wireless link,
which is the optimal situation for multiplexing - it can use a single
connection for all traffic, and a wireless-optimised TCP strategy too.

Do you know of mechanisms for delivering a packet to the handset when
it doesn't have the data connection actively open?  For example, like
the way it can receive an SMS without having a data connection open?

Some WebSocket applications will be "nearly always idle", for example
blog updates.  Those currently have to use regular keepalives (and
that comes to a lot, if you open many pages), just for the possibility
of receiving an update message.  If there is a way to establish that a
message will be received by the client without it actively holding a
data connection open, that would be immensely valuable.

Thanks,
-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Aug 2010 03:18:49 +0100

> make a significant difference to mobile users, and it'll happen.

I believe Nokia and Android phones are already WebSocket capable, if
someone application designer wants to use it, through the use of a
Flash component (Flash has scriptable TCP), relaying messages to the
HTML's javascript.

-- Jamie

---

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 22:24:27 -0400

> work.

But seems to me that can sit on top of the base framing/fragmentation
header layer, and can operate orthogonally to any extension layers?

+1

Couldn't you have one connection open to each server and multiplex all
traffic through it, so that you only have to keepalive on one connection?

---

From: John Tamplin <[email-masked]>
Date: Thu, 26 Aug 2010 22:25:37 -0400

> standardise another (expletive-deleted) protocol :-)

Your response lost the fact that the above paragraphs you are responding to
were from Brian Smth, not me.

---

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 22:27:45 -0400

> Flash component (Flash has scriptable TCP), relaying messages to the

I was thinking of that too, if don't like the way WebSocket ends up.

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Aug 2010 04:14:11 +0100

> Maybe the CPU load factor will end up being more important, and 15/63-bit

CPU load will be lost in the noise simply because any application
which really cares about a few nanoseconds per message and is sending
lots of tiny messages at high speed, should batch its messages
together internally instead.

There is no point micro-optimising something which has much better
alternative solutions readily available.

> the conditionals,

No sane implementation would cap at 126 to "optimise CPU load", if its
messages were large enough that they'd fragment, because it _wouldn't_
reduce CPU load, obviously.

Anyway, the CPU load of header parsing is utterly negligable compared
with the costs of handling the actual data and messages.

The one factor which affects CPU load more than any other for this
type of activity is external memory bandwidth, *not* calculation time.
And that argues for generally keeping things small if there's a high
rate of messages.

> low quality interactive video perhaps or other things we haven't thought

As the web is the substrate for "social networking", and as the
semantic web grows, and as crawlers and interactive agents are growing
to use services first designed for human use, and browser are becoming
high powered platforms (3d! compiled javascript! threads!), that are
now starting to run very well on even the humble phone, it's pretty
hard to predict what it'll be used for at scale.

In the immediate future I expect it used massively for in-browser
games and in-browser interactive applications, and for server-driven
updates to web pages, such as Facebook notifications, chat boxes etc.

-- Jamie

> power distribution is central to the control of fiat and center of power

(This is the wrong place to discuss it.  But, to be fair, it's one
thing Tesla is known for in relatively "mainstream" circles: His power
emitting tower was partly built, and lost the support of investors
when they asked "where can I put the power meter [for users]?")

[About Intel's wireless power transmission]
> Inefficiency increases with frequency.  It is as if they want to
> do it wrong on purpose?
[...]
>  It is all about resonance in the ether.  The resistance drops to 0.

No, efficiency depends on many factors, and Intel are not deliberately
trying to fail.  Intel (and others) are targetting short houshold
distances and small devices, for which VLF is not effective.  Tesla was
targetting towns, cities, and even larger resonant systems.  It still
doesn't drop to 0, although it's possible Tesla believed it could,
given the state of understanding at the time.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Aug 2010 04:31:27 +0100

> header layer, and can operate orthogonally to any extension layers?

Mmm..  sort of.  The primary requirement of opportunistic multiplexing
is that *independent users don't create problems for each other*.

We don't count affecting the timing as a problem - real interference
problems are things like deadlocks causes by a single stream's queue
filling up and blocking all others, and livelocks due to a large
unfragmentable message.  A nasty subtle one is where an application
can't deadlock itself, but can deadlock the collective if it's
production-consumption pattern of messages interacts in the wrong way
with other dependent producer-consumers elsewhere.

To ensure that, effective multiplexing has some requirements which may
or may not be provided by the base framing/fragmentation layer.  If
they aren't provided, that's fine, it just means multiplexing has to
do its own things below, rather than above, the base protocol.
There's nothing wrong with that, and it's preferable, if it means the
base layer is simpler.

I'm a big fan of the base layer being as simple as possible but no
more, with a selection of performance-enhancing extensions that
implementations can choose to opt into or not.  That way very simple
implementations are possible, alongside high performing and network
efficient ones.  (Just like the best of HTTP).

At the moment, I'm not sure if the base layer will end being something
that can be used as the "bottom" (above TCP) underneath multiplexing,
or if it'll be the other way around.  But I am sure it doesn't really
matter, because it'll work either way just fine.

The main thing is that there are places to negotiate and extend which
won't break when you try to use them with existing deployments, and in
such a way that it's unlikely to be rendered unusable by a wide
deployment of broken/buggy or even compliant endpoints and proxies.

I'm really not sure how we're doing on that.

> traffic through it, so that you only have to keepalive on one connection?

Keepalive is expensive in mobile radio power, even for one connection.

If you have 10 pages open (I often have 5+ pages on my phone at the
moment; it's a Nokia), and each page has, say, 3 independent services
(advert updates, dynamic content, whatever) that's 30 connections,
which is a steady flow of radio packets just in keepalives.  Probably
use up your data allowance too :-) That's why it's better to funnel
them all through a single proxy on the other side of the wireless
link, if possible.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Aug 2010 04:43:29 +0100

John Tamplin wrote:
>    Your response lost the fact that the above paragraphs you are
>    responding to were from Brian Smth, not me.

I'm sorry.  It seems I simply cannot tell who is quoting whom in the
HTML mails that are being used here.  From my point of view, the
quoted text was rendered in a section somewhere under your name.  I've
had another look and I still can't tell who's writing what.  Lynx does
a poor job of rendering it.

-- Jamie

---

From: "Shelby Moore" <[email-masked]>
Date: Fri, 27 Aug 2010 06:42:38 -0400

> together internally instead.

Respectfully, I am reasonably sure that you are not aware of the case that
concerned me:

http://www.ietf.org/mail-archive/web/hybi/current/msg03700.html

Above I think you are not factoring that the receiver and sender may have
different priorities, see my link above for explanation.  I do agree that
the case I presented is not likely to happen.  But in engineering we have
to design for those 0.01% cases that are catastrophic.  That is why we
give 25 - 50% margins for our failure thresholds.

> alternative solutions readily available.

I agree, I was only stating that if all the frame size header format
options were roughly equivalent, then why not optimize away the case at
the ABOVE link.  Below link is where I agreed with your point:

http://www.ietf.org/mail-archive/web/hybi/current/msg03709.html

> reduce CPU load, obviously.

Again see what I wrote above.

I think you are not factoring that the receiver and sender may have
different priorities, see my link above for explanation.

> with the costs of handling the actual data and messages.

Again I am referring to PROPORTION load as it affects the absolute limit
on the number of cores that can be managed from the trunk shunt:

http://www.ietf.org/mail-archive/web/hybi/current/msg03677.html

It seems to be a very obscure phenomenon that most people are not aware of
YET.  I have spent innumerable hours having discussions at
Lambda-the-ultimate.com having discussions about STM and many aspects of
parallelization.  I have been thinking about it a lot for the design of my
new language Copute (only at the idea stage), which is designed to be a
Haskel for Javascript programmers (should run Javascript with minimal
changes):

http://copute.com/dev/docs/Copute/ref/
http://code.google.com/p/copute/issues/list
http://hackage.haskell.org/trac/ghc/ticket/3630

(that is not advertisement, I don't need, it is only so you can dwelve
into parallelization as you might feel it is relevant to this discussion)

> And that argues for generally keeping things small if there's a high

Agreed, but respectfully, that was not the case I was considering. Please
refer to my links above if you have time to go for deeper understanding of
my point.  I understand if it is not worth the time. We all must
specialize in core competencies.  I trust your judgement+experience on the
probability of my case being a problem.

> hard to predict what it'll be used for at scale.

Exactly, thank you:

http://www.ietf.org/mail-archive/web/hybi/current/msg03700.html

> updates to web pages, such as Facebook notifications, chat boxes etc.

Exactly how I am planning to use it.

> when they asked "where can I put the power meter [for users]?")

Thank you!

> distances and small devices, for which VLF is not effective.

Agreed.

>  Tesla was
> given the state of understanding at the time.>

Agreed!

John I hope you are reading this:

http://www.ietf.org/mail-archive/web/hybi/current/msg03667.html

As per what I told John in private, it doesn't mean we couldn't have
charged for it.  It may be possible soon/now to make energy so
cheap/abundant (nuclear breeder reactors) that it simply didn't pay to
cheat.

---

From: "Shelby Moore" <[email-masked]>
Date: Fri, 27 Aug 2010 08:05:30 -0400

> unfragmentable message.

Agreed.

You are articulating the PROPORTIONAL (relational) problem of
parallelization, but you are articulating it at the network scale, whereas
in totally separate topic (link below) I was articulating it at the
multi-core:

http://www.ietf.org/mail-archive/web/hybi/current/msg03767.html

>  A nasty subtle one is where an application

Exactly!  Parallelization problem.

Mathematically this will happen whenever there is an actor in your
state-machine that has or depends on side-effects outside of itself. It
can be expressed as a non-pure Lambda calculus, i.e. any function that is
not referentially transparent:

http://copute.com/dev/docs/Copute/ref/Functional_Programming_Essence.html#Lazy_Evaluation

This is why conflation is the worst evil mathematically.  I been trying to
explain/suggest this for 7+ years (and get labeled as nutcase/troll for
doing so):

http://www.ietf.org/mail-archive/web/hybi/current/msg03664.html

> There's nothing wrong with that, and it's preferable, if it means the

Agreed it is preferable.  The more thin the layer, then less chance of
referential transparency errors (conflation).

> more,

(are you reading my mind? hehe) I said that exactly in a prior post:

http://www.ietf.org/mail-archive/web/hybi/current/msg03417.html

"This is one of those cases where you make things as simple and optimized
as possible but not more simple and not more optimized."

> with a selection of performance-enhancing extensions that

+1

Fabulous!

> matter, because it'll work either way just fine.

The simpler we make the base layer, the better chance it can be reused by
the multi-plexing.  If we accidentally put something in the base layer
that breaks multi-plexing, then yes it is all wasted and multi-plexing has
to be shoehorned underneath it.

> I'm really not sure how we're doing on that.

Lets start a new thread to discuss that?  I have a lot of ideas on how to
simply things, and you have the technical knowledge of the issues.

> (advert updates, dynamic content, whatever) that's 30 connections,

No I mean 1 connection per page (server).  Multiplex all services through
the 1 connection.

> them all through a single proxy on the other side of the wireless

Oh I agree, even better if you can proxy at the base station intermediary!

Excellent.

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 30 Aug 2010 07:49:49 +0800

> a poor job of rendering it.

I have a similar problem.  I'm using Outlook, which is supposed to render HTML, though not the HTML produced by John and others (Gmail, I presume).  I too am left guessing.

--Martin

---

From: John Tamplin <[email-masked]>
Date: Sun, 29 Aug 2010 22:29:40 -0400

> I have a similar problem.  I'm using Outlook, which is supposed to render HTML, though not the HTML produced by John and others (Gmail, I presume).  I too am left guessing.

Sorry, I have switched to plain-text mode when replying to the list,
though then line breaks from other mailers show up poorly (ie, your
line above is one long single line).  Hopefully this will be more
readable after passing through the mail gateway.