Subject: [hybi] Adding multiplexing

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 04 Aug 2010 10:58:33 +0200

Let me state upfront that I agree with Ian and Maciej (and maybe others)  
that multiplexing should not be added right away. We should first see how  
the SharedWorker scenario plays out and study the problem more.

One way we can add multiplexing later is that the server indicates in its  
part of the handshake to the client that it would like to do multiplexing  
(this will ignored by conforming "legacy" clients). If the client supports  
it the client starts sending frames prefixed with a byte identifying the  
stream. Maybe with a special frame at the start to identify the details of  
the stream. (I am not quite familiar with what is needed exactly.)

This way framing stays very simple for clients and servers that do not  
have any requirements for multiplexing and complexity is only added for  
servers and clients that do. The expectation is that all browser clients  
(maybe not all clients) will support multiplexing but not all servers will.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 4 Aug 2010 19:44:39 +1000

Anne,

at the f2f (and before) it was argued that we want to have only one framing
mechanism.    Among the reasons for this is that we want to be able to add
features to the protocol without forcing intermediaries and infrastructure
to be updated each time a new frame mechanism is added.

I don't think anybody is advocating an immediate addition of multiplexing to
the protocol.   We are just using multiplexing as one of the use-cases to
evaluate the extensibility of the various proposals, so that if we do need
to add multi-plexing at a later date, then we can do so efficiently and
without breaking more infrastructure than necessary.

There are other extension that should be consider and I don't think we
should get fixated on multiplexing (as much as I doubt it, perhaps it can be
done by shared workers)  Compression and meta-data are other likely
extensions and Julian has even suggested that fragmentation could be done as
an extension.

If you look at the actual proposals, they are all very small to moderate
increments over what is currently proposed (give or take a fixed length or
length of length).     Most of them will have have no effect at all on
non-extended clients and servers. Essentially we are trying to agree an
extension mechanism rather than just having a broad statement that the
protocol can be extended.

regards

On 4 August 2010 18:58, Anne van Kesteren <[email-masked]> wrote:

> This way framing stays very simple for clients and servers that do not have
> any requirements for multiplexing and complexity is only added for servers
> and clients that do. The expectation is that all browser clients (maybe not
> all clients) will support multiplexing but not all servers will.
>
>
> --
> Anne van Kesteren
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 04 Aug 2010 11:56:42 +0200

On Wed, 04 Aug 2010 11:44:39 +0200, Greg Wilkins <[email-masked]> wrote:
> at the f2f (and before) it was argued that we want to have only one  
> framing mechanism.    Among the reasons for this is that we want to be  
> able to add features to the protocol without forcing intermediaries and  
> infrastructure to be updated each time a new frame mechanism is added.

I think it would be good to have a complete list of the arguments.

I do not really like changing the basic framing mechanism to have space  
for "theoretical" future features which might not even become used.

> increments over what is currently proposed (give or take a fixed length  
> or length of length).     Most of them will have have no effect at all on

It is not just a statement. It is actually true. It is however a different  
way from embedding everything into the "default" frames.

(Maybe this thread should be renamed to "extensions" then.)

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 4 Aug 2010 20:42:11 +1000

On 4 August 2010 19:56, Anne van Kesteren <[email-masked]> wrote:

>> framing mechanism.    Among the reasons for this is that we want to be able
>> to add features to the protocol without forcing intermediaries and
> I do not really like changing the basic framing mechanism to have space for
> "theoretical" future features which might not even become used.
>

We are not changing the basic framing mechanism to have space for a future
features.

We are changing the basic framing mechanism because the current one has not
achieved consensus in this WG and the f2f did indicate strong support for a
single framing mechanism.  So perhaps I'm getting a little ahead of the
chairs actually declaring this as a consensus decision, but it seams likely
that if we are going to reach consensus then we have to change the basic
framing mechanism. Once we are decided to change the framing mechanism, we
need to evaluate the choices and considering theoretical future features is
an entirely valid contribution to the evaluation of the alternate proposals.

But, I have also listed the current extension mechanism (hixiesque centrally
control op-code) as one of the alternatives - so we may decide to change to
a single framing mechanism that has the same extension features as the
current protocol.

> way from embedding everything into the "default" frames.

Nobody is proposing embedding everything in the default frames (OK the
robertoesque proposal could be unfairly described that way).

I really don't understand why you can't participate in the evaluation and
refinement of the proposals rather than simply questioning the validity of
having the evaluation in the first place and suggesting that no change is
required when there are clearly many voices calling for change.  The way the
game should be played is that you should try to adapt the proposals to
include the concerns of all, while staying consistent to their particular
design,  and thus attempt to identify a solution that is both a consistent
and satisfies the largest set of concerns.   This is not design by
committee.

regards

---

From: Arman Djusupov <[email-masked]>
Date: Wed, 04 Aug 2010 14:08:32 +0300

Anne van Kesteren wrote:
> that multiplexing should not be added right away. We should first see 
> how the SharedWorker scenario plays out and study the problem more.
> 
> One way we can add multiplexing later is that the server indicates in 
> its part of the handshake to the client that it would like to do 
> multiplexing (this will ignored by conforming "legacy" clients). If the 
> client supports it the client starts sending frames prefixed with a byte 
> identifying the stream. Maybe with a special frame at the start to 
> identify the details of the stream. (I am not quite familiar with what 
> is needed exactly.)
> 

Multiplexing requires just a channelID per frame and control frames for flow control and sub-channel management. 

I agree with you that multiplexing does not need to be implemented right now, but I also agree with Greg that it is best if it is supported on the protocol level rather than on the application level.

I will offer a use case for multiplexing on the protocol level. Instead of looking at the client side, I prefer to look at it from the server's viewpoint. So -- assume you have a range of 3rd-party services hosted on a cloud. All these services might be hosted on a pool of machines and exposed over a cloud endpoint (which obviously is not controlled by the vendor of these services). If there is a standard multiplexing mechanism at the protocol level then the cloud endpoing will be able to transparently demultiplex the virtual channels and relay them internally to the appropriate 3rd-party services within the cloud. Otherwise the cloud endpoint, being unaware of the subprotocols that the services might be using, will be unable to demultiplex and relay virtual channels.

Having multiplexing at the application level will end up in proprietary solutions which will lock in service vendors to particular cloud APIs. IMO WebSocket endpoints and intermediaries should be able to demultiplex and relay frames of virtual channels without having any knowledge about the applications that send them, so multiplexing should be an optional feature of the standard protocol.

Arman

---

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 04 Aug 2010 14:23:26 +0200

On Wed, 04 Aug 2010 12:42:11 +0200, Greg Wilkins <[email-masked]> wrote:
> We are not changing the basic framing mechanism to have space for a  
> future features.
>
> We are changing the basic framing mechanism because the current one has  
> not achieved consensus in this WG and the f2f did indicate strong  
> support for a single framing mechanism.  So perhaps I'm getting a little  
> ahead of the
> chairs actually declaring this as a consensus decision, but it seams  
> likely that if we are going to reach consensus then we have to change  
> the basic
> framing mechanism. Once we are decided to change the framing mechanism,  
> we need to evaluate the choices and considering theoretical future  
> features is an entirely valid contribution to the evaluation of the  
> alternate proposals.

I was at the F2F and my impression was that there was no consensus for  
sentinel framing and strong consensus for a single way to package data --  
i.e. length-based. And I think everyone is on the same line with respect  
to that now. I did not at all have the impression we were also discussing  
possible future extensions while we discussed this.

That is, we discussed

   START DATA END

versus

   START LENGTH DATA

and we did not discuss

   START LENGTH DATA

and also having

   IDENTIFIER START LENGTH DATA

in the future.

> But, I have also listed the current extension mechanism (hixiesque  
> centrally control op-code) as one of the alternatives - so we may decide  
> to change to a single framing mechanism that has the same extension  
> features as the
> current protocol.

Okay, maybe I misunderstood your email in the other thread.

> refinement of the proposals rather than simply questioning the validity  
> of having the evaluation in the first place and suggesting that no  
> change is
> required when there are clearly many voices calling for change.

I have participated in the other threads as well as can be easily seen  
 from the list archives.

My problem with those threads is that they are basically a list of feature  
requests without much justification. I and others have asked for use cases  
and more detail, but it has not really been presented to date. And when I  
ask for it here you say we already have consensus...

> The way the game should be played is that you should try to adapt the  
> proposals to
> design,  and thus attempt to identify a solution that is both a  
> consistent and satisfies the largest set of concerns.   This is not  
> design by
> committee.

I'm not asking for that.

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 4 Aug 2010 14:25:02 +0200

> control and sub-channel management.

It also needs (if you want to specify interoperable multiplexing)
rules for clean tear down of channels, handling per-channel errors,
handling connection level errors, etc.  We did all this in AMQP, and
it gets *complex*.

If you don't care about interoperability and just want space for
experimentation, define a binary frame consisting of length + channel
ID, specify that channel IDs are experimental, and that channel ID
zero is for control frames.  Then define all higher frames types
inside that frame.

But I'd ask: why are we not looking at SCTP[1] as the underlying
protocol rather than TCP?  We'd get all this for free and SCTP has
other significant advantages too, such as multihoming.  SCTP is pretty
easy to use, TCP calls map to it.  Running HTTP over SCTP would give
these advantages to old applications as well.

The point: while multiplexing may be beneficial to some scenarios, it
is probably flawed to try to use WebSockets to gain this.  There are
IETF protocols that already do this, they have been around for a long
time, they have been proven to work better.[2]

-Pieter

[1] http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol
[2] http://www.cis.udel.edu/~leighton/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 4 Aug 2010 23:12:12 +1000

On 4 August 2010 22:25, Pieter Hintjens <[email-masked]> wrote:

> protocol rather than TCP?
>

I'm certainly not saying that we should not consider SCTP. It may well be an
option.

The ability to support multiplexing is not the only criteria that we should
evaluate framing against and just because it can support it, does not mean
that we will adopt  it.  If better solutions emerge or if the need never
materializes, then we don't need to utilize the framing extension mechanism
for multiplexing.   But I would prefer to have the capability and not need
it rather than not have the capability and end up needing it.

regards

---

From: <[email-masked]>
Date: Wed, 4 Aug 2010 15:18:51 +0100

Do firewalls and NATs handle SCTP properly?

While it would be nice to see HTTP become a layer in its own right, rather than only tied to TCP:
http://sat-net.com/L.Wood/dtn/http-dtn
http over SCTP is still pretty much experimental:
http://tools.ietf.org/html/draft-natarajan-http-over-sctp
expired draft.

and only the Uni of Delaware gang care.

L.
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Pieter Hintjens [[email-masked]]
Sent: 04 August 2010 13:25
To: Arman Djusupov
Cc: [email-masked]
Subject: Re: [hybi] Adding multiplexing

-Pieter

---

From: Arman Djusupov <[email-masked]>
Date: Wed, 04 Aug 2010 17:43:24 +0300

> handling connection level errors, etc.  We did all this in AMQP, and 

I have implemented two multiplexing protocols in the past and I agree with you that it is complex; I actually stated so a few months back. What I am advocating is that if multiplexing is to be supported in one way or another then interop will be better served if multiplexing is included as part of the protocol itself rather than left to an application.

Multiplexing can be implemented through control messages as you mention. But we don't need to define a specific solution right now. What we do need to do now is reach consensus on whether multiplexing is to be supported (as an *optional* feature) and what we can/should reserve for it. Imo the proposal with reserved op-codes is good enough for extending the protocol in the future. We also need to take a decision on feature negotiation so that such optional features can be negotiated during the handshake.

> time, they have been proven to work better.[2]

I am not sure whether SCTP would be suitable but AFAIK currently it is not available on .NET/.NET CF/Silverlight/Azure so it is not an option.

Arman

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 4 Aug 2010 17:32:59 +0200

On Wed, Aug 4, 2010 at 4:43 PM, Arman Djusupov <[email-masked]> wrote:

> Multiplexing can be implemented through control messages as you mention. But
> we don't need to define a specific solution right now. What we do need to do
> now is reach consensus on whether multiplexing is to be supported (as an
> *optional* feature) and what we can/should reserve for it.

Indeed.  It seems reasonable to reserve space for experimentation at
all levels in any case, including framing.  Multiplexing will require
negotiation.  It would be ideal IMO if there was full separation
between WebSockets as a protocol and multiplexing as a framing
convention on top of that (not at the application layer but as a
modified / negotiated transport layer).

> I am not sure whether SCTP would be suitable but AFAIK currently it is not
> available on .NET/.NET CF/Silverlight/Azure so it is not an option.

Catch-22 to some extent.  I don't think that rules out SCTP, rather it
suggests WebSockets should work over TCP as well but then it won't
offer the same performance (if indeed multiplexing makes a significant
difference).

-Pieter