Subject: Re: [hybi] Discontinuation of mux standardizaton in favor of WS/HTTP/2.0

From: Roberto Peon <[email-masked]>
Date: Mon, 3 Feb 2014 09:15:33 -0800

I'm interested in supporting the W3C WS API.
I'm interested in correctly muxing, demuxing, and loadbalancing of
streams-of-messages without needing to understand application-layer
semantics.
I'm interested in enabling a single transport-layer session for both HTTP,
WS, and potentially other content.

I'm not at all interested in needing to implement multiple separate
protocols in order to implement the WS API.
I'm uninterested in small-number-of-byte savings.
I am uninterested in any extensions or extension frameworks that require
any extension-specific knowledge in intermediaries.

On Fri, Jan 31, 2014 at 4:13 AM, Tobias Oberstein <
[email-masked]> wrote:

> Hello Yutaka,
>
>
>
> my priorities:
>
>
>
> 1. allows to provide W3C WebSocket API
>
> 2. no "80%-WebSocket-protocol-compatible" design (either compatible with
> full RFC6455 incl. any extensions or outright independent of WS)
>
> 3. does not put servers that provide both native WS and SPDY (but not any
> SPDY/WS) on same origin at a disadvantage
>
> 4. minimizes code coupling between WS and SPDY within implementations
>
>
>
> /Tobias
>
>
>
> *Von:* [email-masked] [mailto:[email-masked]] *Im Auftrag von *Yutaka
> Hirano
> *Gesendet:* Freitag, 31. Januar 2014 11:41
> *An:* Tobias Oberstein
> *Cc:* Roberto Peon; Peter Thorson; [email-masked]
>
> *Betreff:* Re: [hybi] Discontinuation of mux standardizaton in favor of
> WS/HTTP/2.0
>
>
>
> Seriously: _what_ is the goal of this effort?
>
> My motivation is, specifying a multiplexed bidirectional message protocol
> conforming to the W3C WebSocket API on HTTP/2.0.
>
>
>
>  - The W3C WebSocket API
>
>  - Multiplexing
>
> is mandatory for me.
>
> Since It is built on HTTP/2.0, I prefer strongly the following:
>
>  - A mechanism that enables intermediaries to transfer WS over HTTP/2.0
> without fully understanding WS over HTTP/2.0.
>
>
>
> The remaining, though they are preferable, are not mandatory for me.
>
>  - RFC6455 conformance
>
>  - small amount of bandwidth efficiency
>
>  - the spec cleanness
>
>  - less-invasiveness
>
>
>
> Note: It does not mean I can't accept WS-TUNNEL.
>
>
>
> Tobias, Roberto, Peter, can you share your priority?
>
>
>
>
>
> On Thu, Jan 30, 2014 at 8:54 PM, Tobias Oberstein <
> [email-masked]> wrote:
>
> >I don't find the frame-size arguments very compelling since I haven't
> seen or met anyone with a substantial deployment who cares significantly
> about the overhead difference given packet, TLS, etc. overheads.
> >There is no deployment that we know about that uses frame-based
> extensibility- we can (and should) remove it from the standard since it
> hasn't proven to be interesting.
>
> There is a draft for a WebSocket extension ("permessage-priority") being
> currently developed that makes use of this and we will start an
> experimental implementation in Autobahn. It _is_ interesting.
>
> In any case, the ability for framing, and the ability for WS extensions to
> interleave WS messages is part of RFC6455.
>
> Why remove something from RFC6455 just to fit the limited capabilities
> that SPDY offers?
>
>
> >I'm not sure I agree with your statement that libraries supporting HTTP/2
> will include necessarily also have WS framing. It seems unlikely to me-- I
> know of at least one large deployment where this isn't at all true.
>
> I'd put it like this: any networking library that wants to support
> "messaging" on the Web will need to support native WebSocket, plus
> optionally any SPDY-based messaging.
>
> Since, WebSocket is deployed _today_ and HTTP2 won't be everywhere anytime
> soon.
>
> E.g. HTTP/2 support on Windows seems to be coming only on Windows 8+
> (since they have HTTP in-kernel). So native WebSocket is the only messaging
> that works on any IE version on Windows <8.
>
>
> >The HTTP/2 standard has (or shortly will have, as soon as the pull
> request goes in) all support necessary to handle any stream-of-message
> protocol given support of END_MESSAGE.
> >I'd strongly prefer that we map WS to those primitives so it, and any
> other stream-of-message protocols can be handled using loadbalancer/routing
> infrastructure which is and can remain protocol agnostic.
>
> That's fine, let's just not call it WebSocket-over-SPDY then;)
>
> Seriously: _what_ is the goal of this effort?
>
> /Tobias
>
>
>
> -=R
>
> On Wed, Jan 29, 2014 at 5:56 AM, Tobias Oberstein <
> [email-masked]> wrote:
> >The proposal is mapping one WS Message to one HEADERS + multiple DATAs.
> >For large messages, it can achieve less overhead than WS-TUNNEL at the
> cost of some RFC6455 compatibility.
> How can it achieve less overhead?
>
> A SPDY HEADER with information from first WS frame seems to be always
> bigger than a SPDY DATA header (8 octets) + WS frame header.
>
> The only situation I can think of: a large, _fragmented_ WS message is
> reframed into a SPDY HEADER+DATAs sequence.
>
> Effectively, you do away with the individual WS frame information. And
> this exactly breaks full RFC6455 compatibility.
>
> /Tobias
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>
>

---

From: Peter Thorson <[email-masked]>
Date: Mon, 3 Feb 2014 14:36:25 -0600

My priorities:

Important:
1. W3C WebSocket API and muxing go without saying.
2. Dedicated RFC6455 connections remain a viable option in modern browsers.
3. Efficient and clearly defined method for detecting HTTP/2.0 messaging support, choosing HTTP/2.0 vs raw RFC6455, when to fall back, etc.
4. Payload compression with appropriate configurability (either via permessage-deflate support or a native mechanism)

Nice to haves:
5. Avoid significant regressions from RFC6455 in wire overhead, latency, and per message fixed costs for very small (<100byte) messages.
6. Clean spec/avoiding redundant and unnecessary features of RFC6455.

Only if it doesn’t conflict with 1-6:
7. Options for future compatibility with simple RFC6455 extensions

As long as #2 and #3 are satisfied I am less concerned by performance details or RFC6455 compatibility.

Question for Roberto:
> I'm not at all interested in needing to implement multiple separate protocols in order to implement the WS API.

Can you clarify under what context you are referring here? Is your meaning that Chrome.app should not have to implement both HTTP/2.0 and RFC6455? Or that one should be able to create an HTTP/2.0-only product without needing to pull in an RFC6455 library? Something else?

Question for Yutaka:
- A mechanism that enables intermediaries to transfer WS over HTTP/2.0 without fully understanding WS over HTTP/2.0.

Does the introduction of a "message end” flag that HTTP/2.0 intermediaries would have to understand as “do not coalesce across” violate this constraint?

On Feb 3, 2014, at 11:15 , Roberto Peon <[email-masked]> wrote:

> I'm interested in correctly muxing, demuxing, and loadbalancing of streams-of-messages without needing to understand application-layer semantics.
> I'm interested in enabling a single transport-layer session for both HTTP, WS, and potentially other content.
> 
> I'm not at all interested in needing to implement multiple separate protocols in order to implement the WS API.
> I am uninterested in any extensions or extension frameworks that require any extension-specific knowledge in intermediaries. 
> 
> 
> On Fri, Jan 31, 2014 at 4:13 AM, Tobias Oberstein <[email-masked]> wrote:
> Hello Yutaka,
> 
>  
> 
> my priorities:
> 
>  
> 
> 2. no "80%-WebSocket-protocol-compatible" design (either compatible with full RFC6455 incl. any extensions or outright independent of WS)
> 
> 3. does not put servers that provide both native WS and SPDY (but not any SPDY/WS) on same origin at a disadvantage
> 
> Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Yutaka Hirano
> Gesendet: Freitag, 31. Januar 2014 11:41
> An: Tobias Oberstein
> Cc: Roberto Peon; Peter Thorson; [email-masked]
> 
> 
> Betreff: Re: [hybi] Discontinuation of mux standardizaton in favor of WS/HTTP/2.0
> 
>  
> 
> My motivation is, specifying a multiplexed bidirectional message protocol conforming to the W3C WebSocket API on HTTP/2.0.
> 
>  
> 
>  - The W3C WebSocket API
> 
>  - Multiplexing
> 
> is mandatory for me.
> 
>  - A mechanism that enables intermediaries to transfer WS over HTTP/2.0 without fully understanding WS over HTTP/2.0.
> 
>  
> 
> On Thu, Jan 30, 2014 at 8:54 PM, Tobias Oberstein <[email-masked]> wrote:
> 
> >I don't find the frame-size arguments very compelling since I haven't seen or met anyone with a substantial deployment who cares significantly about the overhead difference given packet, TLS, etc. overheads.
> >There is no deployment that we know about that uses frame-based extensibility- we can (and should) remove it from the standard since it hasn't proven to be interesting.
> 
> There is a draft for a WebSocket extension ("permessage-priority") being currently developed that makes use of this and we will start an experimental implementation in Autobahn. It _is_ interesting.
> 
> In any case, the ability for framing, and the ability for WS extensions to interleave WS messages is part of RFC6455.
> 
> Why remove something from RFC6455 just to fit the limited capabilities that SPDY offers?
> 
> 
> >I'm not sure I agree with your statement that libraries supporting HTTP/2 will include necessarily also have WS framing. It seems unlikely to me-- I know of at least one large deployment where this isn't at all true.
> 
> I'd put it like this: any networking library that wants to support "messaging" on the Web will need to support native WebSocket, plus optionally any SPDY-based messaging.
> 
> Since, WebSocket is deployed _today_ and HTTP2 won't be everywhere anytime soon.
> 
> E.g. HTTP/2 support on Windows seems to be coming only on Windows 8+ (since they have HTTP in-kernel). So native WebSocket is the only messaging that works on any IE version on Windows <8.
> 
> 
> >The HTTP/2 standard has (or shortly will have, as soon as the pull request goes in) all support necessary to handle any stream-of-message protocol given support of END_MESSAGE.
> >I'd strongly prefer that we map WS to those primitives so it, and any other stream-of-message protocols can be handled using loadbalancer/routing infrastructure which is and can remain protocol agnostic.
> 
> On Wed, Jan 29, 2014 at 5:56 AM, Tobias Oberstein <[email-masked]> wrote:
> >For large messages, it can achieve less overhead than WS-TUNNEL at the cost of some RFC6455 compatibility.
> A SPDY HEADER with information from first WS frame seems to be always bigger than a SPDY DATA header (8 octets) + WS frame header.
> 
> The only situation I can think of: a large, _fragmented_ WS message is reframed into a SPDY HEADER+DATAs sequence.
> 
> Effectively, you do away with the individual WS frame information. And this exactly breaks full RFC6455 compatibility.
> 
> /Tobias

---

From: Roberto Peon <[email-masked]>
Date: Mon, 3 Feb 2014 12:50:17 -0800

On Mon, Feb 3, 2014 at 12:36 PM, Peter Thorson <[email-masked]>wrote:

> 3. Efficient and clearly defined method for detecting HTTP/2.0 messaging
> support, choosing HTTP/2.0 vs raw RFC6455, when to fall back, etc.
> 4. Payload compression with appropriate configurability (either via
> permessage-deflate support or a native mechanism)
>
> Nice to haves:
> 5. Avoid significant regressions from RFC6455 in wire overhead, latency,
> and per message fixed costs for very small (<100byte) messages.
> Can you clarify under what context you are referring here? Is your meaning
> that Chrome.app should not have to implement both HTTP/2.0 and RFC6455? Or
> that one should be able to create an HTTP/2.0-only product without needing
> to pull in an RFC6455 library? Something else?
>

Yup. One shouldn't have to write parsers for both 6455 and HTTP/2 if one is
only interested in supporting WS via HTTP/2.

Separately, I wouldn't at all mind having an ALPN identifier for RFC6455 WS
on port 443. I view that as a side-benefit of the HTTP/2 work, but not
HTTP/2 itself.

-=R

> Does the introduction of a "message end” flag that HTTP/2.0 intermediaries
> would have to understand as “do not coalesce across” violate this
> constraint?
>
>> In any case, the ability for framing, and the ability for WS extensions
>> to interleave WS messages is part of RFC6455.
>>
>> >I'm not sure I agree with your statement that libraries supporting
>> HTTP/2 will include necessarily also have WS framing. It seems unlikely to
>> me-- I know of at least one large deployment where this isn't at all true.
>>

---

From: Yutaka Hirano <[email-masked]>
Date: Tue, 4 Feb 2014 15:12:01 +0900

> would have to understand as “do not coalesce across” violate this

I think it does not.
I would like to introduce MSG_DONE in the HTTP/2.0 spec so that all
intermediaries conforming to the spec can transfer WS over HTTP/2.0
messages.
The point is they don't have to read the WebSocket over HTTP/2.0 spec if
they don't interested in it.

Thanks,

---

From: Yutaka Hirano <[email-masked]>
Date: Tue, 4 Feb 2014 23:24:53 +0900

Hi Roberto,

> streams-of-messages without needing to understand application-layer

IIUC None of Plan A, B and C provide the ability to detect exact message
boundary without understanding WebSocket level semantics even if we give up
some extensions, because of control message injection.
Introducing START_MESSAGE flag in addition to MSG_DONE enables it with Plan
A or Plan C. Is it acceptable for you?

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Feb 2014 07:02:13 -0800

>I'm interested in correctly muxing, demuxing, and loadbalancing of streams-of-messages without needing to understand application-layer semantics.

What is the unit of muxing/demuxing/loadbalancing?

If the unit is the SPDY _stream_, then we can tunnel WebSocket with any extensions by simply transporting complete
WebSocket frames inside the payload of SPDY data frames - with no flags whatsoever at SPDY header level. 

If the unit is the individual application _message_, then understanding of application message boundaries is required at SPDY level.

Which unit did you think of?

>I'm not at all interested in needing to implement multiple separate protocols in order to implement the WS API.

Sure, but it looks like a practical necessity. RFC6455 is out on the street and widely implemented.

>I am uninterested in any extensions or extension frameworks that require any extension-specific knowledge in intermediaries. 

Not sure what this means ..

WebSocket extensions are allowed to interleave messages explicitly in RFC6455. Intermediaries are not required
to _understand_ the interleaving. They are required to _not touch_ the fragmentation in this case.

A HTTP2.0 intermediary that does it's muxing/demuxing/loadbalancing at the SPDY stream level does not need
to understand anything about tunneled WS.

/Tobias

my priorities:

/Tobias

 
 
 - The W3C WebSocket API
 - Multiplexing
is mandatory for me.

 

/Tobias

-=R

/Tobias

---

From: Peter Thorson <[email-masked]>
Date: Tue, 4 Feb 2014 11:08:17 -0600

On Feb 4, 2014, at 08:24 , Yutaka Hirano <[email-masked]> wrote:

> IIUC None of Plan A, B and C provide the ability to detect exact message boundary without understanding WebSocket level semantics even if we give up some extensions, because of control message injection.
> Introducing START_MESSAGE flag in addition to MSG_DONE enables it with Plan A or Plan C. Is it acceptable for you?

Can we clarify again what plan A, B, and C are?

I don’t see any issue with message boundary detection using only MSG_DONE in either a native HTTP/2.0 message plan or a purely tunneled plan.

Under tunneled plan the only information that is necessary for HTTP/2.0 intermediaries to understand in order to mux/demux/load balance/etc is the stream end bit (unless you mean to balance messages within a stream to different backends). All information about message boundaries and control messages is contained in the stream data itself and the RFC6455 parser eventually reading the stream will be able to pick all that up without any additional help.

Under the native HTTP/2.0 message plan there is no need for control messages at all except implicitly a “close” frame (i.e. a frame with stream end set) which as the last frame in the stream should not conflict with message boundary detection.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 4 Feb 2014 09:38:22 -0800

>From my viewpoint, that depends on how you define it.
If you say that everything until MSG_DONE is part of a message, you're
achieved what I want. If there is metadata in there, that is fine. HTTP/2
is spec'd to ensure that metadata and data are always synchronized.
Metadata 10283 bytes into a message will always be 10283 bytes into a
message.
-=R

On Tue, Feb 4, 2014 at 6:24 AM, Yutaka Hirano <[email-masked]> wrote:

> Introducing START_MESSAGE flag in addition to MSG_DONE enables it with
> Plan A or Plan C. Is it acceptable for you?
>
>
>
>>> 3. does not put servers that provide both native WS and SPDY (but not
>>> any SPDY/WS) on same origin at a disadvantage
>>>
>>> My motivation is, specifying a multiplexed bidirectional message
>>> protocol conforming to the W3C WebSocket API on HTTP/2.0.
>>>
>>>
>>>
>>>  - The W3C WebSocket API
>>>
>>>  - Multiplexing
>>>
>>> is mandatory for me.
>>>

---

From: Roberto Peon <[email-masked]>
Date: Tue, 4 Feb 2014 09:40:24 -0800

On Tue, Feb 4, 2014 at 7:02 AM, Tobias Oberstein <
[email-masked]> wrote:

> If the unit is the SPDY _stream_, then we can tunnel WebSocket with any
> extensions by simply transporting complete
> If the unit is the individual application _message_, then understanding of
> application message boundaries is required at SPDY level.
>

Messages, since I have to worry about other message-based protocols.

> street and widely implemented.

Well, I'm not interested in implementing it :)

> to understand anything about tunneled WS.

s/stream/message, and I agree.
-=R

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Feb 2014 10:11:20 -0800

>Messages, since I have to worry about other message-based protocols.
 
What other protocols? Can we see the complete agenda?

Why is muxing/demuxing/load-balancing at the SPDY stream level not sufficient?

Also: the W3C WS API provides applications with an _ordered_ stream of messages between endpoints.

If you break up the message stream for load-balancing on a "per-message basis", it seems the
intermediary needs to know about the semantics the application applies to it's messages.

IOW, I can't see how you can have both of this:

a) provide W3C WS API
b) have a (general) intermediary that muxes/demuxes/load-balances at the _message_ level

>Well, I'm not interested in implementing it :)

And that's just fine;)

And I am not interested in an only partially-WS-compatible protocol that is more complicated than necessary
and restricts further development of WebSocket itself;)

Why bother with WebSocket at all, when some message-based thing over SPDY with a W3C WS API
would fit the bill? No need to shove WS into SPDY or call it SPDY/WebSocket when it isn't.

>s/stream/message, and I agree.

With that replacement, the sentence becomes invalid;)

/Tobias

-=R

/Tobias

my priorities:

/Tobias

 
 
 - The W3C WebSocket API
 - Multiplexing
is mandatory for me.

 

/Tobias

-=R

/Tobias

---

From: Peter Thorson <[email-masked]>
Date: Tue, 4 Feb 2014 12:11:42 -0600

On Feb 4, 2014, at 11:40 , Roberto Peon <[email-masked]> wrote:

> On Tue, Feb 4, 2014 at 7:02 AM, Tobias Oberstein <[email-masked]> wrote:

How do you mux, demux, or load balance an ordered message stream at the message level without taking into account application-layer semantics?

---

From: Roberto Peon <[email-masked]>
Date: Tue, 4 Feb 2014 10:44:10 -0800

Note that the applications which use HTTP/WS define their own semantics on
top of (or sometimes replacing) that of the transport. I know of a number
of HTTP applications that enforce ordered messages, and I know of a few
usecases for streams-of-messages where the application would prefer
out-of-order delivery.

In any case, back to the thread:
Assuming that we're implementing a  proposal where END_MESSAGE delimits a
message boundary: to mux/demux/loadbalance any stream of messages all you
need to know is the application's policy for that stream of messages, since
END_MESSAGE allows one to differentiate between the different messages.
A loadbalancer can easily discover this policy at stream initialization
time by examining the path, scheme or any other interesting data from the
HEADERS fame which initialized the stream-of-messages, and it can do so
without any requirement to known anything more about the data being carried.

In other words, if a loadbalancer sees that the WS stream-of-messages is
targeted at an endpoint that can process out-of-order messages (by
examining the HEADERS frame initializing the stream-of-messages), then it
is free to demux the messages and forward without worrying about reordering
on the outbound path.

Note that property/mechanism would hold true for both HTTP and WS (and
other, e.g. RPC or API) traffic.

Tobias: We loadbalance myriad things, with RPC and API stuff being part of
the mix.
-=R

On Tue, Feb 4, 2014 at 10:11 AM, Peter Thorson <[email-masked]>wrote:

>> If the unit is the individual application _message_, then understanding
>> of application message boundaries is required at SPDY level.
>>
> How do you mux, demux, or load balance an ordered message stream at the
> message level without taking into account application-layer semantics?
>

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Feb 2014 11:44:18 -0800

>In other words, if a loadbalancer sees that the WS stream-of-messages is targeted at an endpoint that can process out-of-order messages (by examining the HEADERS frame initializing the stream-of-messages), >then it is free to demux the messages and forward without worrying about reordering on the outbound path.

I can't follow the whole argumentation.

Why would an application that does not need an ordered message channel, but is better served by e.g. independent
request/response message exchanges choose an ordered transport protocol in the first place?

And then need some ad-hoc, non-standard / application-specific way of signaling to intermediaries
that in fact it doesn't rely on the ordering the transport otherwise would guarantee?

Such an app should just do HTTP request/response style coms.

An app that needs ordering will request an ordered transport, and _not_ later give up on that again.

And it's no longer a generic intermediary: it must be configured to "know" that
certain targets are in fact allowed to be reordered, despite the transport being ordered
originally.

>Tobias: We loadbalance myriad things, with RPC and API stuff being part of the mix.

I guess you are talking about protocols _above_ HTTP or WS?

Curious: which of those protocols choose an ordered message transport, and in fact
do _not_ rely on the actual ordering guarantees?

Since that is the niche we are now talking about.

In any case: my impression/opinion is, that this is the core of the crux:

The simplest approach of W3C API plus per-stream muxing/demuxing/loadbalancing WS over SPDY with 100% WS compatibility would be full tunneling.

The use case not covered by the latter is "_per message_ muxing/demuxing/loadbalancing".

And that "use case" I can't follow, since it gives up message ordering, and would be better served by plain HTTP anyway ..

There is no "unordered WebSocket" .. that is HTTP;)

/Tobias

On Tue, Feb 4, 2014 at 10:11 AM, Peter Thorson <[email-masked]> wrote:

---

From: Yutaka Hirano <[email-masked]>
Date: Wed, 5 Feb 2014 19:56:06 +0900

> I don’t see any issue with message boundary detection using only MSG_DONE
> in either a native HTTP/2.0 message plan or a purely tunneled plan.

Let me talk about PlanA, WS-TUNNEL with MSG_DONE.
Roberto says he is interested in loadbalancing ***without needing to
understand application-layer semantics***. (1)
I say we can't detect message boundary ***without understanding WebSocket
level semantics***. (2)
You say we can detect message boundary ***with the RFC6455 parser***. (3)

Year, I think (2) and (3) are both true.

I thought that Roberto didn't want to implement an RFC6455 paser for the
loadbalancer.
If it is wrong: Yay! PlanA (with MSG_DONE) seems good.

If it is correct:
I still think MSG_DONE is insufficient for the loadbalancing.
Imagine a WebSocket connection consisting of three WebSocket frames, f1
[without FIN], f2 [with FIN] and f3 [with FIN].
Assume that there is no extensions.
Assume that one HTTP/2.0 frame is mapped to one WebSocket frame for
simplicity.
Let the HTTP/2.0 frames be hf1 [without MSG_DONE], hf2 [with MSG_DONE] and
hf3 [with MSG_DONE].
If the loadbalancer does NOT understand RFC6455, there are two
possibilities,
 - A message consists of (hf1, hf2) and another message consists of (hf3).
 - A data message consists of (hf1, hf3) and a control message consists of
(hf2).
Hence it fails to recognize messages.

Note: For enabling intermediaries to transfer WS over HTTP/2.0 data without
understanding RFC6455, MSG_DONE is sufficient.

On Wed, Feb 5, 2014 at 2:08 AM, Peter Thorson <[email-masked]> wrote:

> I don’t see any issue with message boundary detection using only MSG_DONE
> in either a native HTTP/2.0 message plan or a purely tunneled plan.
>
> Under tunneled plan the only information that is necessary for HTTP/2.0
> intermediaries to understand in order to mux/demux/load balance/etc is the
> stream end bit (unless you mean to balance messages within a stream to
> different backends). All information about message boundaries and control
> messages is contained in the stream data itself and the RFC6455 parser
> eventually reading the stream will be able to pick all that up without any
> additional help.
>
> Under the native HTTP/2.0 message plan there is no need for control
> messages at all except implicitly a “close” frame (i.e. a frame with stream
> end set) which as the last frame in the stream should not conflict with
> message boundary detection.
>

---

From: Peter Thorson <[email-masked]>
Date: Wed, 5 Feb 2014 09:46:24 -0600

On Feb 5, 2014, at 4:56, Yutaka Hirano <[email-masked]> wrote:

> Roberto says he is interested in loadbalancing ***without needing to understand application-layer semantics***. (1)
> I say we can't detect message boundary ***without understanding WebSocket level semantics***. (2)
> I thought that Roberto didn't want to implement an RFC6455 paser for the loadbalancer.
> Imagine a WebSocket connection consisting of three WebSocket frames, f1 [without FIN], f2 [with FIN] and f3 [with FIN].
> Assume that one HTTP/2.0 frame is mapped to one WebSocket frame for simplicity.
> Let the HTTP/2.0 frames be hf1 [without MSG_DONE], hf2 [with MSG_DONE] and hf3 [with MSG_DONE].
> If the loadbalancer does NOT understand RFC6455, there are two possibilities,
>  - A data message consists of (hf1, hf3) and a control message consists of (hf2).
> Note: For enabling intermediaries to transfer WS over HTTP/2.0 data without understanding RFC6455, MSG_DONE is sufficient. 

Let me define "Plan D" more precisely:
HTTP/2.0 DATA frames are used as a dumb transport for an RFC6455 byte stream. There is no 1:1 frame mapping between WS frames and HTTP/2.0 frames. A browser could send two WebSocket frames back to back in one HTTP/2.0 frame. An intermediary could split this frame evenly into four HTTP/2.0 frames each containing half of an RFC6455 frame. The remote endpoint would accept an HTTP/2.0 frame, feed its payload bytes straight to an RFC6455 parser which would handle the rest. A MSG_DONE flag is not required at all even for full ignorant intermediary compatibility.

Benefits:
- 100% RFC6455 compatibility present and future
- Simple implementation:
-- Nothing needs to be added to HTTP/2.0 or RFC6455. 
-- Browsers leverage existing RFC6455 parsers and just add logic to choose whether to use an HTTP/2.0 stream or a raw TCP stream. Data centers demux the streams into RFC6455 message processors.
-- Everyone gets trivial backwards compatibility with existing RFC6455 software and hardware, both on the client and server side.
- Intermediaries that don't care can safely ignore or re-fragment however they like.

Drawbacks
- Requires intermediaries and endpoints that want to understand messages to read RFC6455. (I don't think this is an issue, but it seems important to Roberto)
- Retains unnecessary/redundant features (masking, control messages, frame length fields)

I think this is the best option if we value RFC6455 compatibility or desire ease of integration into existing software and environments. Otherwise, I would prefer that we go the "Plan B extreme" route that ditches RFC6455 entirely and builds a completely native HTTP/2.0 message stream. MSG_DONE/type bits, no masking, no opcode, no rsv, no control frames.

Benefits:
- HTTP/2.0 software doesn't need to know about RFC6455 unless it wants backwards compatibility with non-HTTP/2.0 software
- Doesn't re-create lots of irrelevant/duplicated features from RFC6455
- Optimal wire efficiency.

Drawbacks:
- Intermediaries need to implement "HTTP/2.0 messaging". (it is simple enough now though that that shouldn't be a great hardship.)
- Limited RFC6455 extension compatibility.

I don't have a strong preference for either of these options. I do have a strong preference that we not do something in between. RFC6455 compatibility is useful for me because it allows sharing code/infrastructure and backwards compatibility. Not because the specific features of RFC6455 are preserved.

That is the reason I think adding MSG_DONE to WS-TUNNEL doesn't make sense. It puts us in a situation where *all* HTTP/2.0 implementations must implement a bunch of message related functionality that is already in RFC6455. As Yukata points out MSG_DONE alone is not sufficient to detect messages in a tunneled RFC6455 stream. You'd need to throw in an opcode as well. At this point we end up re-implementing half of RFC6455 in HTTP/2.0 in order to avoid using RFC6455 in a product that already has an RFC6455 parser?? The only gain I see is certain types of limited and specialized intermediaries can drop RFC6455. That doesn't seem like a good tradeoff given that we could just choose Plan B and get rid of RFC6455 for everybody.

Regarding message based load balancing for WebSocket:
I agree with Tobias that this doesn't make sense. An application that wants to send independent messages can send HTTP requests. Such requests can be easily muxed, demuxed, or load balanced. Each message has headers to store application specific information necessary to correctly load balance. This can all be done efficiently with the functionality HTTP/2.0 provides already.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 5 Feb 2014 10:18:56 -0800

On Tue, Feb 4, 2014 at 11:44 AM, Tobias Oberstein <
[email-masked]> wrote:

> examining the HEADERS frame initializing the stream-of-messages), >then it

Neither application semantics nor the format of the data for an application
are defined in a standard-- they're defined by the application. The
loadbalancer should be able to help deal with application semantics without
having to worry about the format of the data.

HTTP/2 allows for the generic demarcation of message-boundaries, which
enables this kind of application-specific loadbalancing with generic rules
that don't need to vary based on the protocol spoken inside the HTTP/2
stream.

> Such an app should just do HTTP request/response style coms.

Why? Using the WS API would be more efficient. The loadbalancer can execute
policy such as "last received wins": Why bother to send back responses for
messages that you know are now uninteresting?

> later give up on that again.
That has not been my experience.

> certain targets are in fact allowed to be reordered, despite the transport

I disagree: It is completely generic as it allows for any protocol,
including application-layer protocol

> do _not_ rely on the actual ordering guarantees?

You're making strong assertions about why people use a protocol, but in
fact, people just use the hammer that is available, and pick the shape
closest to what they need.

Both RPC and API things often don't care about careful ordering of requests
to responses.
They're seeking semantics of both WS and HTTP: from WS-like semantics they
want message-based and guaranteed-single-session and from HTTP-like
semantics they want out-of-order.
They're solving an application-layer problem which the transport doesn't
solve on its own.
Such a "protocol" has not been defined because we do a poor job of
designing APIs.
The intent here is to remedy this by thinking about how the protocol
interacts with the entire ecosystem, in particular with proxies and
loadbalancers.
Why do we have to change the LB code for every protocol? If we do this the
smart way, then we don't have to!

> Since that is the niche we are now talking about.

We're talking about being able to loadbalance *any* bytestream protocol or
*any* stream-of-message protocol. That pretty much covers the gamut.

> The simplest approach of W3C API plus per-stream
> muxing/demuxing/loadbalancing WS over SPDY with 100% WS compatibility would
> be full tunneling.
>

It is simple only if you are willing to commit to adding more code ad
infinitum to servers, one for each protocol.
That seems like a poor tradeoff, especially given that it doesn't actually
offer any more WS compatibility.

> And that "use case" I can't follow, since it gives up message ordering,
> and would be better served by plain HTTP anyway ..
>

WS here is a protocol, and possibly an API. Neither of these specify
in-order responses. Certainly both HTTP and WS specify in-order delivery.
HTTP actually mandates in-order response, which is counter to what many
applications want and is a HORRIBLE thing for demuxing loadbalancers as it
requires potentially infinite buffering. This is a real practical problem :)

Applications generally want to choose from:
unordered/ordered | reliable/unreliable | message/bytestream |
single/multi-session

We don't offer the full range of choice for these, and applications make-do
with the closest they have and then add enhancements to make more possible.

-=R

---

From: Wenbo Zhu <[email-masked]>
Date: Wed, 5 Feb 2014 12:04:28 -0800

> muxing/demuxing/loadbalancing WS over SPDY with 100% WS compatibility would
So the question is whether we tunnel the W3C API or the existing wire
protocol, which happens to exist only to support the API (i.e., not the
case for HTTP).

The load-balancing requirement only concerns the API (in the form of URLs,
headers), or more accurately the message delivery. As for HTTP,
load-balancing often has to address session stickiness, which may or may
not imply ordering, independently of the transport being RPC.

- Wenbo

---

From: Yutaka Hirano <[email-masked]>
Date: Thu, 6 Feb 2014 15:04:06 +0900

Peter,

I don't have a strong preference for either of these options. I do have a
> strong preference that we not do something in between. RFC6455
> compatibility is useful for me because it allows sharing
> code/infrastructure and backwards compatibility. Not because the specific
> features of RFC6455 are preserved.
>

> That is the reason I think adding MSG_DONE to WS-TUNNEL doesn't make
> sense. It puts us in a situation where *all* HTTP/2.0 implementations must
> implement a bunch of message related functionality that is already in
> RFC6455.

I still disagree.
As I said before, enabling intermediaries to transfer WS over HTTP/2.0
*without an RFC6455 parser* is important, I think.

No matter what protocol we choose, we must observe the HTTP/2.0 framing
protocol, which means:
  - The size of a frame must not exceed the fixed limit.
  - We must set and respect the END_STREAM flag appropriately.
  - ...
Adding one rule (MSG_DONE) is harmless, and I think it has a merit.

On Thu, Feb 6, 2014 at 12:46 AM, Peter Thorson <[email-masked]>wrote:

> Note: For enabling intermediaries to transfer WS over HTTP/2.0 data
> without understanding RFC6455, MSG_DONE is sufficient.
>
>
> HTTP/2.0 DATA frames are used as a dumb transport for an RFC6455 byte
> stream. There is no 1:1 frame mapping between WS frames and HTTP/2.0
> frames. A browser could send two WebSocket frames back to back in one
> HTTP/2.0 frame. An intermediary could split this frame evenly into four
> HTTP/2.0 frames each containing half of an RFC6455 frame. The remote
> endpoint would accept an HTTP/2.0 frame, feed its payload bytes straight to
> an RFC6455 parser which would handle the rest. A MSG_DONE flag is not
> required at all even for full ignorant intermediary compatibility.
>
> Benefits:
> -- Browsers leverage existing RFC6455 parsers and just add logic to choose
> whether to use an HTTP/2.0 stream or a raw TCP stream. Data centers demux
> the streams into RFC6455 message processors.
> -- Everyone gets trivial backwards compatibility with existing RFC6455
> software and hardware, both on the client and server side.
> - Requires intermediaries and endpoints that want to understand messages
> to read RFC6455. (I don't think this is an issue, but it seems important to
> Roberto)
> I think this is the best option if we value RFC6455 compatibility or
> desire ease of integration into existing software and environments.
> Otherwise, I would prefer that we go the "Plan B extreme" route that
> ditches RFC6455 entirely and builds a completely native HTTP/2.0 message
> stream. MSG_DONE/type bits, no masking, no opcode, no rsv, no control
> frames.
>
> Benefits:
> - HTTP/2.0 software doesn't need to know about RFC6455 unless it wants
> backwards compatibility with non-HTTP/2.0 software
> - Intermediaries need to implement "HTTP/2.0 messaging". (it is simple
> enough now though that that shouldn't be a great hardship.)
> I don't have a strong preference for either of these options. I do have a
> strong preference that we not do something in between. RFC6455
> compatibility is useful for me because it allows sharing
> code/infrastructure and backwards compatibility. Not because the specific
> features of RFC6455 are preserved.
>
> That is the reason I think adding MSG_DONE to WS-TUNNEL doesn't make
> sense. It puts us in a situation where *all* HTTP/2.0 implementations must
> implement a bunch of message related functionality that is already in
> RFC6455. As Yukata points out MSG_DONE alone is not sufficient to detect
> messages in a tunneled RFC6455 stream. You'd need to throw in an opcode as
> well. At this point we end up re-implementing half of RFC6455 in HTTP/2.0
> in order to avoid using RFC6455 in a product that already has an RFC6455
> parser?? The only gain I see is certain types of limited and specialized
> intermediaries can drop RFC6455. That doesn't seem like a good tradeoff
> given that we could just choose Plan B and get rid of RFC6455 for everybody.
>
> I agree with Tobias that this doesn't make sense. An application that
> wants to send independent messages can send HTTP requests. Such requests
> can be easily muxed, demuxed, or load balanced. Each message has headers to
> store application specific information necessary to correctly load balance.
> This can all be done efficiently with the functionality HTTP/2.0 provides
> already.
>
>

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 6 Feb 2014 04:09:57 -0800

>>That is the reason I think adding MSG_DONE to WS-TUNNEL doesn't make sense. It puts us in a situation where *all* HTTP/2.0 implementations must implement a bunch of message related functionality that is already in RFC6455.
>I still disagree.
>As I said before, enabling intermediaries to transfer WS over HTTP/2.0 *without an RFC6455 parser* is important, I think.

Agree. And Plan D allows this.

>No matter what protocol we choose, we must observe the HTTP/2.0 framing protocol, which means:

MSG_DONE is a problem.

Assume we would

a) set MSG_DONE == FIN Bit of WS Frame and
b) assign semantics to MSG_DONE: everything in the payload octet stream up to MSG_DONE is 1 message

This leads to incompatiblity with RFC6455, which allows WS extensions to _interleave_ messages. Without knowledge of
how the interleaving is done, MSG_DONE is void of meaning. (It also leads to problem with WS control frames.)

If we allow MSG_DONE, the result might be:

SPDY/WS puts practically limits on the design space of WebSocket extensibility.

And this exactly is my problem: why should SPDY, and it's wish to tunnel WS (practically) _limit_ the design space of an unrelated protocol (WS)? 

In fact, I find this idea quite disturbing:

Protocol A that wants to tunnel protocol B puts up limits on protocol A to fulfill it's perceived requirements.

No, Protocol A should just tunnel B "as is". That is tunneling.

Note that Plan D does not have this problems. It also allows to mux/demux/load-balance tunneled WS on
a per-stream basis.

Plan D does not allow to mux/demux/load-balance on a _per-message_ basis.

And as far as I understand Robert, this is his itch. And I can't follow that. Since:

muxing/demuxing/load-balancing an _ordered_ message stream on a _per-message_ basis
breaks the _ordering_ guarantee made in the first place.

And on the other hand: if you want to mux/demux/lb on a _per-message_ basis, great, but not
on top of a strictly ordered, message-based protocol (WS).

So currently I think what Robert wants is a _new_ message-based protocol over SPDY - not WS.

Lets call that Plan E  ("new message-based proto on SPDY - unrelated to WS (other than probably W3C API)").

FWIW, I would support Plan D or Plan E. I think Plan A, B and C are less desirable.

Plan D is straight-forward - in particular taking into account your (Yutaka's) work already done on initial handshake and negotiation.

Plan E: this might be interesting, since

1) HTTP: provides apps with independent request/response message exchange pattern
2) WS: provides apps with strictly ordered message stream

obviously does not cover all message exchange patterns apps might want.

I agree with Robert on that (if I understood his other mail correctly).

But this is a bigger undertaking than "tunnel WS over SDPY".

It involves not only protocol, but also API for app to express their desired message exchange pattern. etc etc ..

Also, there is work being done on WebRTC Data Channels, which provide apps with all combinations of

- ordered/unordered
- reliable/unreliable

message delivery.

In fact, WebRTC has a complete, independent protocol stack on top of UDP.

So as far as I can see, there are at least 3 efforts now on the Web data have business in "message-based" protocols:

- WebSocket
- WebRTC
- SPDY

/Tobias

On Thu, Feb 6, 2014 at 12:46 AM, Peter Thorson <[email-masked]> wrote:

If it is correct:

Benefits:

Drawbacks

Benefits:

Drawbacks:

---

From: Yutaka Hirano <[email-masked]>
Date: Thu, 6 Feb 2014 04:32:38 -0800

Hi Tobias,

> Agree. And Plan D allows this.

I'm sorry, I don't understand how. Can you explain?

FYI, what I'm worrying about is the following.

> Some intermediaries can buffer DATA frames, but they should be flushed at
> each message end. Otherwise, deadlock can occur.

 MSG_DONE is a problem.
> This leads to incompatiblity with RFC6455, which allows WS extensions to
> _interleave_ messages. Without knowledge of

MSG_DONE is used just as a flush signal (at HTTP/2.0 frame level) in Plan A.
I don't think it breaks permessage-priority.

On Thu, Feb 6, 2014 at 4:09 AM, Tobias Oberstein <
[email-masked]> wrote:

> >>I don't have a strong preference for either of these options. I do have
> a strong preference that we not do something in between. RFC6455
> This leads to incompatiblity with RFC6455, which allows WS extensions to
> _interleave_ messages. Without knowledge of
> And this exactly is my problem: why should SPDY, and it's wish to tunnel
> WS (practically) _limit_ the design space of an unrelated protocol (WS)?
>
> Plan D is straight-forward - in particular taking into account your
> (Yutaka's) work already done on initial handshake and negotiation.
>
> It involves not only protocol, but also API for app to express their
> desired message exchange pattern. etc etc ..
>
> So as far as I can see, there are at least 3 efforts now on the Web data
> have business in "message-based" protocols:
>
> - WebSocket
> - WebRTC
> - SPDY
>
> /Tobias
>
>
>
>

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 6 Feb 2014 04:44:03 -0800

>I disagree: It is completely generic as it allows for any protocol, including application-layer protocol

If an app assumes ordered message delivery, how can it be generic when the load-balancer
breaks the ordering (and hence the app)?
 
>Why do we have to change the LB code for every protocol? If we do this the smart way, then we don't have to!

I guess with "smart way" you mean migrate everything to SPDY? =)
 
>We're talking about being able to loadbalance *any* bytestream protocol or *any* stream-of-message protocol. That pretty much covers the gamut.
 
The niche I was talking about is load-balancing an ordered message based protocol on a per-message basis - and hence breaking ordering.

>It is simple only if you are willing to commit to adding more code ad infinitum to servers, one for each protocol.

Please forgive when I won't hold my breath watching all the planet's protocols migrate to SPDY;)

Seriously: one size does not fit all. My opinion. It's good that we have plurality, competition and protocols
with different strengths and focus on the Web now.

>That seems like a poor tradeoff, especially given that it doesn't actually offer any more WS compatibility. 

Plan D is 100% WS compatible. Plan A, B and C not.

>unordered/ordered | reliable/unreliable | message/bytestream | single/multi-session

If an app needs unreliable message based delivery, it can use WebRTC Data Channels.

In a way, WebRTC Data Channels already provide most of above:

- unordered/ordered
- reliable/unreliable
- single/multi-session

all message-based. They don't provide bytestreams.

Since WebRTC Data Channels run over UDP, it can provide "unreliable delivery".

How do you do that with SPDY (or WebSocket), which run over TCP?

But in general, I agree with your point (if I got your right): app requirements are more complex and broad.

I'd also say that "ordering" requirements are more complex than a simple yes/no.

It can be

- no ordering
- strict ordering
- priority-based ordering

for a complete channel. It could also be: strict ordering for the channel, but no ordering for individual messages.

One could also have

- order-barrier messages

That is an unordered channel, but being able to send "order barrier messages" such that all subsequently
sent messages will never arrive before the barrier message.

Etc etc.

So what is the scope of your vision? Is it tackling above (and hence quite big) or is it "just tunnel WS over SPDY"?

>We don't offer the full range of choice for these, and applications make-do with the closest they have and then add enhancements to make more possible.

I agree. It would be good to give apps the complete range of combinations to choose from.

But if that is your goal (which I could totally subscribe to), we shouldn't "hide" that under a innocent flag like "just tunnel WS over SPDY", but outright state the "big" goal.

Just my 2cts,
/Tobias

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 6 Feb 2014 05:03:49 -0800

Hi Yutaka,

>I'm sorry, I don't understand how. Can you explain?

Plan D transfers complete WS frames as payload of SPDY data frames.

Each WS connection is carried on a dedicated SPDY stream.

SPDY is completely agnostic to the payload of it's SPDY data frames. It's just bytes.

There is no need to parse the payload at the SPDY level.

Sorry if I am still unable to explain .. don't know how to do better ..

>Some intermediaries can buffer DATA frames, but they should be flushed at each message end. Otherwise, deadlock can occur.

Could you please expand on that?

For one thing, I think we are now talking about 2 different _purposes_ for MSG_DONE:

a) Flush
b) load-balancing on a per-message basis

b) is what Robert is after (if I understand). 

And you are referring to a), but not b)?

And regarding a): could you describe a dead-lock situation? I have problems to imagine one. Deadlock over 2 WS connections?

>I don't think it breaks permessage-priority.

If it's intended use is a Flush signal to intermedaries, then we should probably

a) call it FLUSH_HINT, and
b) allow multiple WS frames with FIN bit inside a single SPDY data frame.

FLUSH_HINT would be set if _any_ WS frame inside the SPDY data frame would have it's FIN bit set.

This would do away with one additional limitation of Plan A over Plan D:

As it stands now, Plan A does not allow batching of multiple WS messages inside a single SPDY data frame.

/Tobias

---

From: Peter Thorson <[email-masked]>
Date: Thu, 6 Feb 2014 09:26:19 -0600

> I'm sorry, I don't understand how. Can you explain?

Specifically for Plan D..
An RFC6455 parser is required to:
- Add or remove messages from the stream
- Determine WS frame and message boundaries
- Read message contents
An RFC6455 parser is NOT required to:
- Proxy the stream
- Load balance the stream
- Mux a WS stream into an HTTP/2.0 stream
- Demux a WS stream out of an HTTP/2.0 stream
- Flush a stream using an HTTP/2.0 level FLUSH_HINT

If it is important that intermediaries or endpoints be able to read and manipulate these streams on a per-message or per-frame level without an RFC6455 parser then a tunneled solution is a bad choice. If all the intermediaries need to do is move the bytes around then it is fine.

> Adding one rule (MSG_DONE) is harmless, and I think it has a merit.

I agree that the MSG_DONE concept in general is a useful general purpose flag to have in HTTP/2.0. I disagree that it is harmless to use it as a 1:1 message delimiter in a tunneled RFC6455 stream.

> As it stands now, Plan A does not allow batching of multiple WS messages inside a single SPDY data frame.

I am fine with a “flush hint” flag where the sending endpoint *may/should* add it to HTTP/2.0 frames that the application prefers not to have buffered/delayed by intermediaries. I am fine with a separate bit named FLUSH_HINT (useful for either Plan D or native messaging) or re-using MSG_DONE to imply “flush hint” for Plan D streams if MSG_DONE is used elsewhere by HTTP/2.0. I would not want to define specific scenarios where a flush must occur.

Re: Load balancing

Robertos point that HTTP messages traditionally must be answered in order certainly makes HTTP less appealing for the unordered, unrelated RPC style message case. I can see how an application might use WebSocket today in order to get out of order message return delivery.

I am curious if there has there been any discussion to relaxing that requirement in HTTP/2.0?

Nothing in Plan D or Native Messaging would prevent the creation of a per-message WebSocket load balancer. In fact you could also do one today using RFC6455 alone. Under Plan D, said balancer would require an RFC6455 parser as it would need to add/remove messages from a stream. Under native messaging obviously not. The load balancer would clearly need to understand the application semantics of that specific stream (i.e. this is a loosely coupled message stream that can be balanced). Information in the URL, subprotocol, headers, etc could be used to communicate this application specific information.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 6 Feb 2014 11:23:44 -0800

On Thu, Feb 6, 2014 at 7:26 AM, Peter Thorson <[email-masked]> wrote:

> If it is important that intermediaries or endpoints be able to read and
> manipulate these streams on a per-message or per-frame level without an
> RFC6455 parser then a tunneled solution is a bad choice. If all the
> intermediaries need to do is move the bytes around then it is fine.
>

Exactly.

> I agree that the MSG_DONE concept in general is a useful general purpose
> flag to have in HTTP/2.0. I disagree that it is harmless to use it as a 1:1
> message delimiter in a tunneled RFC6455 stream.
>

> I wouldn't really find FLUSH_HINT useful for anything unless the
application-logic in the webpage could affect it.
But.. there is no such capability in the API right now.

>
> I am fine with a “flush hint” flag where the sending endpoint *may/should*
> add it to HTTP/2.0 frames that the application prefers not to have
> buffered/delayed by intermediaries. I am fine with a separate bit named
> FLUSH_HINT (useful for either Plan D or native messaging) or re-using
> MSG_DONE to imply “flush hint” for Plan D streams if MSG_DONE is used
> elsewhere by HTTP/2.0. I would not want to define specific scenarios where
> a flush must occur.
>

HTTP2's MSG_DONE (under a different name) is in the next implementation
draft, coming out shortly.
For HTTP on HTTP/2 it is use for delimiting chunk boundaries on an HTTP
stream, since the lack of this has proven painful for some. Intermediaries
are disallowed from coalescing data across frame where this is set.

>
> Re: Load balancing
>
> Robertos point that HTTP messages traditionally must be answered in order
> certainly makes HTTP less appealing for the unordered, unrelated RPC style
> message case. I can see how an application might use WebSocket today in
> order to get out of order message return delivery.
>

Unfortunately, one can't relax that requirement and still have the
backwards-compatible API that exists today: while HTTP/2 can deliver on
this, HTTP/1.X cannot.
That one need not change their content to deploy HTTP/2 (etc.) is a major
driver of deployment, so it isn't something that could be lightly changed.
IMHO, we'd have to invent a new API (which, honestly I'd be all for, but
that is a completely different set of work).

>
> Nothing in Plan D or Native Messaging would prevent the creation of a
> per-message WebSocket load balancer. In fact you could also do one today
> using RFC6455 alone. Under Plan D, said balancer would require an RFC6455
> parser as it would need to add/remove messages from a stream. Under native
> messaging obviously not. The load balancer would clearly need to understand
> the application semantics of that specific stream (i.e. this is a loosely
> coupled message stream that can be balanced). Information in the URL,
> subprotocol, headers, etc could be used to communicate this application
> specific information.
>

Yup.

-=R

---

From: Peter Thorson <[email-masked]>
Date: Thu, 6 Feb 2014 17:18:16 -0600

On Feb 6, 2014, at 13:23 , Roberto Peon <[email-masked]> wrote:

> Unfortunately, one can't relax that requirement and still have the backwards-compatible API that exists today: while HTTP/2 can deliver on this, HTTP/1.X cannot.
> That one need not change their content to deploy HTTP/2 (etc.) is a major driver of deployment, so it isn't something that could be lightly changed.
> IMHO, we'd have to invent a new API (which, honestly I'd be all for, but that is a completely different set of work).

This statement gets at the heart of the present discussion.

I see the "plan D" option treating WebSocket as HTTP/1.1 is treated by the rest of HTTP/2.0, an existing protocol to be muxed mostly "as is" for the benefit of backwards compatibility with existing systems. I see a Native Messaging option as the "new API” that would allow a cleaner interface for pure HTTP/2.0 applications going forward.

For me, the reality is that any piece of HTTP/2.0 software that I write or deploy in the foreseeable future is going to demux the message stream, convert to RFC6455, and forward to existing systems. This is both due to backwards compatibility as well as the nature of a large chunk of my user base (particularly the embedded non-browser clients) having no use for any of the features that HTTP/2.0 adds over RFC6455 and every reason to want to avoid additional overhead.

Plan D provides a straightforward path for this sort of operation and simplifies things considerably for a good chunk of the infrastructure-type intermediaries without imposing any additional limitations on anyone (other than perhaps additional library/protocol dependencies for the more ambitious intermediaries).

That said, even if I plan to proxy HTTP/2.0 messages right back into RFC6455. I also see the future benefit to having a cleaner messaging option that is aware of HTTP/2.0's new features. My first choice is for this option as it seems more sane long term.. as long as the new protocol is something worth breaking backwards compatibility for. The Plan D case is strong as well and I would not oppose that if the consensus says that backwards compatibility is more important.

---

From: Yutaka Hirano <[email-masked]>
Date: Fri, 7 Feb 2014 16:05:43 +0900

Peter, Roberto, Tobias,
I think we should shrink the battle line.
I propose to choose if we preserve the RFC6455 semantics or not (for
framing).

*Preserve the RFC6455 semantics*
We won't change the RFC6455 semantics, which means we will discuss how to
encapsulate WebSocket frame(s) into HTTP/2.0 frame(s).
All RFC6455 extensions will be supported.
To preserve the RFC6455 semantics, I modify the proposed plans.

Plan A (WS-TUNNEL with MSG_DONE):
 - MSG_DONE is used as a FLUSH signal
 - multiple WebSocket message can be batched in a single HTTP/2.0 frame
I believe that this preserves the RFC6455 semantics.
I support this plan.

Plan C (HEADERS + DATAs for a WebSocket frame):
 - All headers including FIN and length are included in HEADERS.
 - MSG_DONE is mapped to the end of a **WebSocket frame**
  - This means that a WebSocket connection is a stream-of-WS_frames, which
is completely correct.
Hence HEADERS + DATAs represent a complete WebSocket frame.
I support this plan.
I believe Roberto supports this plan.

Plan D (WS-TUNNEL only): as is
Tobias and Peter support this plan.

Choosing "Preserve the RFC6455 semantics", means we choose framing method
from these three plans.
I know that all of us hate some of the plans, but I think this option is
better, because
 - All of us like some of the plans.
 - We can simplify the discussion.
 - if "Not preserve the RFC6455 semantics" seems appealing now, at least
partially that is because it is not concrete.

As the only supporter of Plan A, I promise that I will retract the plan if
a critical problem is found on it (it is not compatible with RFC6455,
MSG_DONE can't be used for the flushing purpose, FLUSH can't be introduced
and so on).

*Not preserve the RFC6455 semantics*
Choosing this option means we abandon Plan A, C and D.

*Questions*
As you notice, there are two questions from me.
1: Are you OK with deciding NOW if we should preserve the RFC6455 semantics
or not?
2: (If the answer for 1 is yes) Should we preserve the RFC6455 semantics?

My answer is, yes & yes.

On Fri, Feb 7, 2014 at 8:18 AM, Peter Thorson <[email-masked]> wrote:

> > That one need not change their content to deploy HTTP/2 (etc.) is a
> major driver of deployment, so it isn't something that could be lightly
> changed.
> I see the "plan D" option treating WebSocket as HTTP/1.1 is treated by the
> rest of HTTP/2.0, an existing protocol to be muxed mostly "as is" for the
> benefit of backwards compatibility with existing systems. I see a Native
> Messaging option as the "new API” that would allow a cleaner interface for
> pure HTTP/2.0 applications going forward.
>
> For me, the reality is that any piece of HTTP/2.0 software that I write or
> deploy in the foreseeable future is going to demux the message stream,
> convert to RFC6455, and forward to existing systems. This is both due to
> backwards compatibility as well as the nature of a large chunk of my user
> base (particularly the embedded non-browser clients) having no use for any
> of the features that HTTP/2.0 adds over RFC6455 and every reason to want to
> avoid additional overhead.
>
> Plan D provides a straightforward path for this sort of operation and
> simplifies things considerably for a good chunk of the infrastructure-type
> intermediaries without imposing any additional limitations on anyone (other
> than perhaps additional library/protocol dependencies for the more
> ambitious intermediaries).
>
> That said, even if I plan to proxy HTTP/2.0 messages right back into
> RFC6455. I also see the future benefit to having a cleaner messaging option
> that is aware of HTTP/2.0's new features. My first choice is for this
> option as it seems more sane long term.. as long as the new protocol is
> something worth breaking backwards compatibility for. The Plan D case is
> strong as well and I would not oppose that if the consensus says that
> backwards compatibility is more important.

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 7 Feb 2014 02:16:17 -0800

Hi Yutaka,

>I think we should shrink the battle line.

Agree. Thanks for condensing the current state into concrete questions!

<snip>

>Questions
>1: Are you OK with deciding NOW if we should preserve the RFC6455 semantics or not?

My answer: yes & yes.

Cheers,
/Tobias

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 7 Feb 2014 21:11:02 +0900

> >2: (If the answer for 1 is yes) Should we preserve the RFC6455 semantics?

Yes & Yes

Takeshi

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 7 Feb 2014 06:11:27 -0700

>> >2: (If the answer for 1 is yes) Should we preserve the RFC6455 semantics?
Eclipse Jetty's answer is yes & yes.
We agree with Roberto and Tobias.
We throw our support to Plan D.
(Oddly, our reading of this thread made is seem that Roberto was in support
for Plan D too)

---

From: Peter Thorson <[email-masked]>
Date: Fri, 7 Feb 2014 07:56:45 -0600

On Feb 7, 2014, at 1:05, Yutaka Hirano <[email-masked]> wrote:

> As the only supporter of Plan A, I promise that I will retract the plan if a critical problem is found on it (it is not compatible with RFC6455, MSG_DONE can't be used for the flushing purpose, FLUSH can't be introduced and so on).

With the understanding that under Plan A MSG_DONE is used as a flush signal and not a literal message end flag I would support Plan A as well. I have no particular use for a flush signal, but I would not object to it being present if others found it necessary.

> 2: (If the answer for 1 is yes) Should we preserve the RFC6455 semantics?

1. Yes
2. As HTTP/2.0 is constructed now with a strong focus on backwards compatibility for the HTTP side, Yes

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 10 Feb 2014 09:09:47 +0000

On Feb 7, 2014, at 9:05 AM, Yutaka Hirano <[email-masked]<mailto:[email-masked]>> wrote:

Questions

My answer is, yes & yes.

yes & yes is also my answer

/Salvatore

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 10 Feb 2014 09:17:58 +0000

as one of HyBi wg chair, but without having been able to coordinate with my co-chair yet
I do think that we should do this in cooperation with HTTP 2.0
and maybe it would be even better move the work directly in the httpbis wg
as from a HyBi chair prospective I see at moment much more energy in httpbis wg than
in the hybi wg, and it would be great to leverage on that energy to also speed up this piece
of work

the best way I would suggest is to submit a proposal to the httpbis wg in form of a draft
(consider that the deadline to submit a draft is Friday 14th, so you have 5 days, if you
want have a chance for the proposal to be discussed to the next IETF meeting in London
that will start March 2nd).

I would really encourage you do submit a draft, it is OK if it contains alternative options,
and so take the opportunity to discuss it in London

if you need any help in editing the draft I can also offer my help

br
Salvatore

On Jan 25, 2014, at 6:24 PM, Peter Thorson <[email-masked]<mailto:[email-masked]>> wrote:

Critical first question here: Do we have the cooperation of the HTTP 2.0 working group or is the WS over HTTP 2.0 effort something being done unilaterally by the WebSocket working group?

---

From: Yutaka Hirano <[email-masked]>
Date: Tue, 11 Feb 2014 16:25:28 +0900

Hi Salvatore,
Thank you for the suggestions.

as one of HyBi wg chair, but without having been able to coordinate with my
> co-chair yet

I am waiting for Roberto's response. If his response is positive, I will
update the draft and it is OK with me to move this topic to httpbis.

For the RFC draft, I am trying. Though it is unclear if I can make it in
time (I have never written one), I will do my best.

Thanks,

On Mon, Feb 10, 2014 at 6:17 PM, Salvatore Loreto <
[email-masked]> wrote:

>  On Jan 25, 2014, at 6:24 PM, Peter Thorson <[email-masked]> wrote:
>
> Critical first question here: Do we have the cooperation of the HTTP 2.0
> working group or is the WS over HTTP 2.0 effort something being done
> unilaterally by the WebSocket working group?
>
>
>

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 11 Feb 2014 07:29:54 +0000

On Feb 11, 2014, at 9:25 AM, Yutaka Hirano <[email-masked]<mailto:[email-masked]>>
 wrote:

Hi Salvatore,

I am waiting for Roberto's response. If his response is positive, I will update the draft and it is OK with me to move this topic to httpbis.

note that I am only suggesting to submit to httpbis to check if the httpbis is willing (and it has the energy) to work on it or not.
In order to discover it, the best way is to submit a draft (with the several options) to httpbis and have a discussion there
in the httpbis mailing list and preferably also during the f2f meeting in London

For the RFC draft, I am trying. Though it is unclear if I can make it in time (I have never written one), I will do my best.

if you need any help in put the text in an draft format I can help without any problem.

br
Salvatore

Thanks,

On Mon, Feb 10, 2014 at 6:17 PM, Salvatore Loreto <[email-masked]<mailto:[email-masked]>> wrote:

br
Salvatore

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 12 Feb 2014 12:50:11 +0900

On Thu, Jan 9, 2014 at 9:13 PM, Tobias Oberstein <
[email-masked]> wrote:

> > The WS API is explicit about the fact that there is no way from
> JavaScript to
> > generate PINGs or check for PONGs. So do I just create my own,
> application
> > level equivalent, and send a ping-like message every so often and assume
>
> You can have a WS server that does automatic ping-pong for connection
> keep-alive/loss-detection.
>
>
This works for server. For client, app level work is needed.

> Or you can do it on app level from JS.
>
> > the connection is dead when I don't get a reply "soonish"? Are there
> libraries
> > that support such a mechanism? Or is every browser supported to regularly
> > PINGPONG with the server, and call the onclose function when there's no
>
> As far as I know, there is no JS API where you could request the browser
> to do automatic ping-pong upon opening a WS connection.
>
> > answer?
> >
>

Right. We've been saying that we should start discussing such a change on
the API, but haven't.

// Send a ping (w/o body) every 5 sec
ws.pingInterval = 5;
// _Close the WebSocket connection_ when it passed 10 sec without receiving
any byte (not limited to PONG)
ws.timeout = 10;

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 12 Feb 2014 13:36:55 +0900

>> >2: (If the answer for 1 is yes) Should we preserve the RFC6455 semantics?

Complement:

I agree per-frame attribute is evil and am not interested in it. Writing
the compression and mux spec motivated me much not to use per-frame
attribute.

I want to keep control frame injection feature (and as a side effect, frame
boundary will be preserved) because I'm not yet sure throwing it away and
re-wire only ping/pong/close with what's available in HTTP/2.0 framing is
also fine as well as killing per-frame attribute. So, I'm inclined to keep
RFC6455 frame boundary, or provide an opaque box at HTTP/2.0 framing level
that is distinguished from normal DATA traffic. That's why I answered "yes"
to the second question.

---

From: Yutaka Hirano <[email-masked]>
Date: Thu, 13 Feb 2014 01:24:55 +0900

Thank you very much, all!

Roberto gave me comments. Let me share his reply.

> I'm uninterested in requiring all 6455 semantics be replicated. I think of
> frame-level extensions as harmful to the ecosystem. I think we messed up
> there.

I could perhaps be convinced that it it acceptable to do as a compromise,
> but it seems like a kitchen-sink feature that should die.
> Message-level extensions seem reasonable to me.

To Roberto:
I'm not an advocate of frame-level extensions, too.
I once thought a WebSocket connection could be a stream of messages if we
gave up such features, but it was wrong because of the control message
injection, as Takeshi says.
And if we give up RFC6455, we should have two specifications for each
extension, even if it is a message level extension.
Now I think (modified) Plan C is a good plan for you.
It maps WebSocket frame to HTTP/2.0 segment naturally and there is no need
to implement an RFC6455 frame parser, though it isn't widely supported in
this thread yet.

On Fri, Feb 7, 2014 at 4:05 PM, Yutaka Hirano <[email-masked]> wrote:

> On Fri, Feb 7, 2014 at 8:18 AM, Peter Thorson <[email-masked]>wrote:
>
>>
>> > IMHO, we'd have to invent a new API (which, honestly I'd be all for,
>> but that is a completely different set of work).
>>
>> I see the "plan D" option treating WebSocket as HTTP/1.1 is treated by
>> the rest of HTTP/2.0, an existing protocol to be muxed mostly "as is" for
>> the benefit of backwards compatibility with existing systems. I see a
>> Native Messaging option as the "new API” that would allow a cleaner
>> interface for pure HTTP/2.0 applications going forward.
>>
>> For me, the reality is that any piece of HTTP/2.0 software that I write
>> or deploy in the foreseeable future is going to demux the message stream,

---

From: Roberto Peon <[email-masked]>
Date: Wed, 12 Feb 2014 09:08:50 -0800

On should steer away from keep-alive (which is anathema on mobile links),
and use liveness detection when one needs to solve these problems.
(I believe that was what you essentially proposed, Martin)

A PING (application-layer equivalent if necessary) should be automatically
generated when the client has been idle for some time and then has any
message to send. If the PONG is not received within some reasonable timeout
(preferably ~2-3RT or so, or within some other human-scale constant), the
connection should be assumed dead, and another should be created.

imho, this should be done for you by the browser (not that it necessarily
*is*, but it *should*)

Keep-alive is fairly evil at the infrastructure layer, and incents
browsers, servers and carriers to do things to the tear down connection
quickly/prematurely.

Liveness-detection doesn't solve the server notification problem (which
requires at least some level of high-level polling, though hopefully on a
longer timescale), but keep in mind that in seeking perfection, one is
likely to cause a significant less than perfect response from the system
and the players within it.

(It is pretty annoying that we've not developed good APIs or behaviors for
doing decent things on mobile devices...)

-=R

On Tue, Feb 11, 2014 at 7:50 PM, Takeshi Yoshino <[email-masked]>wrote:

> // _Close the WebSocket connection_ when it passed 10 sec without
> receiving any byte (not limited to PONG)
> ws.timeout = 10;
>
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 13 Feb 2014 14:14:04 +0900

> (I believe that was what you essentially proposed, Martin)
I want to confirm my understanding on your terms. Do you mean that
- it's kinda illusion that we are able to keep an end-to-end connectivity
alive
- but should just have a way to detect the peer is dead when wanted

> imho, this should be done for you by the browser (not that it necessarily

Thinking of what browser vendors can do at this point:
a) have built-in default-on ping-generate/timeout-close (try to do
something intelligent like TCP's RTO calculation?) feature in browser. one
concern is that people may have skipped implementation of ping/pong
b) same as (a) but default-off and can be turned on by calling some method
on WebSocket object
c) have one-shot ping sending method on WebSocket object with timeout arg
d) have one-shot ping sending method and pong received event. Sounds
overkilling to me
e) periodical version of (c) or (d)

---

From: Roberto Peon <[email-masked]>
Date: Thu, 13 Feb 2014 11:36:57 -0800

> - but should just have a way to detect the peer is dead when wanted

yup.
Essentially, we don't want traffic black-holed (because the connection is
dead but we haven't detected it), and we don't want to incent "the network"
to kill connections unnecessarily (this is particularly an issue on shared
bandwidth mediums like mobile networks).

>
>
>> A PING (application-layer equivalent if necessary) should be
>> automatically generated when the client has been idle for some time and
>> then has any message to send. If the PONG is not received within some
>> reasonable timeout (preferably ~2-3RT or so, or within some other
>> human-scale constant), the connection should be assumed dead, and another
>> should be created.
>>

We have implemenation experience for (a) for HTTP/2/SPDY, and it seems to
work pretty well.
HTTP2/SPDY has the same problem: if the connection is dead, it is a poor
user experience to attempt to keep sending all traffic down it!

The design rationale behind the choice of (a) was:
it transparently fixed the biggest problem case without requiring any user
involvement or knowledge.
It was important to use ping, because ping/pong requires no interaction
with the application-layer logic, and thus is most likely to be consistent
and have timing similar to TCP's PING.

-=R

>> (It is pretty annoying that we've not developed good APIs or behaviors
>> for doing decent things on mobile devices...)
>>
>> -=R
>>
>>
>>>> As far as I know, there is no JS API where you could request the
>>>> browser to do automatic ping-pong upon opening a WS connection.
>>>>
>>>> > answer?
>>>> >
>>>>
>>>
>>> Right. We've been saying that we should start discussing such a change
>>> on the API, but haven't.
>>>

---

From: Adam Rice <[email-masked]>
Date: Fri, 14 Feb 2014 19:07:01 +0900

I don't think anyone has yet made the point that WebSocket connections
cannot be assumed to be stateless, so browsers cannot implement transparent
reconnection without an extension with some server-side state.

I have not so far seen a compelling argument that reconnection logic is
better implemented as a protocol extension rather than at the application
level. Hypothetically mobile networks could provide some kind of WebSocket
proxy with suspend/resume capabilities, but so far no such thing exists.

Currently WebSocket applications can control keepalive behaviour by sending
(or not) pings from the server.

I see two potential areas for improvement:
1) Client side detection of loss of connection
2) More efficient liveness detection

1) could be resolved with something as simple as an "expect_ping_when_idle"
extension which told the client about whatever keepalive policy the server
was attempting to enforce.

Roberto seems to be implying that 2) may in fact be counter-productive,
although I'm not sure I fully understand the argument.

Adam Rice

On 14 February 2014 04:36, Roberto Peon <[email-masked]> wrote:

>>> On should steer away from keep-alive (which is anathema on mobile
>>> links), and use liveness detection when one needs to solve these problems.
>>> imho, this should be done for you by the browser (not that it
>>> necessarily *is*, but it *should*)
>>>
>>
>> b) same as (a) but default-off and can be turned on by calling some
>> method on WebSocket object

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 14 Feb 2014 11:52:40 +0000

at some point a couple of years ago 
we were trying to work on something at the http level:
http://tools.ietf.org/id/draft-loreto-http-timeout-00.txt

but we didn't have enough energy at time to finalise it

/salvatore

On Jan 9, 2014, at 1:51 PM, Martin J. Dürst <[email-masked]> wrote:

> This is a problem that I have already encountered in three different applications, and think there must/should be a well-known solution for. It's not strictly standards-related, but I'm sending it to this lists because I think that's where the experts are.
> 
> It's about detecting broken connections on the application level. Setting up a WS connection and using it while it's working is very easy. However, what is supposed to happen when e.g. the server dies or the cable gets cut (directly or figuratively), or the PC is put to sleep for a while, and then wakes up again?
> 
> The WS API is explicit about the fact that there is no way from JavaScript to generate PINGs or check for PONGs. So do I just create my own, application level equivalent, and send a ping-like message every so often and assume the connection is dead when I don't get a reply "soonish"? Are there libraries that support such a mechanism? Or is every browser supported to regularly PINGPONG with the server, and call the onclose function when there's no answer?
> 
> Any advice or pointers appreciated!
> 
> Regards,   Martin.

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 14 Feb 2014 12:01:38 +0000

On Feb 14, 2014, at 12:07 PM, Adam Rice <[email-masked]<mailto:[email-masked]>> wrote:

I don't think anyone has yet made the point that WebSocket connections cannot be assumed to be stateless, so browsers cannot implement transparent reconnection without an extension with some server-side state.

I have not so far seen a compelling argument that reconnection logic is better implemented as a protocol extension rather than at the application level. Hypothetically mobile networks could provide some kind of WebSocket proxy with suspend/resume capabilities, but so far no such thing exists.

that is a great business idea, but unfortunately it would only work for WebSocket in clear (i.e. on port 80)

/Sal

Currently WebSocket applications can control keepalive behaviour by sending (or not) pings from the server.

1) could be resolved with something as simple as an "expect_ping_when_idle" extension which told the client about whatever keepalive policy the server was attempting to enforce.

Roberto seems to be implying that 2) may in fact be counter-productive, although I'm not sure I fully understand the argument.

Adam Rice

On 14 February 2014 04:36, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:

On Wed, Feb 12, 2014 at 9:14 PM, Takeshi Yoshino <[email-masked]<mailto:[email-masked]>> wrote:
On Thu, Feb 13, 2014 at 2:08 AM, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:
On should steer away from keep-alive (which is anathema on mobile links), and use liveness detection when one needs to solve these problems.
(I believe that was what you essentially proposed, Martin)

I want to confirm my understanding on your terms. Do you mean that
- it's kinda illusion that we are able to keep an end-to-end connectivity alive
- but should just have a way to detect the peer is dead when wanted

yup.
Essentially, we don't want traffic black-holed (because the connection is dead but we haven't detected it), and we don't want to incent "the network" to kill connections unnecessarily (this is particularly an issue on shared bandwidth mediums like mobile networks).

A PING (application-layer equivalent if necessary) should be automatically generated when the client has been idle for some time and then has any message to send. If the PONG is not received within some reasonable timeout (preferably ~2-3RT or so, or within some other human-scale constant), the connection should be assumed dead, and another should be created.

imho, this should be done for you by the browser (not that it necessarily *is*, but it *should*)

Thinking of what browser vendors can do at this point:
a) have built-in default-on ping-generate/timeout-close (try to do something intelligent like TCP's RTO calculation?) feature in browser. one concern is that people may have skipped implementation of ping/pong
b) same as (a) but default-off and can be turned on by calling some method on WebSocket object
c) have one-shot ping sending method on WebSocket object with timeout arg
d) have one-shot ping sending method and pong received event. Sounds overkilling to me
e) periodical version of (c) or (d)

We have implemenation experience for (a) for HTTP/2/SPDY, and it seems to work pretty well.
HTTP2/SPDY has the same problem: if the connection is dead, it is a poor user experience to attempt to keep sending all traffic down it!

The design rationale behind the choice of (a) was:
it transparently fixed the biggest problem case without requiring any user involvement or knowledge.
It was important to use ping, because ping/pong requires no interaction with the application-layer logic, and thus is most likely to be consistent and have timing similar to TCP's PING.

-=R

Keep-alive is fairly evil at the infrastructure layer, and incents browsers, servers and carriers to do things to the tear down connection quickly/prematurely.

Liveness-detection doesn't solve the server notification problem (which requires at least some level of high-level polling, though hopefully on a longer timescale), but keep in mind that in seeking perfection, one is likely to cause a significant less than perfect response from the system and the players within it.

(It is pretty annoying that we've not developed good APIs or behaviors for doing decent things on mobile devices...)

-=R

On Tue, Feb 11, 2014 at 7:50 PM, Takeshi Yoshino <[email-masked]<mailto:[email-masked]>> wrote:
On Thu, Jan 9, 2014 at 9:13 PM, Tobias Oberstein <[email-masked]<mailto:[email-masked]>> wrote:
> The WS API is explicit about the fact that there is no way from JavaScript to
> generate PINGs or check for PONGs. So do I just create my own, application

You can have a WS server that does automatic ping-pong for connection keep-alive/loss-detection.

> the connection is dead when I don't get a reply "soonish"? Are there libraries

As far as I know, there is no JS API where you could request the browser to do automatic ping-pong upon opening a WS connection.

> answer?

Right. We've been saying that we should start discussing such a change on the API, but haven't.

// Send a ping (w/o body) every 5 sec
ws.pingInterval = 5;
// _Close the WebSocket connection_ when it passed 10 sec without receiving any byte (not limited to PONG)
ws.timeout = 10;

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

---

From: Roberto Peon <[email-masked]>
Date: Fri, 14 Feb 2014 11:21:08 -0800

Basically, keep-alives ties up enough network resources and/or device
battery life that OS providers and network providers have strong incentive
to consider a connection idle (and close it/blackhole it) for smaller and
smaller time periods, eventually resulting in an arms race which increases
inefficiencies everywhere.

-=R

On Fri, Feb 14, 2014 at 2:07 AM, Adam Rice <[email-masked]> wrote:

> 1) could be resolved with something as simple as an
> "expect_ping_when_idle" extension which told the client about whatever
> keepalive policy the server was attempting to enforce.
>

---

From: Adam Rice <[email-masked]>
Date: Mon, 17 Feb 2014 11:56:46 +0900

Consider an earthquake early-warning web app. It can provide a 15-second
warning of an event which only happens a couple of times a year. If the
warning is more than 5 seconds late, it is effectively useless.

Polling would be hideously inefficient for this use case. Even if the
OS/network forces you to ping every 5 minutes to keep the connection alive,
it is still going to be several orders of magnitude more efficient than
polling.

There are legitimate uses for long-lived, idle connections. WebSocket is
currently the least-worst browser technology available to satisfy them. If
we can make it less bad, I think that would be worthwhile.

With respect for Salvatore's point, if the suspend/resume facility existed
you could in principle tunnel SSL over it (ie. within the WebSocket
payload) and still get the benefit without trusting the proxy more than you
already have to trust your network provider anyway.

Adam

On 15 February 2014 04:21, Roberto Peon <[email-masked]> wrote:

>>> Essentially, we don't want traffic black-holed (because the connection
>>> is dead but we haven't detected it), and we don't want to incent "the
>>> network" to kill connections unnecessarily (this is particularly an issue
>>> on shared bandwidth mediums like mobile networks).
>>>
>>>
>>>>
>>>>
>>>>> Liveness-detection doesn't solve the server notification problem
>>>>> (which requires at least some level of high-level polling, though hopefully
>>>>> on a longer timescale), but keep in mind that in seeking perfection, one is

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 17 Feb 2014 02:19:32 -0800

Indeed, there are applications with a valid desire to keep connections alive over a long period, and not only that,
but also keep network links in a low-latency state.

I once did some experiments with WebSocket on a mobile 3.5G network, where keeping the WebSocket connection
in a low-latency state ("HSPA DCH") required not only occasional pings/pongs, but a constant load of 4kb/s

http://lists.w3.org/Archives/Public/ietf-http-wg/2012JanMar/1083.html

WebSocket's ability of inducing traffic unrelated to app level using pings/pongs proved to be useful here.

Sidenote: as far as I know, IE11 is currently the only browser that actively uses WebSocket ping/pong (it sends unsolicited _pongs_ every couple of seconds upstream).

Ideally, there would be an API exposed to user apps to give hints to the radio link layer on mobile devices, like: keep up radio for the next 800ms, then go to low power.

This would make sense for the uplink direction. The downlink is more problematic. Unless it's periodic.

FWIW, here are 2 related resources

http://de.slideshare.net/zahidtg/gsma-fast-dormancy-best-practices
https://eggert.org/students/chemmagate-thesis.pdf

/Tobias

Von: hybi [mailto:[email-masked]] Im Auftrag von Adam Rice
Gesendet: Montag, 17. Februar 2014 03:57
An: Roberto Peon
Cc: [email-masked]
Betreff: Re: [hybi] Detection of broken WS connections

Consider an earthquake early-warning web app. It can provide a 15-second warning of an event which only happens a couple of times a year. If the warning is more than 5 seconds late, it is effectively useless.

Polling would be hideously inefficient for this use case. Even if the OS/network forces you to ping every 5 minutes to keep the connection alive, it is still going to be several orders of magnitude more efficient than polling.

There are legitimate uses for long-lived, idle connections. WebSocket is currently the least-worst browser technology available to satisfy them. If we can make it less bad, I think that would be worthwhile.

With respect for Salvatore's point, if the suspend/resume facility existed you could in principle tunnel SSL over it (ie. within the WebSocket payload) and still get the benefit without trusting the proxy more than you already have to trust your network provider anyway.

Adam

On 15 February 2014 04:21, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:
Basically, keep-alives ties up enough network resources and/or device battery life that OS providers and network providers have strong incentive to consider a connection idle (and close it/blackhole it) for smaller and smaller time periods, eventually resulting in an arms race which increases inefficiencies everywhere.

-=R

On Fri, Feb 14, 2014 at 2:07 AM, Adam Rice <[email-masked]<mailto:[email-masked]>> wrote:
I don't think anyone has yet made the point that WebSocket connections cannot be assumed to be stateless, so browsers cannot implement transparent reconnection without an extension with some server-side state.

Adam Rice

yup.

-=R

-=R

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 17 Feb 2014 20:09:36 +0000

On Feb 17, 2014, at 4:56 AM, Adam Rice <[email-masked]<mailto:[email-masked]>> wrote:

sorry then I don't understand how the proxy can provide support if the suspend/resume facility are tunnelled over SSL

/Sal

---

From: Adam Rice <[email-masked]>
Date: Tue, 18 Feb 2014 11:32:50 +0900

For suspend/resume, the existence of data is sufficient to make a decision
about whether a connection is needed or not. The contents of the data need
not be understandable.

Concretely, the client starts by sending a request to the proxy with
headers like

Sec-WebSocket-Extensions: suspend-resume
Sec-WebSocket-Protocol: mux-proxy

The proxy responds, accepting these extensions and protocol and
establishing the connection.

Then for the browser to connect to a wss site it sends a message like:

CONNECT wss.example.com:433

After which all messages on that multiplexed channel are encrypted with the
key of wss.example.com, and the proxy cannot see their contents.

When the channel is idle, the browser sends a frame with opcode Suspend,
the proxy confirms with opcode Suspend, and the connection between the
browser and the proxy is closed. The connection between the proxy and
wss.example.com stays open.

If the browser has a message to send, it reconnects to the proxy and sends
a Resume frame.

If the proxy receives data from wss.example.com while suspended, it does
whatever tranport-specific magic is needed to notify the browser, and then
the browser does the "Resume" handshake as above, and receives the pending
data.

On 18 February 2014 05:09, Salvatore Loreto
<[email-masked]>wrote:

>
>  On Feb 17, 2014, at 4:56 AM, Adam Rice <[email-masked]> wrote:
>
>  sorry then I don't understand how the proxy can provide support if the
> suspend/resume facility are tunnelled over SSL
>
>  /Sal
>
>