Subject: [hybi] Requirement: extensions and sub protocols

From: Salvatore Loreto <[email-masked]>
Date: Wed, 03 Mar 2010 22:44:29 +0200

Hi,

another aspect of the WebSocket protocol we have to discuss is 
extensibility.
As usual here a strawman proposal just to start the discussion...

    Req. The WebSocket Protocol MUST be designed to easily support
    extensions.

    Req. The WebSocket Protocol MUST have a way to declare/negotiate
    extensions.

    Req. The WebSocket Protocol MUST be designed to easily support sub
    protocols

    Req. It SHOULD be easy for sub protocols implementing features like
             channels, multiplexing, (what else?)

please discuss extensions and subprotocols requirements (and feel free 
to suggest better text)
as usual we need to measure the level of the consensus on them

all thoughts or comments are very well welcome and appreciated.

cheers
/Sal

www.sloreto.com

---

From: KOMATSU Kensaku <[email-masked]>
Date: Thu, 4 Mar 2010 10:32:03 +0900

RTT measurements between client and server will be required for some
extensions or sub-protocol. For example, TCP flow-control is so
sensitive both content-size and RTT, so these two-parameters will be
useful for some application's negotiation-phase which is better to use
multiplexing frames in one channel or to use independent channels for
each frames.

Sincerly

---
Kensaku KOMATSU

2010/3/4 Salvatore Loreto <[email-masked]>:
> Req. The WebSocket Protocol MUST be designed to easily support extensions.
>
> Req. The WebSocket Protocol MUST have a way to declare/negotiate extensions.
>
> Req. The WebSocket Protocol MUST be designed to easily support sub protocols
>
> please discuss extensions and subprotocols requirements (and feel free to
> suggest better text)
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 00:51:29 +0000

>    all thoughts or comments are very well welcome and appreciated.

I would like to see a MUST-level placeholder for transport-level
options, where the agents (including proxies) can do things like
opportunistic multiplexing, keepalive merging and negotiation, and so
on without the application having to know about these things at all.

There may still be room for the application to explicitly disable
transaport-level things it doesn't want used, and to be informed about
what does get used if it cares.  But applications which don't care and
use just the basic API should not prevent transport optimisations from
being used automatically by future agents.

We don't necessarily need any actual options to be standardised, only
a mechanism by which they can be negotiated without breaking communications
with agents that don't support them.

To do this requires:

    - The client can send at least one additional request header (or
      something equivalent) which will be recognised by an aware
      server or proxy, and is guaranteed not to impede processing by
      unaware ones.

    - Same for the response.

    - That a client adding the additional header (or something else)
      does not interfere with application-level protocol negotiation.
      This is why it cannot be part of the application-specified
      protocol field.

    - That a server responding to the additional header (or something
      else) does not have false positivies due to matching something
      which could be sent by the application.  This is a second reason
      why it cannot be part of the application-specified protocol
      field.

    - Ideally it would be subject to many of the same validations
      constraints as the main WebSocket handshake:

       + That it's likely to be modified by unaware proxies, or even
         better, some options likely to be modified, some not, so that
         hop-by-hop transport techniques can be negotiated separate
         from end-to-end techniques.  The "Connection" header may be
         useful, if it works.  Perhaps there is a role for "Expect",
         but I vaguely recall reports (on ietf-http-wg list, I think)
         that Expect is not reliably acted upon despite being a MUST
         level requirement in RFC2616.

       + That a "dumb" HTTP server which is not a WebSocket server is
         unlikely to appear, to the client, to understand these
         options by simply reflecting them back.

       + That "dumb" WebSocket servers ;-) must not appear to understand
         these options by simply reflecting them back or similar.

I don't know if that is every detail required.  Hopefully the general
principle is conveyed, and maybe the general principle is enough for
the Requirements document.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 00:57:16 +0000

> multiplexing frames in one channel or to use independent channels for

When you say "use independent channels", do you mean independent TCP
connections?

Because "channel" is often used to mean the individual flows that are
carried inside a multiplexed connection.

Assuming you do mean independent TCP connections:

Can you say a bit more about the circumstances when an endpoint would
choose to use them instead of multiplexing?  What kind of parameter values
are needed for multiple TCPs to be better?

I have not heard of this before, although I appreciate that multiple
TCPs can be faster than a single shared TCP in some scenarios.  (I'm
thinking of bonded-ADSL where each TCP is confined to exactly one of
the links).

Sometimes 4 TCPs is better than 1, but worse than 100, when you have
100 individual flows to transport.  But I'm not sure how to decide
what to use automatically.

Note that full end-to-end RTT, including processing latency, can be
measured by applications sending each other messages.

-- Jamie

---

From: KOMATSU Kensaku <[email-masked]>
Date: Fri, 5 Mar 2010 15:18:38 +0900

> Assuming you do mean independent TCP connections:

Sorry, your assumption is right. I meant "independent TCP connections".
I'll take care about this from now on.

> measured by applications sending each other messages.

Per TCP connection transmission rate is calculated by the formula below.
 <rate> [Mbps] = 65535 / (125 * RTT[msec] )
(I also assumed that TCP receive buffer size = 64KB (winXP's default value))
This formula is reasonable, in my experience, because either the
theory value or the actual measurement value are not so different.

For example.
RTT : 10msec ... 52Mbps
RTT : 15msec ... 35Mbps
RTT : 50msec ... 10Mbps
RTT : 100msec ... 5Mbps

So, if someone wants to provide their service with transmission rate = 10Mbps,
1 tcp connection is enough for RTT < 50msec users. Though, 2 or 3 connection
is required for RTT=100msec users.

I also agree with multiple connections are confortable for web
application developer,
because it offeres an appropriate speed in most cases without
bothersome control.
But this manner requires more connection resources than necessary.

Number of TCP connection affects to connection-sensitive equipment,
global-NAT, proxy, firewall,load-balancer, servers, .... If we use
appropriate number of connection and multiplexing frames for web
applications, the system cost can be made low-priced. And I hope that
this senario will also pleased network providers.

To achieve this senario, preparing some libraries, defining
sub-protocol or extentions and making it easier to use will be
important. That's why I mentioned this-topic before.

For other case, It'll be useful for video service. Because it'll be
useful for the decision of the bandwidth of contents automatically.

Sincerly
---
Kensaku KOMATSU

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 15:44:44 +0100

I think we need to decide who will be doing the extending
and creating new subprotocols.

Ian appears to be advancing the position that if it can't
be done in javascript via the websocket API, then it can't
be done.  Moreover, that if you are not doing it in javascript
then you shouldn't be using websockets.

I disagree with that position (which is hopefully
somewhat close to what Ian actually is saying).

I think that extensions are most likely to be
implemented transparently below the API or
in the path to the server.

Examples that come to mind include:

  Browser implementing a simple multiplexing standard
  to make all websockets from the same tab/window share
  a common connection.

  Browser implementing compression.

  Firewalls acting as aggregators and combining
  multiple base connections into fewer multiplexed
  connections to the business servers.

  Appliances doing SSL offload and converting wss to ws
  connections with injected certificate information.

I also think that there will be extensions done
in javascript by frameworks/applications, but by
definition they work above the API and need no more
support from the base protocol than the API already
provides.

regards

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 05 Mar 2010 18:28:15 +0200

here we are talking about *protocol* requirements
it is not the right place to discuss about the actual solution(s).

what we have to decide clearly is:

1) do we want WebSocket protocol to be an extensible protocol? yes or not?

2) if yes, how do we want this extension be designed
     I mean as extension of the WebSocket (at same layer)
     or as possible layer on top of Websocket.

so please provide feedback on this not on the actual solution.

of course we have to keep in mind that all the extensions MUST not 
complicate
the *basic* protocol of a bit!

cheers
Sal

Salvatore Loreto
www.sloreto.com

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 17:48:01 +0100

> it is not the right place to discuss about the actual solution(s).

I thought I was?

Isn't it important to have a definition of what an extension
is (including who is doing the extension) as part of the requirements?

regards

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 05 Mar 2010 19:31:41 +0200

> is (including who is doing the extension) as part of the requirements?
it is more important have a *reason* for the proposed requirement,
definition can mislead and be interpreted as a different requirement or 
a solution.

at moment we do not have any requirement that talk about the relation
with WebSocket API...
and to tell the true I am a bit surprised W3C WebApps has not proposed
any requirements.
I'll start a separate trade about it

cheers
Sal

Salvatore Loreto
www.sloreto.com

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 18:34:49 +0000

> definition can mislead and be interpreted as a different requirement or 

Well, in regard to extensions, it's not _just_ about the relationship
with the Javascript API.  Other things like opportunistic connection
sharing, aggregators and SSL offload may be completely invisible to
the API (and therefore have no relationship with it), but still need
the base protocol to guarantee to behave in a particular way.

> at moment we do not have any requirement that talk about the
> relation with WebSocket API...  and to tell the true I am a bit
> surprised W3C WebApps has not proposed any requirements.

If you read Ian's recent responses, many of them talk about the
relationship with the API driving specific features or lack thereof.

Even though the queries leading to those responses were from
implementors who aren't using the API and therefore aren't constrained
by it.

I think there is quite some confusion at the moment, among various
people, about which parts of the WebSocket protocol are free to be
used by implementors - especially the type bytes.  That has some
relationship with the API.

Those people expecting to use application-specific type bytes aren't
using the API, but the API is given as a reason why they should not do
it.  This reflects some confusion between constraints due to the API,
and constraints from the protocol definition which should be explicit,
but are implied by the API.

> I'll start a separate trade about it

It's a good idea, thanks.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 18:50:25 +0000

> somewhat close to what Ian actually is saying).

I'm a bit confused about what Ian's saying in that regard, so I don't
know if it's close or not :-)

But anyway, my position is this:

   - If it *can* be done in Javascript via the API, that's a good thing.
     It means the API is versatile enough.

   - The API isn't relevant to non-browser agents, and constraints
     implied by the API don't affect them.  Only the protocol matters,
     so all constraints affecting the protocol should be written in the
     protocol spec, not inferred from the Javascript API.

Categories:

   1. Some extensions really should remain in Javascript.  There is no
      advantage to complicating the browser for them.

   2. Some extensions are *better* if done by the browser, but it is
      still good to have the ability to fall back to Javascript on an
      older browser.  Multiplexing connections from a single tab/window
      falls into this category.

   3. Some extensions cannot be done from Javascript at all.
      Opportunistically multiplexing connections from all tabs/windows
      to the same destinations (or proxy if there is one) falls into
      this category.  Practically, certain buffering strategies fall
      into this too, but hints from Javascript may be valuable.

It might be good to identify which extensions fall into each category.

Right now, all the extensions involving binary frames and chunking
have no relationship with the Javascript API at all, because it
doesn't support those frames.  It's not clear what the future plan is
for this.  Therefore it's not clear which category they'd fall into.

>   connections with injected certificate information.

These two go well together, by the way.  Fewer connections means fewer
SSL negotiations.

> support from the base protocol than the API already

However, if the base protocol grows some things which help
non-Javascript extensions, such as the chunk no-delay flag we've been
discussing, in some cases it may be useful to expose those
capabilities to Javascript as well.  E.g. I can think of a few games
writable in Javascript which could benefit from that.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 19:09:37 +0000

KOMATSU Kensaku wrote:
> So, if someone wants to provide their service with transmission rate
> = 10Mbps, 1 tcp connection is enough for RTT < 50msec users. Though,
> 2 or 3 connection is required for RTT=100msec users.

That was true a few years ago, but things have changed.

I believe Windows later than XP, i.e. Vista and Windows 7, now
automatically enlarge the receive window according to heuristic
measurements.

Current Linux also does the same thing.  It will set the receive
window up to 3.4Mbyte on my laptop, for example, if it thinks the path
needs that much for throughput.

It is for exactly the reason you've described: the old 64k window
limit was too small to get good throughput on 10Mbps links with 50ms
latency, which are getting common.

For that reason, there's no benefit to having several parallel TCPs
any more on that sort of link, on current OSes.

However your point still applies to WebSocket usage on older OSes, and
it may be necessary to use multiple TCPs based on RTT and window size
measurements to get the full throughput on them.

It's a good reason for applications to be able to flag some WebSockets
as "should not be combined with others".  Since there are proxies in
development which automatically combine multiple WebSocket
connections passing through them, it probably should have a way to be
signalled in the handshake, too.

Then again, wouldn't it be even better to multiplex everything
together, and then have a multi-TCP protocol to carry the whole
together when beneficial, so that even one application WebSocket could
get the full link bandwidth?

> important. That's why I mentioned this-topic before.

Exactly, I agree with all of that.

But I think having the WebSocket implementations automatically choose
a number of TCP connections based on dynamic RTT measurement (which
will vary for different destinations, and vary over time) and the
negotiated window size might be simply too difficult.

I think it's better to use OSes which extend the window size
dynamically - they are in a much better position to make the right
measurements, and also to take into account memory used.  It's much
simpler that way too.

> useful for the decision of the bandwidth of contents automatically.

I don't understand that.  To determine bandwidth, something has to
measure the bandwidth.  Counting connections won't do that.  Can you
explain a bit more how you think it will help video delivery?

-- Jamie

---

From: KOMATSU Kensaku <[email-masked]>
Date: Mon, 8 Mar 2010 05:06:24 +0900

> measurements, and also to take into account memory used.  It's much

Thanks a lot!!. I agree with your comment, so I'll choose multiple frames in one
TCP connection in current OSes. I'm sorry my lack of survey.

And, I'll keep challenge automatically choose # of TCP connections based on
RTT measurement for OLD OSes(those don't support above features). Yes, I
know this is hard challenge, but I can't disregard those share.

> explain a bit more how you think it will help video delivery?

Sorry for mis-wording... I meant content's bit rate.

It is based on my old experience... About 7 years before, I served
hi-vision video
services via http/rtsp-tcp streaming. 6Mbps contents worked well when
RTT was small,
but I found that video service(30fps) fall down to picture story(2 or
5 fps) when RTT got
larger. But in those situation, 3Mbps contents was fine(due to tcp
flow control). So,
automatically choose contents-bit rate based on RTT measurement was useful.

But, with your comment, I understand above story is old one.

Sincerly
---
Kensaku KOMATSU