Subject: [hybi] Statement of BWTP requirements

From: Greg Wilkins <[email-masked]>
Date: Sat, 07 Nov 2009 08:27:26 +0900

Primary Requirement
===================

The primary requirements of BWTP are the prime drivers of the BWTP design.
If these requirements cannot be substantially met, then the design/justification
for BWTP should be reviewed.

Bidirectional Web Transport.
----------------------------
To provide bidirectional communication over the web for the same clients,
server, intermediaries and infrastructure currently deployed for HTTP and
able to carry at least same semantic content.

A typical network environment that BWTP would need to traverse is:

 Client <-> local firewall <-> ISP firewall <-> network <-> data center firewall <-> load balancer <-> Server

Diverse clients
---------------
The target clients include, but are not limited to web applications running
within browsers. Other potential clients include may clients that are currently
using HTTP to contact server, such as:

 + thick clients using SOAP, REST or other HTTP based protocol.
 + peer to peer clients
 + protocols such as BOSH, Bayeux carried over HTTP.
 + The websocket JS API of HTML5

HTTP Transport
--------------
BWTP should be able to carry HTTP traffic as well as bidirectional
traffic, so that it can represent a better usage of an existing
connection, rather than the creation of additional connections.

HTTP Addressing
----------------
The same addressing model as HTTP will be used, with all messages being
sent to or from a server side URI.  Like HTTP, messages sent from the URI may
either be public ( usable by multiple components/frames/clients ) or private
(aka non cacheable and destined to single component/frame/client).
While the pure concept of cacheability is unlikely to be sufficient for
bidirectional message addressing, it is important to note that HTTP
already has the concept of a message intended for 1 receiver vs a
message intended for many.

HTTP Security
-------------
When appropriate for the client, the current and proposed origin security
models of HTTP should be applicable to BWTP.

TCP/IP usage
------------
Where possible and applicable, BWTP should make use of TCP/IP (or other
network layer) features - such as flow control.

Secondary Requirements
======================

The secondary requirements are the "nice to have" features.  However, if some
are mutually incompatible or too expensive, then it is not mandatory for all
secondary requirements to be met.

To resolve issues of current Bidirectional HTTP transports
----------------------------------------------------------

** Streaming/Long Polling break HTTP connection management **
HTTP's connection management assumes that requests will be short,
so that a limited(2) connection pool can be shared between many
XHR uses many frames/tabs/windows.  The current bidirectional
techniques of long polling or streaming break this assumption
and effectively remove a connection from the pool, causing
resource starvation if multiple long polls are performed or
unreasonable delay if normal requests are queued behind long
polls.

BWTP should unbreak the connection management for bidirectional
communication.

** differing timeouts of client, server and intermediaries **
Clients, servers and intermediaries all have timeouts that
are individually determined and non discoverable. Since
bidirectional usage of HTTP is more efficient if longer
timeouts are used, there is often a problem with timeouts
being inadvertently exceeded.

BWTP should be designed so as to avoid such timeout issues,
but maintain the ability of any party to close a connection.

** Connected status of clients **
Bidirectional web applications typically need to track which
clients are connected to a server so that server can know
which clients messages can be sent to and can trigger
actions when a client connections and/or disconnects. Current
bidirectional techniques require the use of timeouts and
probabilistic assumptions to determine client connected
status.

BWTP should be designed to improve the detection of
connected and disconnected transitions.

Improve on some HTTP issues.
----------------------------
** Reduce verbosity and/or redundancy of meta data **
HTTP can send many verbose and/or redundant headers
with each message.

BWTP will be designed to reduce both verbosity and
redundancy of meta data.

** Fix pipelining **
A HTTP connection can only be used to send multiple
sequential requests without waiting for responses in
a very limited set of circumstances and the delivery
status of some messages can be unclear if the connection
is closed before all responses are received.

BWTP should be able to transport multiple sequential
HTTP requests in more circumstances and without less
uncertainty of delivery.

** Improved delivery guarantees **
A client, server or intermediary may close an idle connection
at any time, so that messages sent by any party immediately
before the close may not be delivered and applications
have no mechanism to distinguish between are routine idle close
and a network failure.

BWTP will be designed to remove the uncertainty of delivery
status of messages sent when connections are routinely closed.

** Out of order responses **
With HTTP requests may be sent in a pipeline in
some circumstances.  However, even if the requests
are not related and not mutually dependent, the responses
can only be sent in the order that the requests were
received.

BWTP should allow unrelated HTTP requests to be
transported so that responses may be returned in
arbitrary order.

To be a better transport for the Websocket API
-----------------------------------------------
BWTP aims to be a superior transport for the
Websocket API than the proposed Websocket protocol.
Specifically BWTP aims to provide a websocket
transport with at least the capabilities and
desirable attributes already possessed by HTTP.
Note that these extra capabilities need not
be exposed in the websocket API, but can be
used instead by browsers ,intermediaries and
server implementations.

** Better specification **
The BWTP specification will be written using ABNF and
non-algorithmic style. The protocol will be designed
to be strict in what is generated and forgiving in
what is parsed.

** Extensible by infrastructure not only application **
The BWTP protocol should meet the requirements of the
websocket API, but to do so in an extensible manners so
that other bidirectional web clients can share a common
basis, infrastructure, intermediaries and servers as
browser clients.

** Symmetric resource access **
The websocket protocol is currently asymmetric in the
ability send/receive messages from a resource.   At
any time a client may send a message to a server resource
(by opening a new connection), but a server resource may
only send a message to a client if that client has opened
a specific connection to the resource.

BWTP will be designed so that a server side resource
can send a message to any connected client, within the
bounds of the security model.

** Simplified Client connected status **
Bidirectional web applications typically need to track which
clients are connected to a server.  However websockets
connection model is complex as there is no concept of client
and connections are made directly from client component to server
resource. A single client may even be connected to multiple
nodes within a clusters, making it exceedingly difficult for
an application to determine if a client is present or not.

BWTP will be designed so that it may be simply determined
of a client is connected and that there will not be a
partially connected status.

** Client state manipulation **
Websocket allows client cookies to be set only during
connection establishment, while current long polling
solutions allow a set cookie to be sent at any time,
which can assist with tasks such as authentication,
load migration, failover and user preferences.

BWTP will be designed so that client cookies may
be set by the server at any time, within the bounds
of the security model.

** Cached resources **
HTTP resources are by default cacheable unless
otherwise specified.   Websocket messages from a
resource are never cacheable.

BWTP will be designed so that messages from a resource
are by default not cacheable, but optionally may be
taged so that they are cacheable.  A server may thus
use BWTP to override the expires value of previously
cached content.

** Reduce connection usage **
Websocket uses a TCP/IP connection per user per
client component to server resource pairing.

Thus if U users have C client side components
and S server components and each client can have
H HTTP connections, then connections needed
by websocket protocol are in the range
from O(U*max(C,S)) up to O(U*C*S+U*H).

BWTP will be designed so that connection usage
can approach be O(U)

** Fairness **
Websocket allows an application to appropriate a
larger share of network and server resource by opening
multiple connections.

BWTP will disassociate the application interfaces
from connection management and provide a fair resource
allocation between clients.

** Improved delivery guarantees **
Websocket suffers from the same idle close issue that
HTTP pipelining suffers from.

BWTP will be designed to remove the uncertainty of delivery
status of messages sent when connections are routinely closed.

** No sentinel framing **
Sentinel framing is inefficient to parse and vulnerable to
insertion attacks, so that all data must be scanned for
sentinel characters.

BWTP should not use sentinel framing, not even CRLFCRLF
delineation of meta data from content.

** Improved involvement with intermediaries **
Websocket does not allow intermediaries to be explicitly
involved in a connection in a similar way to how HTTP
currently defines hop by hop mechanisms.

BWTP will allows intermediaries to be first class participants
in bidirectional communications.

Non Requirements
================

The following are NOT in themselves requirements of BWTP, but some may be
part of the solution to other requirements.

Text or Binary Protocol
-----------------------
There is no requirement for the protocol to either be text based
or binary based.   Either will work for the semantics required
and the choice is a matter of taste and rough consensus.

Channels or Virtual Connections
-------------------------------
There is no direct requirement to have channels or some other form
of virtual connection.  However such a mechanism is a likely candidate
to meet the connection based requirements.