Subject: [hybi] BWTP Proposal - Bidirection Web Transfer Protocol

From: Greg Wilkins <[email-masked]>
Date: Wed, 10 Jun 2009 19:24:02 +1000

All,

Attached in a very very early draft of an alternative proposal for a
transfer protocol for HTML5 WebSockets.  For lack of a better
name (and not wanting to coopt the hybi name), I've called it

  BWTP - Bidirection Web Transfer Protocol

I wont explain the protocol here, as hopefully the proposal
document does that sufficiently well.

But I will explain why I think this proposal (or something
approximately similar) is superior to the current Websocket
protocol proposal:

 + The protocol is very much an IETF style application
   protocol.  In fact it is just RFC2616 with anything
   non bidirectional ripped out.
   It is not trying to be a revolution in web protocols,
   but simply to solve the problems at hand.

 + The protocol document is written very much in IETF style.
   In fact it is just RFC2616 with anything non bidirectional
   ripped out.   BNF is used to specify the protocol and the
   principal of "be strict in what you generate and forgiving
   in what your parse" is adhered to.

 + Because of it's similarity to HTTP, it is intended that
   existing HTTP clients, servers and intermediaries will be able
   to be minimally updated to support this protocol.
   This will not require entirely new protocol libraries
   to be developed.   Existing development and analysis
   tools will also be able to be easily updated, plus
   the protocol is mostly human readable.

 + It supports full mime encapsulated payloads, but has
   a default mechanism so that most values only need to
   be specified once per connection if they do not change.

   The minimal overhead per message is 11 bytes, which is
   a little more than the websocket proposal, but is hardly
   significant.

 + There is no formal channel mechanism like BEEP has,
   but each message may be to/from a different URI if
   need be.  This makes multiplexing easy to support.

 + There is no formal segmentation mechanism, but
   Content-Ranges are supported so that large content
   cna be sent in smaller bits if desired.

 + The protocol recognizes that intermediaries (proxies) may
   wish to be an active party on a bi direction connection.
   For example, this proposal allows an intermediary to
   initiate and orderly lossless close of the connection.
   I'm sure innovative proxy applications will be
   developed over time, just as they have been for HTTP.

 + it well supports the current HTML WebSocket API
   but is also flexible and extensible so that non
   browser clients may use it and future APIs will
   not need protocol changes.

I'd really appreciate any feedback, co-authors, review,
editing etc.   I don't consider anything in this proposal
to be fixed and all is open to change.

I have the draft in Google Docs and am happy to add
collaborators (there are a few already (who should add
themselves to the authors/acknowlegements)).

cheers
                                                     Greg Wilkins

        Bidirectional Web Transfer Protocol -- BWTP/1.0

Status of this Memo

   This document is written by somebody who knows nothing of IETF
   procedures.  It is desperately in need of co-authors and review.
   It is intended to be an alternative proposal for the
   transport protocol for the WebSocket API from HTML 5.

   This document is heavily based on RFC2616 and borrow structure,
   ideas and text from that document.

   currently the formatting is all manual and the section numbering
   has been kept similar to RFC2616

Copyright Notice

   Copyright (C) The Internet Society (2009).  All Rights Reserved.

Abstract

   The Bidirectional Web Transfer Protocol (HTTP) is an application-level
   protocol that can be used to upgrade an RFC2616 HTTP connection 
   to a bidirectional channel capable of exchanging messages.

Table of Contents

   TBD

1 Introduction

1.1 Purpose

   The Hypertext Transfer Protocol (HTTP) is an application-level
   protocol for distributed, collaborative, hypermedia information
   systems. HTTP has been in use by the World-Wide Web global
   information initiative since 1990, but it is intrinsically a
   request/response protocol and cannot well service the growing
   use-case for data to be pushed from a server to a client.

   The Bidirectional Web Transfer Protocol (BWTP) is an application-level
   protocol that can be used to upgrade a HTTP/1.1 connection to
   be able to exchange bidirectional MIME-like messages, 
   containing metainformation about the data transferred.
   
   Messages are passed in a format similar to that used by HTTP/1.1
   as defined by RFC2616. It is intended that implementations
   of HTTP clients, servers and intermediaries will be able to
   be minimally updated in order to support BWTP.

1.2 Requirements

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [34].

   An implementation is not compliant if it fails to satisfy one or more
   of the MUST or REQUIRED level requirements for the protocols it
   implements. An implementation that satisfies all the MUST or REQUIRED
   level and all the SHOULD level requirements for its protocols is said
   to be "unconditionally compliant"; one that satisfies all the MUST
   level requirements but not all the SHOULD level requirements for its
   protocols is said to be "conditionally compliant."

1.3 Terminology

   This specification uses a number of terms to refer to the roles
   played by participants in, and objects of, the HTTP or BWTP communication.

   connection
      A transport layer virtual circuit established between two programs
      for the purpose of communication.

   message
      The basic unit of BWTP communication, consisting of a structured
      sequence of octets matching the syntax defined in section 4 and
      transmitted via the connection.

   request
      An HTTP request message, as defined in section 5 of RFC 2616.

   response
      An HTTP response message, as defined in section 6 of RFC 2616.

   resource
      A network data object or service that can be identified by a URI,
      as defined in section 3.2. Resources may be available in multiple
      representations (e.g. multiple languages, data formats, size, and
      resolutions) or vary in other ways.

   client
      A program that establishes connections for the purpose of sending
      requests.

   user agent
      The client which initiates a connection. These are often browsers,
      editors, spiders (web-traversing robots), or other end user tools.

   server
      An application program that accepts connections.
      Any given program may
      be capable of being both a client and a server; our use of these
      terms refers only to the role being performed by the program for a
      particular connection, rather than to the program's capabilities
      in general. Likewise, any server may act as an origin server,
      proxy, gateway, or tunnel, switching behavior based on the nature
      of each message.

   origin server
      The server on which a given resource resides or is to be created.

   proxy
      An intermediary program which acts as both a server and a client
      for the purpose of making requests on behalf of other clients.
      Requests are serviced internally or by passing them on, with
      possible translation, to other servers. A proxy MUST implement
      both the client and server requirements of this specification. A
      "transparent proxy" is a proxy that does not modify the request or
      response beyond what is required for proxy authentication and
      identification. A "non-transparent proxy" is a proxy that modifies
      the request or response in order to provide some added service to
      the user agent, such as group annotation services, media type
      transformation, protocol reduction, or anonymity filtering. Except
      where either transparent or non-transparent behavior is explicitly
      stated, the HTTP proxy requirements apply to both types of
      proxies.

   gateway
      A server which acts as an intermediary for some other server.
      Unlike a proxy, a gateway receives requests as if it were the
      origin server for the requested resource; the requesting client
      may not be aware that it is communicating with a gateway.

   tunnel
      An intermediary program which is acting as a blind relay between
      two connections. Once active, a tunnel is not considered a party
      to the HTTP communication, though the tunnel may have been
      initiated by an HTTP request. The tunnel ceases to exist when both
      ends of the relayed connections are closed.

   upstream/downstream
      Upstream and downstream describe the flow of a message: all
      messages flow from upstream to downstream.

   inbound/outbound
      Inbound and outbound refer to the request and response paths for
      messages: "inbound" means "travelling toward the origin server",
      and "outbound" means "travelling toward the user agent"

1.4 Overall Operation

   The BWTP protocol is a HTTP upgrade protocol. A clients a HTTP
   upgrade request to the server in order to establish a BWTP connection.
   After successful upgrade, the BWTP connection is a bidirectional 
   stream of MIME-like messages. There is no relationship between 
   inbound and outbound messages.

   The upgrade request and response may define meta data defaults
   for all messages on a BWTP connection, such as content-type,
   character-set, content-encoding, Location and Origin. 
   Individual messages my override any or all of these
   defaults as well as provide additional meta-data that does not
   have defaults.

   BWTP communication is initiated by a user agent and consists of
   an upgrade request to be applied to a resource on some origin server. 
   In the simplest case, this may be accomplished via a single connection (v)
   between the user agent (UA) and the origin server (O).

       UA -------------------v------------------- O
          upgrade request ----------------------»
          «----------------- 101 upgrade response
          inbound messages ---------------------»
          «-------------------- outbound messages

   Inbound and outbound messages may be sent simultaneously.

   A more complicated situation occurs when one or more intermediaries
   are present in the request/response chain. There are three common
   forms of intermediary: proxy, gateway, and tunnel. A proxy is a
   forwarding agent, receiving a message for a URI in its absolute form,
   rewriting all or part of the message, and forwarding the reformatted
   request toward the server identified by the URI. A gateway is a
   receiving agent, acting as a layer above some other server(s) and, if
   necessary, translating the requests to the underlying server's
   protocol. A tunnel acts as a relay point between two connections
   without changing the messages; tunnels are used when the
   communication needs to pass through an intermediary (such as a
   firewall) even when the intermediary cannot understand the contents
   of the messages.

       UA -----v----- A -----v----- B -----v----- C -----v----- O
          upgrade request ------------------------------------»
          «------------------------------- 101 upgrade response
          inbound messages -----------------------------------»
          «---------------------------------- outbound messages

   The figure above shows three intermediaries (A, B, and C) between the
   user agent and origin server.  All intermediaries must accept 
   the initial upgrade request and pass it onwards towards the 
   origin server. If any intermediary does not understand the 
   upgrade request, the BWTP connection will not be established.
   
   A message that travels the whole chain will pass through four 
   separate connections.

   This distinction is important because some  BWTP communication 
   options may apply only to the connection with the nearest, non-tunnel
   neighbour, only to the end-points of the chain, or to all connections
   along the chain. 
   
   Although the diagram is linear, each participant may
   be engaged in multiple, simultaneous communications. For example, B
   may be receiving requests from many clients other than A, and/or
   forwarding requests to servers other than C, at the same time that it
   is handling A's request.

   Any party to the communication which is not acting as a tunnel may
   inject messages into either the inbound or outbound stream.
   
   The following illustrates if B has injects both an inbound
   and an outbound message:

       UA -----v----- A -----v----- B -----v----- C -----v----- O
          upgrade request ------------------------------------»
          «------------------------------- 101 upgrade response
          inbound messages -----------------------------------»
          «---------------------------------- outbound message»
          «-------outbound message--|--inbound message -------»
          inbound messages -----------------------------------»
          «---------------------------------- outbound messages

   BWTP communication usually takes place over TCP/IP connections. The
   default port is TCP 80 [19], but other ports can be used. This does
   not preclude BWTP from being implemented on top of any other protocol
   on the Internet, or on other networks. BWTP only presumes a HTTP 
   connection can be established over a reliable transport;
   any protocol that provides such guarantees can be used.

2 Notational Conventions and Generic Grammar

2.1 Augmented BNF

   All of the mechanisms specified in this document are described in
   both prose and an augmented Backus-Naur Form (BNF) as defined by
   section 2.1 of RFC 2616.
   
   Implementers will need to be familiar with the
   notation in order to understand this specification.

2.2 Basic Rules

   The Basic Rules as defined by section 2.2 of RFC 2616 are used
   throughout this specification to describe basic parsing constructs.
   The US-ASCII coded character set is defined by ANSI X3.4-1986 [21].

3 Protocol Parameters

3.1 BWTP Version

   BWTP uses a "«major».«minor»" numbering scheme to indicate versions
   of the protocol. The protocol versioning policy is intended to allow
   the sender to indicate the format of a message and its capacity for
   understanding further BWTP communication, rather than the features
   obtained via that communication. No change is made to the version
   number for the addition of message components which do not affect
   communication behavior or which only add to extensible field values.
   The «minor» number is incremented when the changes made to the
   protocol add features which do not change the general message parsing
   algorithm, but which may add to the message semantics and imply
   additional capabilities of the sender. The «major» number is
   incremented when the format of a message within the protocol is
   changed. See RFC 2145 [36] for a fuller explanation.

   All messages on a BWTP connection will be of the same version
   that is negotiated with the upgrade request and response that
   establishes the BWTP connection:

       BWTP-Version   = "BWTP" "/" 1*DIGIT "." 1*DIGIT

   Note that the major and minor numbers MUST be treated as separate
   integers and that each MAY be incremented higher than a single digit.
   Thus, BWTP/2.4 is a lower version than BWTP/2.13, which in turn is
   lower than BWTP/12.3. Leading zeros MUST be ignored by recipients and
   MUST NOT be sent.

   The BWTP version of a connection is the highest BWTP version for
   which all intermediaries and the agent and origin server can handle.

3.2 Uniform Resource Identifiers

   URIs as defined by 3.2 of RFC2616 are used to identify BWTP
   resources.

3.4 Character Sets

   BWTP uses the same definition of the term "character set" as that
   described in section 3.4 of RFC2616.

3.4.1 Missing Charset

   If not otherwise specified, a character set of UTF-8 may be assumed.

3.5 Content Codings

   The content codings defined by section 3.5 of RFC2616 and 
   registered with the Internet Assigned Numbers Authority (IANA)
   MAY be applied to a BWTP message.

3.6 Transfer Codings

   HTTP-like transfer codings MUST NOT be applied to BWTP 
   connections or messages. Specifically the chunked transfer
   encoding is not supported by BWTP; Range limited messages
   should be sent in situations where chunked transfer is
   desirable.

3.7 Media Types

   HTTP uses Internet Media Types [17] in the Content-Type 
   header field in order to provide open and extensible data typing 
   and type negotiation. The media-type is defined in section
   3.7 of RFC2616 as:

       media-type     = type "/" subtype *( ";" parameter )
       type           = token
       subtype        = token

3.12 Range Units

   BWTP allows a message to carry only a part of an identified resource.
   BWTP uses range units in the Content-Range header field. A range-unit
   is defined in section 3.12 of RFC2616 as:

      range-unit       = bytes-unit | other-range-unit
      bytes-unit       = "bytes"
      other-range-unit = token

   An entity can be broken down into subranges according
   to various structural units.

   The only range unit defined by BWTP/1.0 is "bytes". BWTP/1.0
   implementations MAY ignore ranges specified using other units.
   BWTP/1.0 has been designed to allow implementations of applications
   that do not depend on knowledge of ranges.

4 BWTP Message

4.1 Message Types

   BWTP messages consist of inbound messages from client to server and 
   outbound message from the server to client:

       BWTP-message   = Inbound-message | Outbound-message    ; BWTP/1.0 messages

4.2 Generic Messages

   Inbound and Outbound messages use the generic message format of 
   RFC 822 [9] for transferring entities (the payload
   of the message). Both types of message consist of a start-line, zero
   or more header fields (also known as "headers"), an empty line (i.e.,
   a line with nothing preceding the CRLF) indicating the end of the
   header fields, and possibly a message-body.

        generic-message = start-line
                          *(message-header CRLF)
                          CRLF
                          [ message-body ]
                          [ *CRLF ]

        start-line      = "BWTP" SP message-URI SP content-length

        message-URI     = "*"               ; the connection default resource
                        | absoluteURI
                        | abs_path

        content-length  = 1*DIGIT

   In the interest of robustness, servers and clients SHOULD ignore any empty
   line(s) received where a start-line is expected. 

4.2 Message Headers

   BWTP header fields follow the same generic format as
   that given in Section 3.1 of RFC 822 [9]. Each header field consists
   of a name followed by a colon (":") and the field value. Field names
   are case-insensitive. The field value MAY be preceded by any amount
   of LWS, though a single SP is preferred. Header fields can be
   extended over multiple lines by preceding each extra line with at
   least one SP or HT. Applications ought to follow "common form", where
   one is known or indicated, when generating HTTP constructs, since
   there might exist some implementations that fail to accept anything
   beyond the common forms.

       message-header = field-name ":" [ field-value ]
       field-name     = token
       field-value    = *( field-content | LWS )
       field-content  = «the OCTETs making up the field-value
                        and consisting of either *TEXT or combinations
                        of token, separators, and quoted-string»

   The field-content does not include any leading or trailing LWS:
   linear white space occurring before the first non-whitespace
   character of the field-value or after the last non-whitespace
   character of the field-value. Such leading or trailing LWS MAY be
   removed without changing the semantics of the field value. Any LWS
   that occurs between field-content MAY be replaced with a single SP
   before interpreting the field value or forwarding the message
   downstream.

   The order in which header fields with differing field names are
   received is not significant. 

   Multiple message-header fields with the same field-name MAY be
   present in a message if and only if the entire field-value for that
   header field is defined as a comma-separated list [i.e., #(values)].
   It MUST be possible to combine the multiple header fields into one
   "field-name: field-value" pair, without changing the semantics of the
   message, by appending each subsequent field-value to the first, each
   separated by a comma. The order in which header fields with the same
   field-name are received is therefore significant to the
   interpretation of the combined field value, and thus a proxy MUST NOT
   change the order of these field values when a message is forwarded.

4.3 Message Body

   The presence of a message-body in a request is signalled by a non
   zero content-length in the message start-line. If a Content-Length
   field is included in the message-headers, it is ignored.

   The message body is defined as 

       message-body  = *OCTET

4.4 Message Length

   The message body length is determined by the content-length
   specified in the message start-line. 

4.5 General Header Fields

   There are a few header fields which have general applicability for
   both inbound and outbound messages, but which do not apply to the
   entity being transferred. These header fields apply only to the
   message being transmitted.

       general-header = Connection
                      | Date           
                      | Via              

   General-header field names can be extended reliably only in
   combination with a change in the protocol version. However, new or
   experimental header fields may be given the semantics of general
   header fields if all parties in the communication recognize them to
   be general-header fields. Unrecognized header fields are treated as
   entity-header fields.

5 Inbound Message

   An inbound message from a client to a server is a generic
   message with the restriction of the included headers to be 
   either general, inbound or entity headers:

     inbound-request   = "BWTP" SP message-URI SP content-length
                        *(( general-header | inbound-header | entity-header ) CRLF) 
                        CRLF
                        [ message-body ]        
                        [ CRLF ]

5.1 Inbound Message URI

   The inbound message-URI is a Uniform Resource Identifier and
   identifies the resource on the origin server to which the
   message is destined.   A message-URI of "*" means that the
   default URI of the connection.

   If the abs_path form is used for the inbound message URI, then
   the host of the resource is defined either by a Host header
   in the message or a Host header in the original upgrade request.
   
   The inbound message URI is not used by proxies to route messages.
   A message, even with an absolute URI or Host header that specifies
   a different origin server will be delivered to the origin server 
   identified in the original upgrade request.

   If the message-URI is encoded using the "% HEX HEX" encoding
   [42], the origin server MUST decode the Request-URI in order to
   properly interpret the request. 

5.3 Inbound Header Fields

   The inbound-header fields allow the client to pass additional
   information about the message, and about the client itself, to the
   server. These fields act as message modifiers, with semantics
   equivalent to the parameters on a programming language method
   invocation.

      inbound-header = Authorization         
                      | Host                  
                      | Origin
                      | Proxy-Authorization  
                      | User-Agent
                      | Cookie           ; TODO Maybe not?        

   inbound-header field names can be extended reliably only in
   combination with a change in the protocol version. However, new or
   experimental header fields MAY be given the semantics of request-
   header fields if all parties in the communication recognize them to
   be request-header fields. Unrecognized header fields are treated as
   entity-header fields.

5.4 Inbound Message Examples

   A minimal inbound message with default destination and
   content-type is

       BWTP * 12

       Hello World!

   An inbound message with default destination and content-type 
   and a content range

       BWTP * 5
       Content-Range: bytes 1-5/12

       Hello

   An inbound message that explicitly identifies a destination,
   content type and content encoding is

       BWTP /destination/path 12
       Content-Type: text/plain;charset=utf-8

       Hello World!

   An empty message with a Connection header:

       BWTP * 0
       Connection: close

6 Outbound Message

   An outbound message from a server to a client is a generic
   message with the restriction of the included headers to be 
   either general, outbound or entity headers:

     inbound-request   = "BWTP" SP message-URI SP content-length
                        *(( general-header | outbound-header | entity-header ) CRLF) 
                        CRLF
                        [ message-body ]        
                        [ CRLF ]

6.1 Outbound Message URI

   The outbound message-URI is a Uniform Resource Identifier and
   identifies the resource on the origin server from which the
   message has been sent.   A message-URI of "*" means that the
   default URI of the connection applies.

   If the abs_path form is used for the inbound message URI, then
   the host of the resource is defined either by a Host header
   in the message or a Host header in the original upgrade request.
   
   The outbound message URI is not used by proxies to route messages.
   All outbound messages on a connection will be delivered to the 
   user agent that initialized the connection.

   If the message-URI is encoded using the "% HEX HEX" encoding
   [42], the user agent  MUST decode the message-URI in order to
   properly interpret the message. 

6.2 Outbound Header Fields

   The outboudn-header fields allow the server to pass additional
   information about the message,
   These header fields give information about the server and about
   further access to the resource identified by the Message-URI.

       outbound-header = Location              
                       | Server                 
                       | WWW-Authenticate    
                       | Set-Cookie
                       | Content-Origin  TODO - the origin stuff needs more thought  

   Outbound header field names can be extended reliably only in
   combination with a change in the protocol version. However, new or
   experimental header fields MAY be given the semantics of response-
   header fields if all parties in the communication recognize them to
   be response-header fields. Unrecognized header fields are treated as
   entity-header fields.

6.3 Outbound Message Examples

   A minimal outbound message with default source and
   content-type is

       BWTP * 12

       Hello World!

   An outbound message with default destination and content-type 
   and a content range

       BWTP * 5
       Content-Range: bytes 1-5/12

       Hello

   An outbound message that explicitly identifies a source,
   content type, content encoding and Origin is

       BWTP /destination/path 12
       Content-Type: text/plain;charset=utf-8
       Origin: www.acme.com

       Hello World!

   An empty message with a Connection header:

       BWTP * 0
       Connection: close

7 Entity

   In this section, both sender and recipient refer to either the client
   or the server, depending on who sends and who receives the message.

7.1 Entity Header Fields

   Entity-header fields define metainformation about the entity-body or,
   if no body is present, about the resource identified by the request.
   Some of this metainformation is OPTIONAL; some might be REQUIRED by
   portions of this specification.

       entity-header  = Content-Encoding        
                      | Content-Language       
                      | Content-Location     
                      | Content-MD5         
                      | Content-Range      
                      | Content-Type      
                      | Expires          
                      | Last-Modified    
                      | extension-header

       extension-header = message-header

   The extension-header mechanism allows additional entity-header fields
   to be defined without changing the protocol, but these fields cannot
   be assumed to be recognizable by the recipient. Unrecognized header
   fields SHOULD be ignored by the recipient and MUST be forwarded by
   transparent proxies.

8 Connections

8.1 Negotiation
  
8.1.1 Upgrade Request
   A BWTP connection is initiated by a client issuing an upgrade request,
   which is a valid HTTP/1.1 GET request with an Upgrade header field set
   to the desired BWTP-Version:

     BWTP-Upgrade-Request = "GET" SP URI SP "HTTP/1.1" CRLF
                            *(message-header CRLF)
                            "Upgrade" ":" BWTP-Version
                            *(message-header CRLF) 
                            CRLF
                            [ message-body ]

   Since Upgrade is only for the immediate connection, the Upgrade Request SHOULD
   also contain a Connection header with "Upgrade" in it's values.

   The default values for all inbound messages on the resulting BWTP connection 
   are taken from:
     * The default message-URI is the URI of the upgrade request.
     * The default host is the host given either in an absolute URI or in a
       Host header.
     * Any header field with a name starting with "BWTP-" is used as 
       a default field of all inbound BWTP messages, with the leading 
       "BWTP-" stripped from the name.

   An example Upgrade Request example

     GET /path HTTP/1.1
     Host: host.com:80
     Upgrade: BWTP/1.0
     Connection: Upgrade
     Origin: the.origin.com
     Authorization: BASIC amd73ksosh3hj210sd3d0dk1
     BWTP-Content-Type: text/plain;charset=utf-8

   This request, if accepted, has the following inbound messages defaults:
     message-URI    /path
     host           host.com
     port           80 
     Content-Type   text/plain;charset=utf-8

   If an intermediary in the HTTP connection accept the upgrade, they MUST 
   forward the Upgrade header in their requests towards the origin server.

8.1.2 Upgrade Response

   When an origin server receives an Upgrade request and the resource specified
   by the URI accepts the upgrade, then acceptance is indicated by sending
   an Upgrade Response:

     BWTP-Upgrade-Response = "HTTP/1.1" SP 101 SP "Bidirectional Web Transfer Protocol" CRLF
                            *(message-header CRLF)
                            "Upgrade" ":" BWTP-Version
                            *(message-header CRLF) 
                            CRLF

   The default values for all outbound messages on the resulting BWTP connection 
   are taken from:
     * The default message-URI is the URI of the upgrade request.
     * The default host is the host given either in an absolute URI or in a
       Host header of the upgrade request.
     * Any header field in the response with a name starting with "BWTP-" is used as 
       a default field of all outbound BWTP messages, with the leading "BWTP-" stripped
       from the name.
   
   An example Upgrade Response example is

     HTTP/1.1 101 Bidirectional Web Transfer Protocol
     BWTP-Content-Origin: the.origin.com
     BWTP-Content-Type: text/plain;charset=utf-8

   This request, if sent in response to the prior Upgrade Request example, has the 
   following outbound messages defaults:
     message-URI    /path
     host           host.com
     port           80 
     Content-Type   text/plain;charset=utf-8
     Content-Origin the.origin.com

8.2 Flow Control

   BWTP/1.1 servers SHOULD maintain connections and use TCP's
   flow control mechanisms to resolve temporary overloads, rather than
   terminating connections with the expectation that clients will retry.
   The latter technique can exacerbate network congestion.

8.3 Closing Connections

8.3.1 Spontaneous Connection close

   Either the client or server or any intermediary MAY at any time 
   close the BWTP. Message delivery is not guaranteed if a spontaneous
   close is performed.  Clients, servers and intermediaries SHOULD 
   use orderly connection close if guaranteed delivery is required.
   
8.3.2 Orderly Connection close
   
   Either the client or server or any intermediary MAY initiate an
   orderly shutdown of a BWTP connection by sending a message with 
   a "Connection" header field with a value of "Close". After sending 
   such a message, messages MUST NOT be sent on the connection,
   but messages SHOULD continue to be received and processed until  
   amessage is received that also contains a Connection Close header.
   The connection may then be closed.

   A client or server that is not closing the connection, on receipt of 
   any message that contains a Connection Close header should immediately 
   flush any pending or queued messages for the connection and the last
   message sent should have a Connection close header. After sending
   the last message, the connection may be closed.

   The connection close header may be sent on any valid BWTP message.
   If no message is queued or available to carry a close header, then
   an empty BWTP message may be sent:

       BWTP * 0
       Connection: close

11 Access Authentication

   HTTP provides several OPTIONAL challenge-response authentication
   mechanisms which can be used by a server to challenge a client
   request and by a client to provide authentication information. The
   general framework for access authentication, and the specification of
   "basic" and "digest" authentication, are specified in "HTTP
   Authentication: Basic and Digest Access Authentication" [43]. This
   specification adopts the definitions of "challenge" and "credentials"
   from that specification.

14 Header Field Definitions

15 Security Considerations

16 Acknowledgments

17 References

18 Authors' Addresses

19 Appendices

20 Index

   Please see the PostScript version of this RFC for the INDEX.

21.  Full Copyright Statement

   Copyright (C) The Internet Society (1999).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 10 Jun 2009 19:20:48 +0200

> document does that sufficiently well.

I'd be interested in reviewing and eventually implementing this, but
it would be helpful if you could explain BWTP in terms of the
differences between it and RFC2616.  As presented, I need to do a
manual diff to see what the significant changes are.

Cheers,
Pieter Hintjens

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 11 Jun 2009 08:24:30 +1000

Pieter,

I've added you as a collaborator on the google doc.
I've also added an appendix on the differences with HTTP/1.1, which
for the benefits of the list is

Appendix A. Key differences with HTTP/1.1

   The BWTP has been substantially based on HTTP/1.1 and the RFC2616 document
   that specifies it.  The key differences with HTTP/1.1 are:
     * A BWTP connection cannot be directly established by a client, but
       must be upgraded from a HTTP connection.
     * There is no relationship (like request/response) between inbound
       and outbound messages
     * There are no methods or statuses associated with messages.
     * Messages may either be completely self describing or rely on meta
       data associated with the connection that is established during upgrade.
     * A message content length is included in the message start line
     * An orderly close sequence is specified
     * There is no caching mechanism

With regards to work needed on the document - the security mechanism
needs to be reviewed and an Origin and acceptable-Origin mechanism
defined.

Also the document needs section 14 to be completed with Header Field
Definitions.  These will mostly be from RFC2616, but some will not
be applicable, some will need to be modified and there should be
a few new ones.

Oh and the document needs to be converted to something that does
not have hard coded formatting.  What is commonly used for
this? nroff? docbook?

cheers

---

From: Pieter Hintjens <[email-masked]>
Date: Thu, 11 Jun 2009 09:41:39 +0200

> Appendix A. Key differences with HTTP/1.1

Fair enough.  Would it be correct to describe this as an asynchronous
message framing protocol designed to be layered on top of HTTP?  The
reason for using this protocol is then to implement asynchronous
client-to-server or server-to-client messaging with minimal overhead,
using an established HTTP connection as the basis.

Thus, the complex one-off work (like authentication) is done using
synchronous, pedantic HTTP while the high volume work (message
transfer) is done using fast, asynchronous BWTP?  Presumably after
switching to BWTP, one does not return to HTTP.

This is a good pattern IMO.

> not have hard coded formatting.  What is commonly used for

My proposal would be to use a wiki, where the text can be edited
collaboratively, and where we can track issues and todos.  I usually
use wikidot.com for such things (e.g. www.restms.org).  Glad to set
that up if you need it.

It's pretty simple to convert from any random wiki text to an IETF
standard layout when it's time to do that.  Hard formatting before
that point is just a PITA.

Cheers,
Pieter

---

From: Kris Zyp <[email-masked]>
Date: Thu, 11 Jun 2009 08:25:27 -0600

Greg Wilkins wrote:
>
> + Because of it's similarity to HTTP, it is intended that existing
> HTTP clients, servers and intermediaries will be able to be
> minimally updated to support this protocol. This will not require
> entirely new protocol libraries to be developed.   Existing
> development and analysis tools will also be able to be easily
> updated, plus the protocol is mostly human readable.
Maybe I missing something about this in discussions, but I thought one
of the core ideas of WebSockets was to use CONNECT to
tunnel through existing proxies. Is there hope that BWTP would somehow
be properly handled by existing intermediaries, or would it require an
update to proxies? It certainly seems that if it is possible to build
the protocol such that it will work over existing proxies, that would
be greatly preferable. If this won't work across in current proxied
environments, uptake will be much slower (if at all), won't it?

- --
Kris Zyp
SitePen
(503) 806-1841
http://sitepen.com

---

From: Martin Tyler <[email-masked]>
Date: Thu, 11 Jun 2009 15:37:57 +0100

Hi Kris,

That is the intention, I mentioned it to Greg and he's added a TODO to the
document.

I think Greg might have confused a couple of people with his first email on
this.. it basically an alternative to the existing WebSocket proposal, works
the same way fundamentally, but in a way that fits better in style with HTTP
both in the protocol itself, and how it is documented.

Regards
Martin

On Thu, Jun 11, 2009 at 3:25 PM, Kris Zyp <[email-masked]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Greg Wilkins wrote:
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkoxE9YACgkQ9VpNnHc4zAwxQACgrEphCoLbrdic89wlmXmVOidh
> P14AoKipXqcTqol4KOMYqYgU+ECNf43c
> =SN7W
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Jun 2009 00:38:44 +1000

Kris,

The intent of BWTP is that eventually intermediaries would be
updated to understand it and even participate in the connection (for
example initiating an orderly close).

However, if the proxy does not support the upgrade request,
then BWTP would also able to be sent over a tunnel established
by a CONNECT.

Note that this is the same for the websocket when it tries to
do an upgrade but encounters a transparent proxy that does not
support the upgrade.

However, because BWTP is so similar to HTTP, the work needed
to upgrade a proxy should be minimal.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Jun 2009 01:02:46 +1000

Pieter Hintjens wrote:
> message framing protocol designed to be layered on top of HTTP?  

Not layered "on top" of HTTP, but dependent on HTTP to open a
connection, but that connection would then be upgraded to BWTP
and HTTP would no longer be used.

> The

Yes

> switching to BWTP, one does not return to HTTP.

I've not planned to support a downgrade to HTTP.

> This is a good pattern IMO.

cool!

> use wikidot.com for such things (e.g. www.restms.org).  Glad to set

a wiki would be better than google docs, because it would have
history.

I've created a site at  http://bwtp.wikidot.com
You can join at http://bwtp.wikidot.com/system:join
and the password is MustHaveMeta

I've not yet moved the document over.

> that point is just a PITA.

+1

cheers

---

From: Pieter Hintjens <[email-masked]>
Date: Thu, 11 Jun 2009 18:26:43 +0200

> I've not planned to support a downgrade to HTTP.

That's fine, the pattern of synchronous control followed by
asynchronous data is simple, and matches the real world use cases I've
seen.  This is naturally handled by two protocols, and HTTP -> BWTP
feels exactly right.

Presumably we could do a kind of BWTP emulation over HTTP, for
prototyping or proof of concept.

FWIW, I've been trying to get this pattern into AMQP (www.amqp.org)
for about two years, without success.

> and the password is MustHaveMeta

You might want to add some legal framing so that contributions are
properly reusable.  Digistan.org has a template (spec.digistan.org),
which we used for restms.org.  It just avoids the problem that a
contributor 'accidentally' adds something they then patent.

There is also a legal package at packages.wikidot.com, which may be sufficient.

Cheers,
Pieter

---

From: Paul Prescod <[email-masked]>
Date: Thu, 11 Jun 2009 12:14:11 -0700

Could you justify the "non-obvious" changes from RFC2616?

>       must be upgraded from a HTTP connection.

Okay.

>     * There is no relationship (like request/response) between inbound

Is this "simplification" worth the cost of being different from RFC2616? As
we all know, one can implement fire-and-forget on top of request/response or
vice versa. One could imagine a trivial mod_async which merely returns 200
OK to every message before queuing it for internal processing. On the client
side, async HTTP already exists.

I also wonder the simplification of the protocol is worth the cost of being
different for all of the following:

>     * There is no caching mechanism

HTTP's caching mechanism is already optional, right?

One virtue of staying closer to HTTP is that one can easily imagine how
existing AJAX APIs and code could be adjusted to use the BWTP socket easily.
For that reason, I am leaning towards the belief that BWTP should try to be
a superset of unidirectional HTTP.

 Paul Prescod

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Jun 2009 07:47:50 +1000

Paul,

with regards as keeping the messages identical to RFC2616, I did consider
this because it would be cool to just work through existing intermediaries.

But the fundamental problem is that because it is request/response
the number of responses must equal the number of requests.  So if you
have traffic that is substantially one way, you end up sending lots
of empty messages the other direction to keep things balanced.  This
degenerates to more-or-less long polling - which we can do today.

I guess there could be an incremental improvement in HTTP long
polling by sending a full pipeline of requests and requiring the
server to examine that incoming pipeline for messages, but there
would be queueing delays and not all intermediaries can handle
pipelines anyway.

Also the websocket guys appear very keen to byte squeeze the packets,
so changing the start-line to include content length was a good
way to achieve this.

Finally, for the sake of debugging, it would be good if a
BWTP protocol packet can be identified as such by inspection without
state.

So that's why I've gone down the path that I have.

But if you think that a pure RFC2616 like approach is worthwhile,
then I'd certainly review and contribute to an alternate alternate
proposal.

cheers

Paul Prescod wrote:
> Is this "simplification" worth the cost of being different from RFC2616?
> As we all know, one can implement fire-and-forget on top of
> request/response or vice versa. One could imagine a trivial mod_async
> which merely returns 200 OK to every message before queuing it for
> internal processing. On the client side, async HTTP already exists.
>  
> I also wonder the simplification of the protocol is worth the cost of
> being different for all of the following:
> 
> existing AJAX APIs and code could be adjusted to use the BWTP socket
> easily. For that reason, I am leaning towards the belief that BWTP
> should try to be a superset of unidirectional HTTP.
> 
>  Paul Prescod
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Jun 2009 15:16:30 +1000

All,

I've put the latest BWTP proposal up on http://bwtp.wikidot.com/

cheers

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 16 Jun 2009 14:58:06 +0300

Hi Greg,

I would say that the main difference with the WebSocket protocol is that 
BWTP includes META-DATA
that make BWTP working in a similar way to HTTP.
It would be interesting understand why WebSocket does not have defined 
'meta';
if there is any specific reason why there aren't.
Any idea?
is there any reason from a browser prospective to avoid the inclusion of 
META-DATA in this kind of protocol?

maybe that a question for Ian!

one question more relate to the draft. In Section 1.4 there is the 
following statement:
"/The upgrade request and response may define meta data defaults for all 
messages on a BWTP connection, such as content-type, character-set, 
content-encoding, Location and Origin. Individual messages my override 
any or all of these defaults as well as provide additional meta-data 
that does not have defaults./"
it is not clear if it would be possible only override the defaults with 
ad-hoc per message, or it would also possible send messages without any 
META-DATA.
In the latter case then the BWTP could also work in a WebSocket mode.

> example initiating an orderly close).
the fact that an intermediaries would understand the protocol and even 
participate in the connection
is an interesting feature, that however needs to be investigate 
especially the security considerations and described more in details .

Furthermore  I have  some doubts  on the possibility  that "/any 
intermediary MAY initiate an orderly shutdown of a BWTP connection by 
sending a message with a "Connection" header field with a value of 
"Close"/".
> do an upgrade but encounters a transparent proxy that does not
It would be also interesting discuss in the draft the CONNECT scenario, 
and explicitly say that in this case
the proxy that does not support upgrade can not participate in the 
connection.

btw, the response to a GET with an Upgrade header, should contain an 
Upgrade header as well.
you should correct the example in Section 8.1.2 in the following way:

|HTTP/1.1 101 Bidirectional Web Transfer Protocol
||Upgrade: BWTP/1.0|
|BWTP-Content-Origin: the.origin.com
BWTP-Content-Type: text/plain;charset=utf-8|||

> to upgrade a proxy should be minimal.
right the work to upgrade a proxy would be minimal, moreover having the 
proxy recognizing
the protocol would also help the bidirectional communication coming 
through it.

cheers
Sal

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 16 Jun 2009 23:28:31 +1000

> that make BWTP working in a similar way to HTTP.

I would agree that this is indeed the key substantive difference between
the proposals, plus the elements that flow from the ability to have
standard headers (eg support for ranges, content encodings etc.).

Another difference is the recognition of intermediaries as potential
active participants in a connection.

There are also significant differences in the "style" of the
protocol which can have an impact of the protocols acceptance
and ease of implementation.

> if there is any specific reason why there aren't.

I did post a proposal to define a websocket frame type that would
carry mime encoded content.   However, I don't think the websocket
authors accept meta-data as a requirement and appear to advocate
that meta-data could be transported within the payload by mutual
assumption between client and server.

> META-DATA in this kind of protocol?

One of the objections of meta data that have I tried to address
in BWTP was that it can often be verbose and redundant.

For that reason, I have proposed move the Content-Length
to the start-line and have a default mechanism so that common
meta-data need only be sent once per connection.

I'm not sure if there are other objections - Ian?

> In the latter case then the BWTP could also work in a WebSocket mode.

As written, the only way to override default meta-data is to supply
alternative meta-data (with potentially a null value).
Note that it is always possible to change the defaults for the connection
by sending new defaults with any message (I'm not sure this is clear in
the draft - but I will make it so).

Note that I would say that WebSocket also has meta-data.  Just that
it is fixed and cannot be overridden.  For example the plain
text UTF-8 datagram is the meta-data default for the sentinel
framing of websocket.

cheers

---

From: Ian Hickson <[email-masked]>
Date: Tue, 16 Jun 2009 17:17:41 +0000 (UTC)

On Tue, 16 Jun 2009, Salvatore Loreto wrote:
> 'meta'; if there is any specific reason why there aren't.

The WebSocket protocol is designed on the principle that there should be 
minimal framing (the only framing that exists is to make the protocol 
frame-based instead of stream-based, and to support a distinction between 
Unicode text and binary frames). It is expected that metadata would be 
layered on top of WebSocket by the application layer, in the same way that 
metadata is layered on top of TCP/IP by the application layer (HTTP).

Conceptually, WebSocket is really just a layer on top of TCP/IP that adds 
a Web "origin"-based security model for browsers; adds an addressing and 
protocol naming mechanism to support multiple services on one port and 
multiple host names on one IP address; and layers a framing mechanism on 
top of TCP to get back to the IP packet mechanism that TCP is built on, 
but without length limits. Other than that, it adds nothing. Basically it 
is intended to be as close as possible to just exposing raw TCP/IP to 
script as possible given the constraints of the Web. (It's also designed 
in such a way that its servers can share a port with HTTP servers, by 
having its handshake be a valid HTTP Upgrade handshake also.)

HTH,
-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Ian Hickson <[email-masked]>
Date: Tue, 16 Jun 2009 17:20:59 +0000 (UTC)

On Tue, 16 Jun 2009, Greg Wilkins wrote:
> 
> > is there any reason from a browser prospective to avoid the inclusion 
> > of META-DATA in this kind of protocol?
> 
> [...]
> 

It would be best to ask browser vendors directly, but off-hand I can't 
think of any reasons other than the necessary added complexity (parsing 
the metadata, handling exposing it to scripts, defining all the error 
handling to ensure bogus servers are handled predictably, etc).

> Note that I would say that WebSocket also has meta-data.  Just that it 
> is fixed and cannot be overridden.  For example the plain text UTF-8 
> datagram is the meta-data default for the sentinel framing of websocket.

Insofar as framing is metadata, yes.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 17 Jun 2009 09:07:37 +1000

> handling to ensure bogus servers are handled predictably, etc).

Note that I think the discussion of what should be exposed/changed in
the API is a different one to the transport protocol.

I think changing the API is something that can happen more easily over
time than changing network infrastructure.

I think trying to future proof and API for unknown future use-cases
mostly leads to strange APIs, while future proofing transport protocols
to future use-cases is good practise.

For example, already the websocket protocol proposal has more
capabilities than the API supports.  With BWTP I'm proposing that
this additional capability be taken to be equally capable as HTTP.

> Insofar as framing is metadata, yes.

I was really refering to an implied text/plain;charset=UTF-8 content type
with no content-encoding.

cheers

---

From: Ian Hickson <[email-masked]>
Date: Wed, 17 Jun 2009 00:13:43 +0000 (UTC)

On Wed, 17 Jun 2009, Greg Wilkins wrote:
> >> it is fixed and cannot be overridden.  For example the plain text 
> >> UTF-8 datagram is the meta-data default for the sentinel framing of 
> >> websocket.
> > 
> I was really refering to an implied text/plain;charset=UTF-8 content 
> type with no content-encoding.

I don't really think it's correct to think of the WebSocket system as 
implying such a type; you could just as easily be sending something else 
like JSON, or XML, or whatever. It's just that the basic data structure is 
Unicode instead of octets.

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 17 Jun 2009 02:24:53 +0100

> message framing protocol designed to be layered on top of HTTP?

No, because it is incompatible with HTTP.

It will not work over existing proxies, it will not work over port 80
on some parts of the existing internet, and it will not work through a
general purpose HTTP server which passes things to per-URL handlers,
like most HTTP servers do.

But those are all solvable problems :-)

> This is a good pattern IMO.

I agree, except by dropping HTTP entirely after the negotiation, it
does implicitly suggests you need multiple connections if the complex
things, like authentication, need to be different for separate
back-end processes.

That's probably not a common scenario, and most protocols work like
that, so that's fine.  (But as an observation, when it occurs it'll be
less network-efficient than pipelined HTTP.)

Unfortunately, that prevents client implementations from automatically
sharing connections between multiple API instances.  That seems to
inherit an awkward (mis-)feature of the WebSocket API: multiple
applications running side by side can either not share connections at
all, or do so by using a different API entirely ("workers"), a
situation which will discourage network efficiency.

> use wikidot.com for such things (e.g. www.restms.org).  Glad to set

Is there a system for annotating the document with comments?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 17 Jun 2009 02:30:29 +0100

> prototyping or proof of concept.

Far more useful: For actual deployability on the real port 80 internet.

You can't expect Upgrade to work on port 80 on a substantial portion
of the internet (possibly growing, due to 3G mobile phones), and a
good number of places don't allow access to general internet services
via any port other than 80.

BWTP or anything else "emulated" over HTTP is a way to sort that out.

Of course then it is just another protocol over Comet (etc.), with
sucky performance.

So something of great value would be a way to Upgrade to the proper
protocol over the real internet in a way works despite transparent
proxies on port 80, to the extent that it's possible, and work
properly when it is not possible to upgrade.

> for about two years, without success.

The "authenticate first then do fast messaging" pattern is extremely
common.

I'm not familiar with AMQP's details.  In what way does it not fit that
pattern?

Or do you mean emulation over HTTP?  I think I saw an implementation
of AMQP over HTTP advertised not long ago.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 17 Jun 2009 03:08:51 +0100

Greg Wilkins wrote:
> So if you have traffic that is substantially one way, you end up
> sending lots of empty messages the other direction to keep things
> balanced.

It's not immediately obvious, but BWTP will send empty messages in the
other direction anyway.

They are called TCP ACKs, and you'd be surprised how little difference
a few "200 OK" lines makes per ACK, if they are piggybacked properly.

(Of course they aren't always piggybacked properly.)

I do like the idea of responses being optional though.  It's just
nicer, when you are layering another protocol on top of the messaging
service anyway - and the other protocol probably has it's _own_ form
of replies.

In my own bidirectional-HTTP-like implementation, the server always
generates a response, but each request has a field indicating if the
response should be sent or not.  (The same field is used to coordinate
out-of-order responses in fact).

> This degenerates to more-or-less long polling - which we can do
> today.

Not really.  With long polling, you need two sockets: one to send
messages, and one to receive them on.  Bidirectional messaging is an
improvement already.

> would be queueing delays and not all intermediaries can handle

I'm curious about that.  What known issues are there with
intermediaries and pipelines?  I'm aware of lots of problems with
origin servers and pipelines, but not intermediaries.

I guess known issues should go in the Comet best practices document?

> Finally, for the sake of debugging, it would be good if a BWTP
> protocol packet can be identified as such by inspection without
> state.

That makes good sense, I like that.

> then I'd certainly review and contribute to an alternate alternate

There are several problems and limitations with HTTP and pipelined
HTTP that this has a chance to fix, not least what to do on errors,
and what to do when a connection has been idle for a while.

Here's a couple of RFC2616 issues which are, imho, best avoided when
reinventing it bidirectionally:

RFC2616 specifies that under certain conditions, a connection must be
terminated _abruptly_ when an error response is sent to certain
requests like POST, to stop a potentially large request from being
sent whole.  Any pipelined requests after it will be lost, but you
cannot know which ones the server received and started processing.

You don't really want that here: it's much better that there's a way
to halt one request without breaking the connection for other requests.

When sending a request to an RFC2616 idle connection (say after 30
seconds), you don't know if the server has decided to close it or not.
So on sending, you might get a connection error, and then you don't
know if your request was processed or not, and whether it's safe to
repeated it.  There are other reasons why you might get the same
connection error, such as network faults and NAT timeouts.

For that reason, even persistent HTTP cannot be used for many requests
safely and automatically, so clients don't.  E.g. for POST.  Let alone
pipelined HTTP, which has the same problem in more cases.

You have, thankfully, already addressed the orderly shutdown problem.
Real HTTP servers use a lingering half-closed socket, to ensure the
last response will actually reach the client if the client pipelined
another request.  This is necessary for reliable behaviour (due to a
TCP quirk), yet not mentioned in RFC2616.

All messages over BWTP would all fall into the category of not being
safe to repeat automatically by the BWTP implementation.  Now, because
BWTP is intended for "TCP-like" applications, they will already be
aware of the possibility of broken connections and such things, and
have their own method of recovering, so it's not such a big problem.

But that means you cannot simply layer standard _HTTP_ requests over
BWTP safely.  Only application-specific protocols which know what to
expect can use it, even though they have access to HTTP-style message
metadata.

It _is_ possible to layer standard HTTP over BWTP safely, and to avoid
the same problems in BWTP-aware applications (making the applications
simpler).  But not by taking RFC2616 and simply making it bidirectional.

> > should try to be a superset of unidirectional HTTP.

I like that idea a lot (I'll help make it possible), but I also like
the idea of having the option to transport simple non-HTTP messages too.

See above about why you have to be careful about layering HTTP
messages automatically over anything else that dabbles in persistence
and pipelining.  There are assumptions about what behaviour to expect,
and what requests are safe to repeat automatically in the protocol
implementation, and when to do it.  AJAX apps expect to call
XmlHttpRequest and not have to do things like recover from
broken/timed-out connections - they expect the protocol implementation
to sort it out, but in a safe way.

-- Jamie

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 17 Jun 2009 10:35:57 +0300

> metadata is layered on top of TCP/IP by the application layer (HTTP).
I don't see any minimal framing requirement from the requirements list 
you sent on this list a couple
of months ago:
http://www.ietf.org/mail-archive/web/hybi/current/msg00007.html

actually, in theory also BWTP meets all the requirements you have listed 
in that mail.
> a Web "origin"-based security model for browsers; 
the problem here is that WebSocket are not built directly on top of TCP/IP,
but it tries to reuse the HTTP infrastructure to in somehow perform a 
downgrade from the HTTP protocol to
the TCP protocol and then start to build on top of it.
So IMHO the protocol should try to interwork in a fair way with the 
existent HTTP infrastructure,
asking for as minimal as possible update to the elements present in that 
infrastructure.

regards
Sal

> adds an addressing and

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 17 Jun 2009 10:44:08 +0300

> the API is a different one to the transport protocol.
I tend to agree with Greg, on this point.
Here we should only focus on the transport protocol, and try to design a 
good protocol that will meet the requirement
"/to provide two-way communication from a Web page in a clean way/"
where the clean way, I think should be read as in a way that does not 
destroy and it is compatible with the HTTP infrastructure.

regards
Sal

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 17 Jun 2009 17:50:27 +1000

> (Of course they aren't always piggybacked properly.)

Wow - it's like listening to myself when I defend long-polling vs streaming :)

> origin servers and pipelines, but not intermediaries.

A good example is apache modproxy, which is request based
and will not forward another request until it has a response
from the last forwarded.

> cannot know which ones the server received and started processing.

I'd really like to have graceful shutdown supported, so that any
party (including intermediaries) can close a connection without
any message loss.   Of course this does not stop unexpected
closures.

> TCP quirk), yet not mentioned in RFC2616.

I still think BWTP needs a little bit more, so that an intermediary
can initiate a graceful shutdown.  It's not enough for it to just
inject a Connection:close message, as a message may arrive from the
other direction.   So I think there needs to be a Connection:closing
message.

> things, like authentication, need to be different for separate

BWTP gives the choice of per connection default or per message headers.

It would be feasible to have each message carrying an authentication
header to be differently authenticated from other messages.

But generally more thought is needed on the whole security/Origin issue.

cheers

---

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 17 Jun 2009 09:52:32 +0200

> the API is a different one to the transport protocol.

The transport protocol still needs to be implemented so the added complexity of BWTP over WS does not seem that great in that respect.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Jonas Sicking <[email-masked]>
Date: Wed, 17 Jun 2009 01:00:48 -0700

On Wed, Jun 17, 2009 at 12:35 AM, Salvatore
Loreto<[email-masked]> wrote:
> So IMHO the protocol should try to interwork in a fair way with the existent
> HTTP infrastructure,

What is your goal of wanting to "interwork in a fair way with existent
HTTP infrastructure"? It seems very unlikely that whatever protocol
we're describing here will ever be compatible enough with HTTP that
you can reuse HTTP products such as apache or proxies directly.

Is your goal to be able to reuse the code from existing products in a
websocket/BWTP/whathaveyou server? Or to make it easier for people
familiar with HTTP to write a server and/or proxy for the new
protocol?

The fact that WebSocket downgrades from HTTP does not seem to me to be
because we think HTTP fulfilling particular use cases, but because we
want WebSocket to be able to connect through existing proxies. So
compatibility with HTTP in general doesn't seem obviously beneficial
to me.

/ Jonas

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 17 Jun 2009 11:24:58 +0300

> you can reuse HTTP products such as apache or proxies directly.
I am perfectly aware of this fact, I mean that whatever will be the 
protocol we won't be able to
reuse HTTP product as they are right now.

The point is, if you want WebSocket of whatever protocol to be able to 
connect through
existing proxies you have to be sure that the traffic generate works in 
a fair way with the HTTP traffic.

Moreover it should be useful start to discuss more about if
- the WebSocket protocol or whatever protocol works with the actual HTTP 
infostructures, or
  if there are problems that need to be investigate and eventually 
solved: e.g. what if it is present
  a transparent proxies ?
- would it be a problem for the proxies as they are right now, if they 
start to receive an huge number
  of request to connect through them?
etc...

regards
Sal

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 17 Jun 2009 12:03:54 +0200

> Is there a system for annotating the document with comments?

Good idea.  I've added comments on the main page, and a Talk page for
the proposal.  You can 'watch' the site, or proposal page and
automatically get emails when things change.

Putting comments on the proposal page itself is easy to do but it
rapidly gets confusing, as the comments become out of date.  Thus the
Talk page.

This is easy to change and improve, so feel free to suggest changes
and/or join the site and change it.

-Pieter

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 17 Jun 2009 12:20:15 +0200

> properly when it is not possible to upgrade.

Sounds right.

> I'm not familiar with AMQP's details.  In what way does it not fit that

Not just authentication, in AMQP's case, but also resource creation
and 'wiring'.  AMQP is a messaging protocol and the initial
conversation between client and server is about creating queues and
bindings, to do message routing and selection within the server.  In
the current designs, this is allowable at any point, and it's not
clearly separated from message flow.  Thus the protocol is a mix of
work which I think is naturally synchronous, with work that is
naturally asynchronous.  The control dialogs (synchronous and low
volume) should IMO be text based, the messaging should be binary
framed (if one wants to use binary framing).  Right now all of AMQP is
binary framed, so expensive to implement, incompatible between
versions, etc.  I'd prefer to see two protocols, a synchronous
text-based protocol for resource setup, upgraded to a thin async
binary protocol for message transfer.  Seems off the table, however.

> of AMQP over HTTP advertised not long ago.

That would probably be RestMS (www.restms.org).  It uses a RESTful
model for resource wiring, and has been waiting for Greg to propose
BWTP for the message transport. :-)

-Pieter

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 17 Jun 2009 12:15:28 +0100

> binary protocol for message transfer.  Seems off the table, however.

I'm fine with text-based protocols if you prefer that, but I think the
ability to rewire in the middle is quite useful, as it permits a lot
of useful automatic protocol adaptation to be hidden from applications.

It can be cleanly separate from the message flow without being
restricted to just the beginning, just as HTTP cleanly separates
message headers from the body.  If the problem is that it's readily
visible, update your diagnostic tools.  Diagnostically there's not
much difference between observing negotation at the start of a new
connection, and observing it in the middle of an existing one, if the
protocol is framed well.

For which I do advocate a sort of "big frame" surrounding a message
stream of little frames, so diagnostic/security tools don't have to
inspect the whole data stream to catch the big frame negotiations.

> BWTP for the message transport. :-)

:-) I've been thinking similarly, and I hope my proposals in the
pipeline will be a good fit for AMQP.  (Very similar to BWTP, but
based on practical experience using the obvious form of bidirectional HTTP).

-- Jamie

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 17 Jun 2009 13:47:16 +0200

> of useful automatic protocol adaptation to be hidden from applications.

It is sensible to switch from a low-volume text based synchronous
control protocol to a high-volume binary asynchronous message transfer
protocol, but not vice-versa.  It's also not sensible afaics to try to
mix the two in one connection (AMQP does this, and it is horrid).

So I'd say, if stacks want to rewire while they are sending/receiving
messages, they just do that on a separate connection.

> inspect the whole data stream to catch the big frame negotiations.

Indeed.  I like using HTTP for the control work because it is so
familiar to the infrastructure.  It's a big, friendly frame.

> based on practical experience using the obvious form of bidirectional HTTP).

This is the thin message transfer protocol we eventually built for our
AMQP implementation: http://wiki.amqp.org/spec:6.  The AMQP protocol
itself is not open to random contributions, unfortunately.

Cheers,
-Pieter

---

From: Ian Hickson <[email-masked]>
Date: Wed, 17 Jun 2009 18:23:36 +0000 (UTC)

On Wed, 17 Jun 2009, Salvatore Loreto wrote:
> > 
> > The WebSocket protocol is designed on the principle that there should 
> > be minimal framing (the only framing that exists is to make the 
> > protocol frame-based instead of stream-based, and to support a 
> > distinction between Unicode text and binary frames). It is expected 
> > that metadata would be layered on top of WebSocket by the application 
> > layer, in the same way that metadata is layered on top of TCP/IP by 
> > the application layer (HTTP).
>
> you sent on this list a couple of months ago: 

Minimal framing isn't a requirement. Being able to implement the 
server-side component easily is:

| - It must be possible to implement a fully-conforming server-side 
|   component for this in a few dozen lines of scripting code, in the case 
|   where sharing the connection with an HTTP server isn't required.

(I haven't tried implementing BWTP to see if it's possible to implement it 
in a few dozen lines of code; WebSocket can be implemented in Perl using 
only Encode, IO::Socket, and IO::Select in about 100 lines of code.)

I should say, though, that I have no objection to BWTP personally (or to 
any other protocol for that matter). It's more complex than what I would 
ideally like, but if there are implementors or authors who think it is the 
better solution than Web Socket then I encourage them to use it.

> > Conceptually, WebSocket is really just a layer on top of TCP/IP that adds a
> > Web "origin"-based security model for browsers; 
>
> the problem here is that WebSocket are not built directly on top of 
> TCP/IP, but it tries to reuse the HTTP infrastructure to in somehow 
> perform a downgrade from the HTTP protocol to the TCP protocol and then 
> start to build on top of it.

WebSocket stands alone, it just happens to be compatible with an HTTP 
Upgrade handshake. I expect most uses of WebSocket to not involve any HTTP 
servers and to just be a client connecting straight to a WebSocket server.

This is why, for instance, the handshake isn't actually defined in terms 
of HTTP, but in terms of bytes that happen to be compatible with HTTP.

It's wrong to consider WebSocket to be based on HTTP more than TCP.

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 18 Jun 2009 11:58:02 +1000

> and/or join the site and change it.

We need google wave :)

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 29 Jun 2009 09:10:25 +1000

> |   where sharing the connection with an HTTP server isn't required.

Ian,

Can you expand on why you think this is a requirement?

Are there any other protocols/experiences that you base this on where
average developers have needed to program a server from a raw socket
rather than using a library.

> only Encode, IO::Socket, and IO::Select in about 100 lines of code.)

The intention for BWTP is not for servers/intermediaries to be written
from scratch.   Rather existing HTTP server and proxies servers should be
able to be updated to support BWTP.      The additional code needed
to modify an existing HTTP parser to BWTP should be no more than a
few lines of code.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 29 Jun 2009 00:25:46 +0100

Greg Wilkins wrote:
> The additional code needed

I think it's disingenuous to portray BWTP as a few lines of code,
because nobody worries about a "HTTP parser", and nobody uses a "HTTP
parser library" which would be changed for BWTP.

Implementations are always a server, client or both, with the "parser"
being some trivial little part that really is not worth discussing.

Most servers only parse requests and emit responses, and dispatch those
requests and responses in a particular way, often with multiple
processes or threads, or sometimes in sequence.

BWTP changes that pattern quite significantly, and that's where the
interesting part goes.

Of course any bi-directional protocol we're discussion does the same.

The point is that with the effort to build something which handles
bi-directional messaging, the actual *parser* is not likely to be
something to worry about.  Really, the tricky stuff is in keeping
sockets alive, timeouts, flow control, orderly shutdowns, retries,
handling protocol errors.

That stuff applies with any of the bi-directional protocols, even the
ones which haven't said anything about it because they haven't been
implemented and tested yet.  Some of them expect Javascript
applications to deal with it (which imho is a nice way to ensure low
quality implementations.)

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 29 Jun 2009 09:47:01 +1000

> parser library" which would be changed for BWTP.

Jamie,

that is true - but I was trying to compare apples with apples, as
I think a perl program that runs the websocket protocol is little
more than a parser.

> BWTP changes that pattern quite significantly, and that's where the

Indeed.   That part falls into providing an application framework
and I think is vital work to abstract an application developer
away from the nuts-and-bolts of a protocol.

> Of course any bi-directional protocol we're discussion does the same.

Agreed.  But this work is also not included in the "few lines" estimate
for the websocket implementation.

> handling protocol errors.

Here I think BWTP will be able to expand on existing HTTP connection
handling.  However it remains to be seen how much will need to be
changed - and I expect it will be more than initially expected.

> quality implementations.)

I generally agree with the sentiment of your post, because I think that
the lines-of-code measure is a poor metric on which to evaluate proposed
protocols.

Having said that, the complexity of implementation is indeed a good
gross measure that should be considered.   For example, my earlier
advocacy for considering BEEP has been dulled because people pointed
out some of the non-trivial complexities that would need to be implemented.

cheers

---

From: Ian Hickson <[email-masked]>
Date: Mon, 29 Jun 2009 00:51:40 +0000 (UTC)

> Can you expand on why you think this is a requirement?

If we make it difficult to write a compliant implementation, then we won't 
get many compliant implementations. This will either be because we don't 
get many implementations at all, or because we get many poor quality 
implementations.

Having few implementations is bad because it leads to a monoculture where 
a single vulnerability means the entire ecosystem is vulnerable.

Having many poor quality implementations is bad because interoperability 
will be difficult to achieve, making it less likely that users will have a 
good experience.

> rather than using a library.

CGI is one example of a technology (though not a protocol) where authors 
have written many implementations, and been able to do so because of the 
relative ease of writing an implementation. I think WebSocket will be 
treated as very similar to CGI on the server-side; while there will 
probably be some widely used libraries, I expect there will also be many 
simple implementations that don't want to rely on third-party software.

I think it's important for this technology to be usable by authors as soon 
as browsers support it; in the case of WebSocket this means not requiring 
that HTTP servers be upgraded, which can take a long time. There are still 
many Web hosts using Apache 1.3, for instance. WebSocket is intended to be 
usable by anyone who can open a Unix socket, which is more or less anyone 
who can write a CGI script today.

> > (I haven't tried implementing BWTP to see if it's possible to 
> > implement it in a few dozen lines of code; WebSocket can be 
> > implemented in Perl using only Encode, IO::Socket, and IO::Select in 
> > about 100 lines of code.)
> 
> able to be updated to support BWTP.  The additional code needed to 
> modify an existing HTTP parser to BWTP should be no more than a few 
> lines of code.

Right, BWTP and WebSocket serve different goals. WebSocket isn't intended 
to be related to HTTP at all (though it has handshake that appears like 
HTTP and can therefore share a port with an HTTP server).

---

From: Ian Hickson <[email-masked]>
Date: Mon, 29 Jun 2009 01:06:40 +0000 (UTC)

On Mon, 29 Jun 2009, Greg Wilkins wrote:
> Jamie Lokier wrote:
> > Greg Wilkins wrote:
> >> The additional code needed to modify an existing HTTP parser to BWTP 
> >> should be no more than a few lines of code.
> > 
> that is true - but I was trying to compare apples with apples, as I 
> think a perl program that runs the websocket protocol is little more 
> than a parser.

It is a complete compliant implementation of the entire protocol, 
including all of the code that would be needed to actually use it. It's 
not just the parser, it also does all of the work of listening for new 
connections, handling closing connections, etc.

Naturally, this isn't a particular high quality implementation in terms of 
handling high load, etc, and I expect that more elaborate implementations 
will exist to handle scaling better, etc. However, the 99-line sample is 
more than just a demo; it can actually be used to run a real service.

> for the websocket implementation.

As far as I can tell, it does:

   http://damowmow.com/playground/demos/websocket/blank-server.pl

For example, here is a two-player tic-tac-toe server built using the same 
code:

   http://damowmow.com/playground/demos/websocket/tic-tac-toe.pl

The only part missing in that code is the tic-tac-toe win detection code, 
other than that it's a complete server that could run actual two-player 
games. It's 148 lines of code (using three built-in Perl libraries 
IO::Socket, IO::Select, and Encode, though you could do it using the core 
functions directly without much change; I just prefer the OOP-based 
interface of the IO libraries).

(I'm not making any claims that the code above is great or scalable or 
anything, only that it implements the protocol in a compliant manner and 
could be useful in practice for simple situations.)

> the lines-of-code measure is a poor metric on which to evaluate proposed 

Agreed. The requirement for WebSocket wasn't a specific number of lines of 
code; it was just that it should be possible to write the core without 
much effort ("a few dozen" lines of code). The idea is just to ensure that 
writing a fully-compliant server from scratch is feasible, unlike, say, 
with HTTP, where nobody can seriously consider writing a server from 
scratch unless they have several weeks or months of time to dedicate to 
the task.

I think there are roles to play for both technologies with a tiny server 
footprint (like WebSocket) and technologies with serious power and many 
features (like HTTP). They are not mutually exclusive.

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 29 Jun 2009 05:55:48 +0100

> who can write a CGI script today.

and

> WebSocket isn't intended to be related to HTTP at all (though it has
> handshake that appears like HTTP and can therefore share a port with
> an HTTP server).

When it shares a port with an HTTP server, you're going to need the
HTTP server to accept WebSocket (non-HTTP) traffic and forward it to
the WS server, or have the WS server forward HTTP traffic to the HTTP
server.

Having the HTTP server forward WebSocket traffic would require
upgrading the HTTP server, as WebSocket protocol is not compatible
with HTTP.  But you said that must not be required, so the only way to
do it is forwarding everything through a WebSocket->HTTP proxy.

Which is a good idea anyway (see cometd and optimal high-load Comet
architectures), but doesn't sit well with the notion of implementing
WebSocket casually.

Secondly, sharing port 80 is going to hit HTTP proxies, low-timeout
NATs, mobile IP connection drops, and HTTP-filtering firewalls.

It's the port 80 issues - HTTP proxies, filters and spontaneous
connection drops - which I think of when I say that a real
implementation intended for the larger Web is not trivial.  Whether
the tricky things are dealt with in WebSockets itself, or in client
applications written by people who only fix what they react to, and
thereby provide a low quality / broken experience for everyone behind
a low quality web connection.  I favour infrastructure which makes
that less likely, rather than more.

I don't think of server scalability as a problem, which is a
relatively easy with modern libraries and frameworks.  I was surprised
you mentioned it.

-- Jamie

---

From: Ian Hickson <[email-masked]>
Date: Mon, 29 Jun 2009 05:19:41 +0000 (UTC)

On Mon, 29 Jun 2009, Jamie Lokier wrote:
> Ian Hickson wrote:
> > I think it's important for this technology to be usable by authors as 
> > soon as browsers support it; in the case of WebSocket this means not 
> > requiring that HTTP servers be upgraded, which can take a long time. 
> > There are still many Web hosts using Apache 1.3, for instance. 
> > WebSocket is intended to be usable by anyone who can open a Unix 
> > socket, which is more or less anyone who can write a CGI script today.
> 
> and
> 
> When it shares a port with an HTTP server, you're going to need the HTTP 
> server to accept WebSocket (non-HTTP) traffic and forward it to the WS 
> server, or have the WS server forward HTTP traffic to the HTTP server.

Yes, when you want to reuse a port it's going to be much more work than 
100 lines of code, that's for sure.

> Having the HTTP server forward WebSocket traffic would require upgrading 
> the HTTP server, as WebSocket protocol is not compatible with HTTP.  
> But you said that must not be required, so the only way to do it is 
> forwarding everything through a WebSocket->HTTP proxy.

I'm not sure what you mean. You can use WebSocket without an HTTP server 
just by using a port that isn't taken by an HTTP server, that's simple 
(assuming you control the server). If you want to coexist with a server, 
then you have much more work ahead of you, but WebSocket does support that 
case too, and it shouldn't be that hard to do really (you just have to 
catch the Upgrade request before you send any bytes back, so that the 
WebSocket part of the server can do the handshake).

> implementation intended for the larger Web is not trivial.

Well you can still use port 80 (or 443) without an HTTP server if you have 
slightly more control over the server, without having to do much more work 
in the way of coding. I agree that if you want to use the same IP and port 
for both HTTP and WebSocket that it will be a lot more work, but that's 
not required for casual use, you can just use another port. It won't work 
in all situations (e.g. when the user has an over-eager firewall), but 
then that's the cost of being able to write a server in 100 lines of code.

> Whether the tricky things are dealt with in WebSockets itself, or in 
> client applications written by people who only fix what they react to, 
> and thereby provide a low quality / broken experience for everyone 
> behind a low quality web connection.  I favour infrastructure which 
> makes that less likely, rather than more.

I don't understand what you mean here. What would a client have to do? 
What would WebSocket have to do that it doesn't already do?

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 30 Jun 2009 13:05:52 +1000

Ian,

I think lines of code and software quality are orthogonal.

There are many many programmers who are able to produce
very poor quality code in just 1 line.  Even with simple CGI,
I have had many experiences where poor quality client "implementations"
have had unintended consequences.

I believe the path to a good user experience is to give them an
environment that abstracts away the details of the protocol.
Websocket does this on the client side, so I don't see why this
should not be done server side.

But I do agree with you that the protocol should not be difficult to
implement and that we do not want a monoculture.   But it is overly
simplistic to say a protocol that needs more than a few dozen
lines of code is going to be rarely implemented or of poor
quality.

cheers

---

From: Ian Hickson <[email-masked]>
Date: Tue, 30 Jun 2009 03:20:33 +0000 (UTC)

On Tue, 30 Jun 2009, Greg Wilkins wrote:
> environment that abstracts away the details of the protocol. Websocket 
> does this on the client side, so I don't see why this should not be done 
> server side.

I would definitely enourage people to write libraries and the like to 
support WebSocket, indeed. I don't think just because it should be 
possible to write your own WebSocket server that everyone should want to 
do so. (Just like with CGI, most people use libraries, but it's possible 
and easy to not do so if all you want to do is something simple.)

> simplistic to say a protocol that needs more than a few dozen lines of 
> code is going to be rarely implemented or of poor quality.

It is definitely a generalisation, yes. I think there's probably a 
correlation between the typical size of a fully compliant implementation 
and the implementation complexity of the spec, and there's probably a 
correlation between the complexity of the spec and the likelihood that 
implementations will contain bugs. However, it's certainly not the only 
metric to spec or implementation quality.