Subject: [hybi] Apples and Orangutans

From: Mark Lentczner <[email-masked]>
Date: Wed, 8 Apr 2009 08:47:29 -0700

From the point of view of our customers, the application level  
authors (both client and server side), I think there are two major  
questions they would ask to characterize any potential protocol:

1) What is the data model?
	Minimal (binary blob and/or UTF-8 string)
	Structured (JSON object or XML stanza)
	Message (data and meta-data)

2) What is interaction model?
	Event (datagram at the senders whim)
	Request/Response (request at senders whim, with response from the  
receiver)

While issues of data framing, efficient use of TCP, and ease of  
implementation, will influence their view, I don't think they will be  
nearly as important as these first two questions[*].

The protocols we're discussing can be characterized by these two  
questions:
	WebSockets: Minimal, Event
	BOSH, Bayeaux: Structured, Event
	rHTTP:  Message, Req/Resp

Other protocols can be fit into this framework, with perhaps some  
extension:
	XMPP: Structured , Event
	TCP: Minimal, Stream
	SMTP: Message, Event
	HTTP: Message, Req/Resp

I think the first thing to look at is which 'buckets' are we trying to  
serve? And only then compare items within a bucket. I don't think it  
makes sense to discuss WebSocket vs. rHTTP -- The discussion will  
inevitable be more about Minimal Event vs. Message Req/Resp.

Now, while clearly any of these be used to implement the others, there  
is a cost. Where simpler models are implemented with more complex,  
then there is overhead on the wire and processing. Where more complex  
models are implemented on simpler, then the burden falls to the  
application developer.

I'm think it will be productive to discuss which models we are trying  
to serve. I suppose this is the same sort of thing Roy has been asking  
for. I'll try to get this into the wiki, too. I'll leave my personal  
views for a second post...

	- Mark

[*] Okay, I admit, there is actually one even more important question:

0) Is the protocol available in the environment that the application  
is built on?

For protocols described here, some are trying to support the web-page  
environment, others are not, though often could. Kudos for WebSockets  
for describing a DOM level API. I realize that it isn't generally IETF  
practice to do so, but having a description of a base expected API  
(not to mention implementations) is actually very important to  
developers.

Mark Lentczner
Sr. Systems Architect
Technology Integration
Linden Lab

[email-masked]

Zero Linden
[email-masked]

---

From: Ian Hickson <[email-masked]>
Date: Wed, 8 Apr 2009 19:48:43 +0000 (UTC)

On Wed, 8 Apr 2009, Mark Lentczner wrote:
> 
> I think the first thing to look at is which 'buckets' are we trying to serve?

I think it would make sense to serve several of these buckets. Personally 
I am interested in "Minimal, Event", but I don't see any reason why we 
wouldn't also have the other models.

It's not clear to me how we should proceed, though. Would it make sense to 
split into subgroups to work on the various buckets that people are 
interested in?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Paul Prescod <[email-masked]>
Date: Wed, 8 Apr 2009 13:02:10 -0700

On Wed, Apr 8, 2009 at 8:47 AM, Mark Lentczner <[email-masked]> wrote:
> From the point of view of our customers, the application level authors (both
> client and server side), I think there are two major questions they would
> ask to characterize any potential protocol:
>

I would add a "data synchronization" data model. I find that most
often the goal is to have the web browser report the state of a
resource or resource collection to a user, and then inform the user of
changes to it.

The resource might be "chat transcript", "current location of a
train", "objects currently visible in the game", "state of the game
board in a Chess game", "set of currently visible emails."

For example, one could imagine it would be a list of key-value pairs,
or SQL rows or DOM nodes (I worked on such a protocol for DOM nodes a
long time ago).

 Paul Prescod

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 09 Apr 2009 07:13:59 +1000

> split into subgroups to work on the various buckets that people are 

I see that there are 3 areas where the IETF could help, so perhaps
we could form subgroups or sub-discussions about:

1) What can we do if we can't change HTTP
-----------------------------------------

Ie is there anything that we can do to help best practises with
the HTTP specification as it is.

We could define common headers that identify long-polls and thus give
hints to infrastructure to better handle them (eg pipeline only other
longpolls behind a pipeline).

Another example of what could be done is that the open ajax alliance wrote
an open list of demands to the browser vendors listing the key pain points
for Ajax / Comet development (eg lack of pipeline control, need for
better interframe eventing etc.).    Perhaps the IETF could do
similar from the network point of view.

Specifically I think the IETF should be saying
is something about the 2 connection limit (either it is or
it is no OK to raise it and the IETF should be able to
clearly say which it is).    We could also recommend how
pipelining as it is currently defined could be better
used for long polling... if only the clients allowed
control of pipelining.  etc.

2) What can we do to improve HTTP for bidirection usage
-------------------------------------------------------

If there was a new HTTP spec developed (and I have no idea
if one is on the horizon or not),  what could we change
to improve it's use for bidirectional use-cases.

My favourite here is 1xx methods with content, but
there are sure to be other things we could change.

3) What can we do to improve usage of other protocols
-----------------------------------------------------

Are there recommendations we can make to improve the
generic handling of upgrade so that proxies and
gateways could be expected to better handle upgrade
to specific (XMPP, WebSocket, BEEP) and arbitrary
TCP based protocols.

cheers

---

From: Ian Hickson <[email-masked]>
Date: Wed, 8 Apr 2009 21:48:11 +0000 (UTC)

On Thu, 9 Apr 2009, Greg Wilkins wrote:
> Ian Hickson wrote:
> > It's not clear to me how we should proceed, though. Would it make 
> > sense to split into subgroups to work on the various buckets that 
> > people are interested in?
> 
> I see that there are 3 areas where the IETF could help, so perhaps we 
> could form subgroups or sub-discussions about:
> 
> We could define common headers that identify long-polls [...]
>
> If there was a new HTTP spec developed (and I have no idea if one is on 
> the horizon or not), what could we change to improve it's use for 
> bidirectional use-cases. [...]
> 
> Are there recommendations we can make to improve the generic handling of 
> upgrade so that proxies and gateways could be expected to better handle 
> upgrade to specific (XMPP, WebSocket, BEEP) and arbitrary TCP based 
> protocols.

These seem to all boil down to extensions or changes to HTTP. It seems 
that those should probably be discussed in the ietf-http-wg mailing list 
as part of HTTP working group discussions.

> Another example of what could be done is that the open ajax alliance 
> wrote an open list of demands to the browser vendors listing the key 
> pain points for Ajax / Comet development (eg lack of pipeline control, 
> need for better interframe eventing etc.).  Perhaps the IETF could do 

The browser vendors are present here on this list, I don't think there's 
any need to write "demands" to them.

I would be interested in taking a specification for a "minimal, event" 
protocol for implementation directly by Web browsers to the standards 
track. This seems orthogonal to the above work. What would be the best way 
to move forward with such a goal?

---

From: Mark Lentczner <[email-masked]>
Date: Wed, 8 Apr 2009 17:06:40 -0700

Now, for part two, my personal view... (apologies for length)

I see the attraction of just giving something very simple to  
application developers. Giving them a protocol that does Minimal  
Events is easy to explain and would have a simple API. However, I have  
seen that over time, applications often need to build most of the  
framework that is already provided by more complex abstractions. Where  
the cost of those abstractions is small, and the ability to use them  
in simpler ways easy, it is often better to supply a richer  
abstraction to all, since the uniformity of the often implemented bits  
means better code for all.

For example, the overhead of structured data (in XML or JSON or some  
such) is generally seen as small when talking about internet needs  
(other than, say, streaming media). As such, many protocols and APIs  
these days are happy to use a structured data form exclusively, even  
when some applications will only make use of it to transport simpler  
data.

Similarly, Request/Response can be used to implement events, by  
treating the requests as the events, and giving only a minimal  
acknowledgement of receipt as the response (which could be ignored.) I  
note that many event systems at the lower level actually have this  
acknowledgement response (i.e. AMQP).

At Linden Lab, we've found the level of abstraction offered by HTTP's  
messages and request/response framework to be right in the sweet spot:  
It has a lot of the common things applications want (or eventually  
need) in their communications, and adds them in a rather low overhead  
manner. Judging by the large number of APIs written to work over HTTP,  
even where a browser isn't involved, I don't think we're the only ones.

Hence, I'm interested in looking at protocols in Message, Req/Resp.  
bucket.

Now, while clearly any protocol in any of these buckets can be used to  
implement the others, there is a cost. Where simpler models are  
implemented with more complex, then there is overhead on the wire and  
processing. Where more complex models are implemented on simpler, then  
the burden falls to the application developer.

Example: Implementing a Minimal, Event protocol with Message, Req/Resp  
is just a matter of accepting the overhead of the framing of the  
simple data blob, and the cost of the minimal response message. On the  
other hand, implementing a Message, Req/Resp. protocol with Minimal,  
Events, means building a whole framework for encoding message parts  
(address, meta-data and data) into the minimal data units (strings?  
binary blobs?) and coming up with a way to correlate responses to  
requests, etc... all in the application layer.

If you think that the features of Message, Req/Resp. are common  
enough, and the overhead of those features when not used low enough,  
then I think building a Message, Req/Resp. protocol is the winner.  
Having common support in server frameworks, client libraries, and  
Internet infrastructure will give rise to wider interoperability than  
if each application has to build the features they need.

So, how to proceed? How can the IETF process help? We could determine,  
for each bucket that there is interest in, how to achieve it a)  
without changing HTTP (the fall-back), and b) using newly defined  
protocol (via either wholesale new protocol, or HTTP extension). This  
would provide guidance to software authors for best practices, lead to  
interoperable server and client components, ensure applications work  
well with infrastructure, and enable infrastructure to better support  
these applications.

I'm deeply interesting in supporting applications that want Message,  
Req/Resp. semantics. I'd like to help with applications that have  
needs in other buckets. I actually think we could do this with just  
one protocol (and fall-back methodology), and would love to try to  
achieve that.

	- Mark

Mark Lentczner
Sr. Systems Architect
Technology Integration
Linden Lab

[email-masked]

Zero Linden

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 09 Apr 2009 12:28:55 +0900

> resource or resource collection to a user, and then inform the user of

That is indeed a very good point. Most kinds of collaborative activity 
can easily be looked at that way. In its full incarnation, maybe it 
would just allow an application programmer to say: synchronize this part 
of this page (i.e. DOM) with this part of that page over at that client.

Regards,    Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 09 Apr 2009 15:31:40 +1000

> as part of HTTP working group discussions.

I was hoping that this list could have a discussion about
the general requirements and possible solutions of bidirectional
HTTP.   Then if some consensus views were developed, these could
be taken to the appropriate forums: ietf-http-wg etc.

But then I'm not familiar with ietf processes.  Should I be taking the
concerns I've raised here to the http wg list?

> any need to write "demands" to them.

"demands" was not a good choice of words.
The point was that the OAA prepared a qualified/vetted list of issues
as a consensus document to take to the browser vendors.

> to move forward with such a goal?

How would this protocol be different from other existing protocols
such as BEEP?

cheers

---

From: Ian Hickson <[email-masked]>
Date: Thu, 9 Apr 2009 06:24:36 +0000 (UTC)

On Thu, 9 Apr 2009, Greg Wilkins wrote:
> > track. This seems orthogonal to the above work. What would be the best 
> > way to move forward with such a goal?
> 
> How would this protocol be different from other existing protocols such 
> as BEEP?

BEEP isn't minimal.

(Also, BEEP would have to be extended to support appropriate security 
checks for the Web's same-origin model.)

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 09 Apr 2009 08:14:22 -0600

> checks for the Web's same-origin model.)

The same-origin model is often invoked here. Is there a good definition
of it? Perhaps it would be helpful for someone to write a small I-D
about this.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: "Linner, David" <[email-masked]>
Date: Thu, 9 Apr 2009 17:08:35 +0200

Hello Ian and all,
I'm new to the list and currently trying to catch up with the discussion. Paul and Martin raised an interesting point here. What is really needed, a tool for bidirectional communication between browser and server, a tool to enable real-time collaboration between browsers, or both?
Looking into the hybi wiki and the use cases listed there (e.g. collaborative editing, multiplayer games) I get the feeling state synchronization is quite important. So, I think a shared memory interaction model would be suitable for these use cases. Implementing such an interaction model on top of bidirectional messaging between browser and server is not a problem, but a realization as JavaScript library would result in a significant loss of performance.
I would like to go for a fast protocol with HTTP semantics (put, get, update, delete), support for timely notifications of state changes (from server to client) and preparation for concurrent modifications.
Best regards, David

> -----Ursprüngliche Nachricht-----
> Von: [email-masked] [mailto:[email-masked]] Im Auftrag
> von "Martin J. Dürst"
> Gesendet: Donnerstag, 9. April 2009 05:29
> An: Paul Prescod
> Cc: [email-masked]
> Betreff: Re: [hybi] Apples and Orangutans
> 
> 
> 
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 10 Apr 2009 11:06:52 +1000

>> How would this protocol be different from other existing protocols such 

BEEPs framing is pretty minimal, yet full featured with respect optional
headers, chunking, multi response etc etc.

But I do agree that the channel negotiation stuff is a bit non-minimal

However, IF we are in the game of picking winners or designing new protocols,
we could do a lot worse than starting with a close inspection of BEEP and
the concerns that they address.

Maybe a BEEP-lite protocol would be a good upgrade target and one
capable of carrying arbitrary other protocols through firewalls
and gateways?

> checks for the Web's same-origin model.)

If we were using the upgrade mechanism, then I believe that most
of these concerns are addressed.  The standard HTTP security model
would be used to establish a connection and then the upgrade is
unable to break out of that connection etc.

This applies for websocket, rHTTP or any other upgrade protocol.
If there are security issues that exist even within such an
upgrade model, then it would be good task for this group
to identify them as a measure for any candidate upgrade protocols.

cheers

---

From: rektide <[email-masked]>
Date: Fri, 10 Apr 2009 23:33:45 -0400

> of this page (i.e. DOM) with this part of that page over at that client.

This seems like something that would be ideal to layer on top of a HyBi/"Server Initiated
HTTP" transport.  It doesnt sound like something that everyone using duplex HTTP would want.
Whether or not the conversation about this possible protocol belongs here, I truly do not
know, but I want to call it out right now as something I see as decidedly "not transport",
which was the question I interpretted as lying at the heart of Apples and Orangutans.

rektide

---

From: rektide <[email-masked]>
Date: Fri, 10 Apr 2009 23:29:40 -0400

rektide

---

From: rektide <[email-masked]>
Date: Fri, 10 Apr 2009 23:23:24 -0400

On Wed, Apr 08, 2009 at 08:47:29AM -0700, Mark Lentczner wrote:
> From the point of view of our customers, the application level authors 
> (both client and server side), I think there are two major questions they 
> would ask to characterize any potential protocol:
>

A messaging protocol ought be able to meet all of the interaction & data models above, 
ought be designed to convey any content, and potentially ails one of the greatest pitfalls 
of the web: the two/multiple connection limit[1].  

If the web ecosystem evolves towards w3c widgets and integrative software networks ala 
OpenSocial, the connection limit must be addressed.  And not by juvenile hacks to permit 
more connections (eg by creating subdomains); the real answer is that the web needs a 
multiplexed communication channel.  I can think of nothing more suitable for holding multiple 
conversations over a single wire than a messaging protocol.

Recently in #persevere (April 7), reflecting on HyBi:
<rektide> i really love greg wilkins constant chime ins that, http is a great model, why
break complete from it

HTTP already meets many of the critereon for being a suitable messaging protocol, and I
think this is of capital importance.  HTTP pipelining allows asychronous data sending, Kris
Zyp has developed HTTP Channels[2] to permit decoupled asychornous data recieving, and if
these are leveraged together, it leaves only the outstanding issue of proxies and multiplexing:
adapting HTTP to be a multichannel bidirectional messaging system supporting meta-data. 

Kris' model in HTTP Channels introduces a client-id and a sequence-id that can be used to
maintain conversations while clients are hurling a stream of pipelined HTTP requests and a
server is sending data to the clients in an out of order manner.  These tokens are sent as
HTTP headers, and although not presently a part of the server response in Kris' HTTP Channels 
spec, I believe ought be adapted as the primary multiplexing mechanism.

The one issue I think isnt particularly well addressed is broadcast messaging; can the
server send events to multiple or all client-id's, and how do these broadcast messages get
serialized?

Assuming the proxy issue can be adapted to be non terminal, assuming a persistent TCP
connection is not a deal breaker (i.e. a non connection oriented transport such as UDP is not
required), it ought be possible to fully extend HTTP to be a asychronous bidirectional
multiplexed communication channel, and further, I believe it would be possible to implement
a client side implementation on top of existing web browsers entirely client side (although
browser code for strictly enforcing multiplexing constraints would be "a good thing").

[1] http://www.openajax.org/runtime/wiki/The_Two_HTTP_Connection_Limit_Issue
[2] http://cometdaily.com/2008/05/13/http-channels-2/

> Other protocols can be fit into this framework, with perhaps some  

To emphasize, we already have the tooling to make HTTP sending and (most of the tools for
making) HTTP recieving asychronous.  This Event model HTTP still supports Req/Resp http
communication as well.

> is a cost. Where simpler models are implemented with more complex, then 
> there is overhead on the wire and processing. Where more complex models 
> are implemented on simpler, then the burden falls to the application 
> developer.

The cost savings I'd like to see most is being able to leverage HTTP, by simply removing
its fault of being so sychronous & undirectional.  If anyone believes a fully duplex multiplexed
communication channel is of itself insufficient or ill suited for your needs, please speak 
as to why.  If you find the HTTP framing unsuitable, eg a header & a payload, that too would
be intesting to hear.

Good day all, 
rektide

---

From: "Linner, David" <[email-masked]>
Date: Mon, 20 Apr 2009 11:34:01 +0200

> which was the question I interpretted as lying at the heart of Apples
I don't agree. Some application requirements are better answered on
transport level. You can realize something like Martin's "synchronize
this part of this page (i.e. DOM) with this part of that page over at
that client" with HTTP Channels on top of a bidirectional message
protocol over TCP. But how many clients can be served? What latency do
users experience? 

If you have a use case such as real-time state sharing (e.g. multiplayer
games) you need to care about fast handling of concurrent writes to the
same resource by several users, but you perhaps don't have to care about
lost packets with updates too much.
For example we implemented a round based client-server protocol for Web
state sharing on UDP. The server manages the actual states of all
resources and moderates the synchronization rounds. A round has a length
of 100-300ms and comprises three phases, i) call clients to instantly
send updates, ii) wait a predefined time (>2*network delay) and collect
update requests from all clients, iii) apply updates, resolve conflicts,
send difference to last state to all clients (and kick-off next round).
The communication is client initiated; the short round time ensures NATs
are kept open. Packet loss means that contained updates are communicated
one round later if still required. Clients always only get the
difference of their state to the current state at the server (no already
overwritten states). Tests with a Firefox extension and some real-time
Web applications went very well.

Protocols like the above are not suitable for bi-directional
communication of messages with arbitrary length, but they meet pretty
well the requirements of a real-time game or an application for
real-time collaborative work. If the WebSocket aims at such use cases, I
think it is at least worth to discuss if some protocol concepts or
parameters can be included with the specs discussed on this mailing list
to explicitly support state sharing on transport level. Our experiments
clearly showed that any protocol semantic coded in JavaScript and
interpreted at runtime affects the communication performance.

On Thu, Apr 16, 2009 Jamie Lokier wrote
> S. Mike Dierken wrote:
> > If I were writing a multi-player game, I'd definitely want to
> identify
> > portions of game state and media assets with something like a URI,
> have
> > various standardized content types, consistent methods (uniform
> interfaces),
> > etc. So HTTP concepts sound helpful.
> 
> I agree.  But I know some people don't want those concepts; to them,
> they are overhead.  I'm leaning towards a protocol which makes the
> concepts easily available (and I think most people will use them), but
> makes them optional.
That would be a good starting point. 

Regards,
David

p.s. Sorry for the late reply, I didn't find time last week.