Subject: [hybi] Request-response inversion...or not

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 7 Jul 2009 22:40:10 -0500

I spent a lot of time recently thinking about the problem of latency in the context of long-polling.

(Based on hints I'm getting, I suspect that this wont be any sort of revelation to others, but it was to me.  Maybe I'm just slow, but I've not seen anything written down to this effect so far.)

When long-polling, the current philosophy uses POST and inverts the standard request-response arrangement.  HTTP POST requests carry responses from the client to previously received HTTP responses.

POST makes sense because it's the one request that lets you just send data to a resource.  All the other methods have all these semantics.

Thus, there is the problem that there is a forced period of latency between when the server can send one unit of data and the next.  The server must wait for the previous unit to travel to the client, and then wait until it receives the next HTTP request from the client.

No Pipelines

The problem here is that you can't pipeline POST.  A POST can't be sent until all previous requests are completed.

Pipelining in this context is quite attractive.  If pipelining were viable, a client could send multiple requests, giving the server a number of opportunities to send data before exhausting the outstanding requests.  The number of outstanding requests could be tuned to ensure that plenty of requests were always available.

That sounds nice, but you need idempotence to use pipelining and none of the idempotent methods can be used for this sort of interaction.  You can't use idempotent methods to get this behaviour...or can you?

A RESTful shift of perspective?

Reading about and implementing REST has led me to think of new ways to mangle old concepts into a shape that is more compatible with the philosophy.  That led me to a realization about how thinking about request-response inversion was the wrong way to think about the long-polling problem.

You see, I was focussed too directly on the problem and the way it was formulated: how does a server request that a client send it some data?

When you have a long-poll, the natural tendency is to think of the HTTP response as a "request" and some later HTTP request as a "response" to that "request".  That's a pretty well-worn paradigm and one that I'm familiar with, comfortable with.

That way of thinking is a trap.  HTTP isn't going to help you if you think that way.

RESTful long-polling

Here's how I'm going to implement long-polling instead.  There are a few details that might change, but the basic principle seems sound.

My application starts off in the standard realm.  It happens to use a POST request, but that's largely immaterial.  If a client sends information that indicates that it's willing to respond to "requests" for information, then the response will indicate a URI where the client can long-poll.

The resource that the client retrieves will just contain a document.  This document will describe what information the server desires.  On first retrieval, the response will be immediate and will likely say that the server desires nothing.

The client then begins long-polling to monitor for changes in resource state, using GET.  This will probably need the new header being proposed for request time limit, in combination with If-None-Match or If-Modified-Since.  The server waits to respond until either the time limit expires or the resource state changes.

When the server wants the client to send it something, it changes the resource.  The resource then contains an indication of what information is desired, and where that information is desired, as a URI.  This might be a temporary URI, but that's largely irrelevant and up to the application.

The client, upon receiving a response, processes the document, sees that some information is required and uses PUT to provide the information to the indicate URI.

In this fashion, idempotent requests are used.  Pipelining can be used.  Other requests to the same server can continue, and if we're making requests to other requests to the same server, those are unaffected.

The client goes back to monitoring the resource, which can revert to its original state, or remain as it was.  Only changes in the resource trigger the client to PUT.

So, all that is necessary is a way to monitor the state of a resource.  The rest we already have.

No surprise?

My guess is that this use case - or something very much like it - is what motivated the set of goals in draft-loreto-http-bidirectional.  It's just a shame that this hasn't been clearly elucidated to date.  There's a leap: from the description of long-polling to the set of conclusions.

So my question is: is this the model that we want to codify and support with protocol modification?

Cheers,
--Martin

------------------------------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise private information.  
If you have received it in error, please notify the sender
immediately and delete the original.  Any unauthorized use of
this email is prohibited.
------------------------------------------------------------------------------------------------
[mf2]

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 08 Jul 2009 18:13:48 +1000

Martin,

various random comments in-line....

Thomson, Martin wrote:
> When long-polling, the current philosophy uses POST and inverts the standard request-response arrangement.  HTTP POST requests carry responses from the
> client to previously received HTTP responses.
> 
> POST makes sense because it's the one request that lets you just send data to a resource.  

Well GET's can carry data as well.

The other thing about POSTs is that they cannot be retried automatically, so you get some
protection from duplicate messages.

> All the other methods have all these semantics.
>
> Thus, there is the problem that there is a forced period of latency between when the server can send one unit of data and the next.  The server must wait for
> the previous unit to travel to the client, and then wait until it receives the next HTTP request from the client.

A lot of people are worried about this long polling latency,  but I'm not sure they should be.

If your long poll is returning on average every 30s, and a typical network transit is
50ms, then an event only has a 0.333% chance of hitting the window and then it will get an
average extra latency of (50+50)/2, so that 0.166ms of extra latency on average.

So your average latency of 50.166ms instead of 50ms!

OK, this will be more if messages are more frequent, but even if you send a message
every second, you are only looking at a 10% chance of hitting the bad spot, so your
average latency will go from 50ms to 55ms.

Many networks have much lower latencies than 50ms and thus the numbers are even better.
For networks with larger latencies, then those are the sorts of networks where
packet corruption and/or loss are going to give you more jitter in your latency than
anything caused by long polling.

Remember this is over TCP/IP, so you are going to get jitter as packets can be delay,
rerouted, corrupted of just lost and retransmission will be required.
If your wifi corrupts the data, then a nack has to be sent. If the packet is lost then
you have to wait for a timeout or subsequent packet to be sent so that the lost
packet is noticed.

So normal network stuff is going to inject far more sporadic latency than the occasional
long poll refresh and the maximum latency can be seconds. If you want to remotely control
a heart pacemaker, then long polling over HTTP over TCP/IP is not the protocol for you!

> The problem here is that you can't pipeline POST.  A POST can't be sent until all previous requests are completed.

GETs can be pipelined, but browsers don't let you do that.  So while pipelining long poll GETs might be workable
in some circumstances, I don't think it is a panacea.

More over, I think the complexity is just not work the small gains it will give.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 8 Jul 2009 12:34:01 +0100

Greg Wilkins wrote:
> A lot of people are worried about this long polling latency, but I'm
> not sure they should be.  If your long poll is returning on average
> every 30s, and a typical network transit is 50ms, then an event only
> has a 0.333% chance of hitting the window and then it will get an

Obviously it's not an issue for things which update every 30 seconds.

> OK, this will be more if messages are more frequent, but even if you
> send a message every second, you are only looking at a 10% chance of
> hitting the bad spot, so your average latency will go from 50ms to
> 55ms.

Peak latency jumps to 100ms.  Yes, I see that's not nice but it's not
going to significantly affect something with update messages once per
second.

On the other hand, I see internet links with latencies more like 400ms
if you're behind a 3g carrier, 160ms if you're behind a modem, and
those are on good days when everything is working smoothly.

The variance in latency can quite significant.  For 3g, I see ping
times fluctuating between 150ms and 2s.

In people's houses, it's quite common to have good quality cable
connections with high bandwidth, low latency from the internet, but
high upload contention, and therefore in the upstream direction high
loss, high average latency and high latency variance.

In other words, latencies are sometimes asymmetric.

So do the calculations for a service where the server has new messages
4 times a second to be displayed graphically, over a link with
50-100ms latency server-to-client and 50-5000ms latency client-to-server.

    - With long-polling, time between received messages is
      0.2 to 5.1 seconds, and buffer+flight time is 0.05 to 4.85 seconds.

    - With streaming, time between received messages is
      0.2 to 0.3 seconds, and buffer+flight time is 0.05 to 0.1 seconds.

Compare 5.1 seconds against 0.3 seconds (peak inter-update delay)
Compare 4.85 seconds against 0.1 seconds (peak message origin to display time)

Those figures are chosen to make a point, of course, and you may argue
that such network environments are too hostile to support such web
applications.

But they are realistic figures, and the streaming figures are meant to
show that under certain conditions, long-polling makes a large and
unnecesary difference.  You can do certain things with streaming which
you simply can't do with long-polling.

I routinely see those sorts of latencies when browsing the web or
using SSH at places where people have a shared router, and I routinely
see them on 3g mobile connections for a significant fraction of the day.

The long-poll latency obviously makes that sort of application
impossible on those occasions.  But with HTTP streaming, or even
pipelined long-polling, it can work.

There are lots of reasons to prefer long-polling anyway.
Compatibility and simplicity come to mind.

> Many networks have much lower latencies than 50ms and thus the

Yes.  LANs are typically <1ms, 100 microseconds even.

A lot depends on whether you have a low message rate like 1 every few
seconds, or a high "interactive" message rate from the server.

You wouldn't notice much jitter if you have a message rate like 1 per
second over a reliable <50ms network, or 1 per 5 seconds over a
slightly contended home internet connection.

Imho, Hybi should also cater to applications wanting a higher message
rate than that.  The HTML5-era browsers can handle it in their
rendering engines and DOMs.

> For networks with larger latencies, then those are the sorts of
> networks where packet corruption and/or loss are going to give you
> more jitter in your latency than anything caused by long polling.

I don't agree.  Client-to-server latency is particularly variable in
locations where there is peer-to-peer traffic from multiple clients
going through a shared router.  There, no corruption is expected, and
all the packet loss is in the upstream direction - downstream could be
sending messages at a high rate with consistent low latency.  Although
upstream affects ACKs, that doesn't stop a steady flow of downstream
small messages.  (As long as TCP_NODELAY is set).

3g mobile connections aren't quite the same, but they suffer a similar
effect.

> rerouted, corrupted of just lost and retransmission will be required.

If you have loss in the data carrying direction, yes.  Which is
another reason why long-poll isn't so nice: it amplifies the effects
of asymmetric packet loss affecting the client-to-server direction.

> So normal network stuff is going to inject far more sporadic latency
> than the occasional long poll refresh and the maximum latency can be
> seconds.

This is untrue when you have a rapid stream of messages to send over a
high latency link.

Video, real-time status displays of one thing or another, and
multiplayer 2d action games (like Flash games) are examples.

Obviously we're not sending video over long-polling, but the other two
are prime candidates for HTML5-era web applications.

> If you want to remotely control a heart pacemaker, then long polling
> over HTTP over TCP/IP is not the protocol for you!

Quite.  But there's an intermediate level of streaming quality where
long-polling adds enough latency - or variance - to affect some people
behind some kinds of network quite a lot, and message streaming over
TCP flows smoothly over the same networks.

> GETs can be pipelined, but browsers don't let you do that.

Don't they, at all?  (This is a question, not rhetorical, I'm curious):

What about using IMG GETs instead of XmlHttpRequest?  What about
IFRAMEs where the HTML contains a script?  What about script elements?
Are none of those scriptable GETs pipelined in current browsers?

> So while pipelining long poll GETs might be workable

I agree, mainly because HTTP pipelining is already a disaster area.
Partly due to bad implementations and compatibility issues, and partly
due to the design of HTTP pipelining itself.

But I do hope that Hybi performance isn't subject to the same disaster.

And, if there is any _reliable_ way to tunnel messages over HTTP with
streaming instead of long-polling - or any equivalent cunning trick -
I'd love to know about it.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 09 Jul 2009 09:01:33 +1000

> Obviously it's not an issue for things which update every 30 seconds.

Note that just because they update every 30s does not mean that low
latency is not important.

A forex trader/application waiting for a price that might change once
ever 30s or so, will still want to know about the change with low
latency.

> that such network environments are too hostile to support such web

True. Long polling is just not up to handling something
that updates 4 times a second.   That's an average inter-message
delay of 250ms which in your scenario is less than the network
traversal time.

So yes - long polling sux badly if messages are frequent.
Long polling is pretty good at waiting for infrequent messages that
can then be delivered with pretty good average latency.

So for the current crop of web-2.0 application, I think long polling
is well suited.

> you simply can't do with long-polling.

indeed, which is why websocket/bwtp/hybi etc is needed for the next
generation of web applications.

> TCP flows smoothly over the same networks.

Good point and well put.

I think that is actually very key in our considerations.  TCP/IP
does have a lot of smarts in it to keep data flowing as smoothly
as possible over unreliable networks.  Protocols like
websocket/BWTP/BEEP/something are going to better access that
capability than something built on top of HTTP.

So Martin... I should not have said "don't worry about long polling
latency".   I should have said "don't worry about long polling
latency if you are content with the current web-2.0 application
capabilities".

> Don't they, at all?  (This is a question, not rhetorical, I'm curious):

Oh browsers can pipeline.... they just don't let javascript
control the pipeline (too my knowledge).

You can't say "send this XHR pipelined behind that XHR".
The browsers handle pipelining transparently and may implement
strategies for pipelining unknown to the application.
Once you used all your available connections, it is a lottery
as to which connection is picked to use as a pipeline.

> Are none of those scriptable GETs pipelined in current browsers?

they can be pipelined, but you have no control.  So in a 2
connection environment, if you do:

  1. send long poll XHR
  2. send publish XHR
  3. send publish XHR

You have no control over which connection 3. will be pipelined on.
Obviously you want it to be on the same connection as 2, but I have
seen it pipelined behind 1, which is kind of disastrous.

cheers

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 8 Jul 2009 18:14:43 -0500

Hi Greg,

I had hoped that my email might trigger some discussion on something other than long-polling latency.

If you read through my post, the conclusions I came to had little - if not nothing - to do with latency.  Consider it colour, storytelling.  I'm telling you where it came from.  While I agree to some extent with your sentiments, and to a certain extent with Jamie, the topic is largely peripheral to the central theme. 

> protection from duplicate messages.
 
Incidentally, the "protection" offered by POST is less of a protection and more of a liability in most cases.

Sure, an HTTP client wont try again, but that's more as a consequence of it having to protecting itself against a bigger problem: inadvertently initiating some action twice.  In the event of an error, the HTTP layer doesn't know if the POST got through and has no option but to pass the problem to a higher layer.  At least with GET and PUT (and DELETE), the request can just be reattempted.

> > All the other methods have all these semantics.

Ta,
--Martin

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 09 Jul 2009 13:31:06 +0300

Hi Martin,

you are right that there is a leap in the draft from the description of 
long-polling to the possible Future work ...
we should have spent more energy to motivate the set of goals... but at 
least even a list has been able to trigger some discussion.
Shortly the motivations behind the list is:

    *how to improve the long polling, and then the current web-2.0
    application capabilities,
    extending HTTP with no or minimal impact on the current HTTP
    infrastructure*.

let me then start to write down some consideration:

I do not completely agree with your statement that with long-polling

    the natural tendency is to think of the HTTP response as a "request" and some later HTTP request as a "response" to that "request"

it really depends how it is implemented.
For example BOSH, to prevent data from being pipelined behind long poll 
request, uses *two HTTP connections*
(BOSH is then in some way similar to your proposal to implement long 
polling, but your proposal has the advantage to
let the server explicitly request data to the client).
Then using more then one HTTP connection give some advantages, but the 
problem is:

   HTTP [HTTP-1.1] section 8.1.4 recommends that a single user client
   should not maintain more than two connections to any server or proxy,
   to prevent the server from being overloaded.

so if you use both the two connection to implement long-polling, it 
would be almost impossible to send
a "plain" request from the client to the server, if not mixing "plain" 
and "long-polling" requests on one of the two
HTTP connections used to implement long-polling.
So perhaps a way (e.g. an header) to differentiate long-polling request 
form a "plain" request could improve the situation,
even if it won't offer any "protection " in the usage of POST for the 
situations outlined in your last email.

The header proposed for request time limit provides a simple way to 
indicate that the request is a "long-polling" one,
moreover it gives the possibility to the user to set (e.g. longer or 
shorter it) the timeout and then use different values (tune it) for 
different applications;
it also gives the advantage to understand that the response to that 
request can arrive any time until the timeout expires (value indicate in 
the header).
If the server or the proxies don't understand the header the application 
will just fall-back to the current situation.

About the latency that long polling generates, I have to say that of 
course there are way to improve and reduce it,
but it would be never possible to reduce it to the one generate by 
message streaming,
on the other end as also Jamie has pointed out there are a lot of 
reasons to prefer long-polling over HTTP streaming anyway.
Moreover, how to do streaming communication that would help the next 
generation of web application
and which protocol to use (even better which requirement the streaming 
communication should meet)
is something that relates to the HyBi "long term" solution discussion.

cheers
Sal

Thomson, Martin wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 9 Jul 2009 20:55:59 -0500

Hi Sal,

There's a lot in your email to digest, but I'll take a stab at a few points.

> -----Original Message-----
> From: Salvatore Loreto [mailto:[email-masked]]
> Sent: Thursday, 9 July 2009 8:31 PM
> To: Thomson, Martin
> Cc: [email-masked]
> Subject: Re: [hybi] Request-response inversion...or not
> 
> Hi Martin,
> 

I don't think we're going to get away with no impact on the protocol, but we can certainly ensure that the existing infrastructure can continue to operate with no support of the extensions.  I believe that what I've proposed does this.

>     the natural tendency is to think of the HTTP response as a
> "request" and some later HTTP request as a "response" to that "request"
> 

As always, my comments are coloured by my own experience.  Certainly, you can think of it in other ways.  Consider this more of the story if you like.  s/the natural/my natural/ if you like :)

I believe that BOSH-XMPP is largely uni-directional, but the 'ack' in BOSH forms a "response" of a sort.  The practice of connection specialization (one for long-polling, one for other requests) is a sensible paradigm.  We might like to publish down some recommendations on that front.

We might also like to consider how relevant the 2 connection limit is on the current Internet, where most of the world is wilfully ignoring that prescription.  My browser is particularly flagrant about it:

  network.http.max-persistent-connections-per-server = 6
  network.http.max-persistent-connections-per-proxy = 8 

There are good reasons for doing this, but these are mostly due to protocol limitations, more so than any direct need.  We can do a lot to reduce the tendency to increase these values further (if that is indeed a worthy goal ;).

> will just fall-back to the current situation.

Indeed - it seems to be quite an elegant solution.

> is something that relates to the HyBi "long term" solution discussion.

I see no reason that pipelining can't be used, but that's a separable issue.

> > (Based on hints I'm getting, I suspect that this wont be any sort of
> revelation to others, but it was to me.  Maybe I'm just slow, but I've
> not seen anything written down to this effect so far.)
> >
> > When long-polling, the current philosophy uses POST and inverts the
> standard request-response arrangement.  HTTP POST requests carry
> responses from the client to previously received HTTP responses.
> >
> > POST makes sense because it's the one request that lets you just send
> data to a resource.  All the other methods have all these semantics.
> >
> > Thus, there is the problem that there is a forced period of latency
> between when the server can send one unit of data and the next.  The
> server must wait for the previous unit to travel to the client, and
> then wait until it receives the next HTTP request from the client.
> >
> >
> > No Pipelines
> >
> > The problem here is that you can't pipeline POST.  A POST can't be
> sent until all previous requests are completed.
> >
> > Pipelining in this context is quite attractive.  If pipelining were
> viable, a client could send multiple requests, giving the server a
> number of opportunities to send data before exhausting the outstanding
> requests.  The number of outstanding requests could be tuned to ensure
> that plenty of requests were always available.
> >
> > That sounds nice, but you need idempotence to use pipelining and none
> of the idempotent methods can be used for this sort of interaction.
> You can't use idempotent methods to get this behaviour...or can you?
> >
> >
> > Reading about and implementing REST has led me to think of new ways
> to mangle old concepts into a shape that is more compatible with the
> philosophy.  That led me to a realization about how thinking about
> request-response inversion was the wrong way to think about the long-
> polling problem.
> >
> > You see, I was focussed too directly on the problem and the way it
> was formulated: how does a server request that a client send it some
> data?
> >
> > When you have a long-poll, the natural tendency is to think of the
> HTTP response as a "request" and some later HTTP request as a
> "response" to that "request".  That's a pretty well-worn paradigm and
> one that I'm familiar with, comfortable with.
> >
> > Here's how I'm going to implement long-polling instead.  There are a
> few details that might change, but the basic principle seems sound.
> >
> > My application starts off in the standard realm.  It happens to use a
> POST request, but that's largely immaterial.  If a client sends
> information that indicates that it's willing to respond to "requests"
> for information, then the response will indicate a URI where the client
> can long-poll.
> >
> > The resource that the client retrieves will just contain a document.
> This document will describe what information the server desires.  On
> first retrieval, the response will be immediate and will likely say
> that the server desires nothing.
> >
> > The client then begins long-polling to monitor for changes in
> resource state, using GET.  This will probably need the new header
> being proposed for request time limit, in combination with If-None-
> Match or If-Modified-Since.  The server waits to respond until either
> the time limit expires or the resource state changes.
> >
> > When the server wants the client to send it something, it changes the
> resource.  The resource then contains an indication of what information
> is desired, and where that information is desired, as a URI.  This
> might be a temporary URI, but that's largely irrelevant and up to the
> application.
> >
> > The client, upon receiving a response, processes the document, sees
> that some information is required and uses PUT to provide the
> information to the indicate URI.
> >
> > In this fashion, idempotent requests are used.  Pipelining can be
> used.  Other requests to the same server can continue, and if we're
> making requests to other requests to the same server, those are
> unaffected.
> >
> > The client goes back to monitoring the resource, which can revert to
> its original state, or remain as it was.  Only changes in the resource
> trigger the client to PUT.
> >
> > My guess is that this use case - or something very much like it - is
> what motivated the set of goals in draft-loreto-http-bidirectional.
> It's just a shame that this hasn't been clearly elucidated to date.
> There's a leap: from the description of long-polling to the set of
> conclusions.
> >

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Jul 2009 13:38:30 +1000

Two quick comments  -

1) The real problem with POST is that many (not all) implementations  
ALWAYS use a new connection for POST; using an idle pconn is too  
dangerous, precisely because you can't retry a POST request, so it  
isn't done (esp. in proxies).

2) Pipelining isn't practically useable on the internet, except in  
very well-defined use cases; many implementations don't support it.  
This can change, of course, esp. if we define a case, but it's an  
uphill battle; I wouldn't rely upon it.

Cheers,

On 08/07/2009, at 1:40 PM, Thomson, Martin wrote:

> revelation to others, but it was to me.  Maybe I'm just slow, but  
> I've not seen anything written down to this effect so far.)
>
> POST makes sense because it's the one request that lets you just  
> send data to a resource.  All the other methods have all these  
> semantics.
>
> number of opportunities to send data before exhausting the  
> outstanding requests.  The number of outstanding requests could be  
> tuned to ensure that plenty of requests were always available.
>
> That sounds nice, but you need idempotence to use pipelining and  
> none of the idempotent methods can be used for this sort of  
> interaction.  You can't use idempotent methods to get this  
> behaviour...or can you?
>
>
> "response" to that "request".  That's a pretty well-worn paradigm  
> and one that I'm familiar with, comfortable with.
>
> My application starts off in the standard realm.  It happens to use  
> a POST request, but that's largely immaterial.  If a client sends  
> information that indicates that it's willing to respond to  
> "requests" for information, then the response will indicate a URI  
> where the client can long-poll.
>
> The resource that the client retrieves will just contain a  
> document.  This document will describe what information the server  
> desires.  On first retrieval, the response will be immediate and  
> will likely say that the server desires nothing.
>
> Match or If-Modified-Since.  The server waits to respond until  
> either the time limit expires or the resource state changes.
>
> When the server wants the client to send it something, it changes  
> the resource.  The resource then contains an indication of what  
> information is desired, and where that information is desired, as a  
> URI.  This might be a temporary URI, but that's largely irrelevant  
> and up to the application.
>
> its original state, or remain as it was.  Only changes in the  
> resource trigger the client to PUT.
>

--
Mark Nottingham     http://www.mnot.net/

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 16 Jul 2009 15:56:18 +0100

> seen it pipelined behind 1, which is kind of disastrous.

What happens if the server sends "Connection: Close" in a long-polling
(or streaming) response, without delay, the delay coming after headers
and before data.

I'd naively expect that to tell all browsers that there's no point
pipelining a further request, and that any request already pipelined
will definitely be discarded, so should be reissued immediately on
another connection.  So I'd naively expect all browsers to not reuse
the connection for anything else.

Does "Connection: close" from the server do the trick with all tested
clients?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 16 Jul 2009 16:50:35 +0100

Thomson, Martin wrote:
> The practice of connection specialization (one for long-polling, one
> for other requests) is a sensible paradigm.  We might like to publish
> down some recommendations on that front.

Interesting.  I have a different paradigm, which is to _alternate_
long-polling across connections, never using the same one.

There are a couple of reasons.

First, I've read some browsers will silently close an idle long-poll
without reliably informing Javascript, and anyway NATs often do that,
so you have to have a keepalive client process starting a new
long-poll periodically.

To avoid a pause when server messages are available at the same time
as client starts a new long-poll, the client sends a "new long-poll"
request on the connection which is _not_ currently being used for
long-poll.  That request becomes the new long-poll, and the server
responds on the old long-poll, ending it.  Any pending messages from
the server at that time will be sent on the old long-poll.

This way, control is transferred from one long-poll to another without
delaying server messages.

Second, to make sure that clients don't pipeline GETs behind long-poll
requests, the server sends "Connection: close" on a long-poll
response, telling the client not to pipeline anything, and that any
GETs it did send will be dropped so it should resend them on another
connection.

Together, these mean that I don't re-use long-polling connections, and
the long-poll jumps from one connection to the next, keeping it
reliable over NATs and browsers without reconnection delays.  At
least, that's the theory.  I haven't tested enough browsers.

It's not friendly to the network to keep closing connections, but
we're all used to it by now.  I can make it behave a little bit nicer
if I have a dedicated pair of subdomains to carry long-polling
requests only, or if it's acceptable to use ports other than 80. Then
the "Connection: close" isn't required and connections can be
persistently reused.  A pair is used to make sure they alternate.
However I'm not sure if this method is entirely reliable over NATs.

Finally, there's another technique I've been thinking about if I want
the timing performance of streaming but the implementation reliability
of long-polling.  In other words, if I want to remove the round-trip
delay between receiving a batch of messages from the server, sending a
new long-polling request immediately after, and getting the next batch
of messages.  I'll keep more than one outstanding long-poll at the
same time, with the server instructed to end each long-poll after an
idle timeout, and use the alternation method across two subdomains or
ports, or more than two if an application benefits from it.

-- Jamie

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 16 Jul 2009 10:05:22 -0600

> delaying server messages.

This is essentially what we've done on the XMPP network since 2004 using
the BOSH technique:

http://xmpp.org/extensions/xep-0124.html

And yes, it works quite well.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 16 Jul 2009 18:26:27 -0500

Thanks for the clarification.  I missed that detail in my reading of BOSH.

I presume that the server sees the first open request, then another request arrives on the second connection.  This second request carries a "response" to an earlier "request".  In seeing the "response", the server determines that this now establishes a channel for further "requests", and closes the older channel.

This implies that the server needs to be able to correlate the two requests.  Presumably, this is done through parameters in the additional layer provided by BOSH.  Could this be done in the absence of correlating parameters?  Would a more generic mechanism require the addition of a parameter to achieve this?

--Martin

> From: Peter Saint-Andre [mailto:[email-masked]]
> Sent: Friday, 17 July 2009 2:05 AM
> To: Jamie Lokier
> Cc: Thomson, Martin; [email-masked]
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkpfT8IACgkQNL8k5A2w/vwNEwCg49zPoaD/hBuXhHu/vXIE6nbk
> BuEAnibmYNerrEI6HaVuVWFB+KwK10s2
> =P4ze
> -----END PGP SIGNATURE-----

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 21 Jul 2009 14:37:28 +1000

> Does "Connection: close" from the server do the trick with all tested

Jamie,

I believe Connect:close does prevent pipelining of subsequent requests.

This is probably a good technique for a site that is mostly idle and
there are long waits between events.  In this case, the extra cost of
a new connection will be pretty small.

If the site was busy with lots of events, then close could be
pretty expensive.

But an interesting idea none-the-less!

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 21 Jul 2009 13:11:43 +0100

> This implies that the server needs to be able to correlate the two requests.  Presumably, this is done through parameters in the additional layer provided by BOSH.  Could this be done in the absenc…

You do need a correlating parameter, but it can be as simple as a
unique id chosen for that browser instance, sent with the HTML or the
first AJAX response.

You can send it any way you like.  As part of the URL, as a query
paramter, as an HTTP header, as part of the POST body and response body.

Much like a cookie, except you probably want one per open window/tab,
rather than per browser.

Watch out for browsers who resurrect a page without reloading it when
visiting history.  You can end up with two instances of the same page
with the same correlating parameter if not careful.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 21 Jul 2009 13:07:51 +0100

> If the site was busy with lots of events, then close could be

On a busy site, what do you use instead?

The only alternative I've come up with is to use additional domains
reserved exclusively for non-overlapping long-poll requests.  Multiple
additional domains if overlapping them to reduce latency.

> But an interesting idea none-the-less!

It does have the advantage of being simple :-)

By the way, I've met quite a few people who say long-polling is
trivial, just a few lines of PHP etc.  I believe that means they've
never opened two tabs/windows to the same long-polling site, and never
been stuck behind a NAT with timeouts.  But maybe I'm just overcomplicating?

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 22 Jul 2009 08:30:11 +1000

> On a busy site, what do you use instead?

Cometd will not issue a long poll until the last has returned (or failed),
so it is vulnerable to the small extra latency between long polls.  But
then on a busy application, you rarely get to the end of the long poll
and events will cause it to return anyway - so there are small latency
"holes" after every server event.

There is nothing in the long poll requests to discourage persistent
connections, so persistent connections are used by most browsers.

To  avoid pipelining requests behind the long poll, cometd is very strict
about never issuing more that 2 requests.   So even if the client is
busy publishing messages, only a single publish request will be out
standing at once.

The issue here is that if there is non-cometd Ajax/image/css requests happening
at the same time as a cometd long poll and a cometd publish, then there
is some chance that a request is pipeline (or even just internally queued
on some browsers) behind the long poll.

Our recommendation is that static content is fetched from a different
domain.  Also cometd can be used in cross domain mode so it can
avoid the other ajax traffic.

We will shortly be implementing support for FF3.5 cross domain
support (https://developer.mozilla.org/En/HTTP_access_control) so
that cross domain comet will use the same XHR based transport rather
than a src element "hack".

> additional domains if overlapping them to reduce latency.

So far I've not worked with any applications that require such steps
to reduce latency.   Even some trading applications have been satisfied
that the extra latency long-polling introduces is not a problem for
it's traders (forex and shares).   Games such as chess and poker are
also fine with resulting latency profiles.

> been stuck behind a NAT with timeouts.  But maybe I'm just overcomplicating?

Long polling is essentially trivial.   A few lines of PHP will get you a
comet server running and the modern browsers have broken the 2 connection
limit, so multiple tabs is not initially seen as a problem.

The hard parts are indeed mostly edge cases - but ones that do
occur enough that non-trivial scalable sites do need to deal
with them.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 22 Jul 2009 00:46:51 +0100

> limit, so multiple tabs is not initially seen as a problem.

And NATs / stateful firewalls with 30 second timeouts (I see those
everywhere)?  I'm pretty sure you need to issue a new request every so
often to keep it alive.  Is that right?  How often do big sites find
they need it?

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 24 Jul 2009 13:43:03 +1000

> often to keep it alive.  Is that right?  How often do big sites find

For wide deployment I do recommend a long poll time of 30s.
But I have also seen many users with 300s timeouts and no noticeable
complaints from users.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 24 Jul 2009 16:27:13 +0100

> But I have also seen many users with 300s timeouts and no noticeable

That's interesting.  I've encountered two NATs on my travels with <60s
timeouts.  I'm pretty sure the local 3g phone network has it at 30s.

For VPNs I always use a timeout <30s for that reason.  I see OpenVPN
suggests keepalives every 15 seconds for users who travel around!

Perhaps users don't complain much when server-initiated messages don't
work, as long as user-initiated functionality (pushing buttons on the
client) still behaves properly.  For "chat"-type web apps, I guess
when the user types a message, updates will be received even if a long
poll was lost - if it's reasonably well engineered.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 27 Jul 2009 09:31:17 +1000

> poll was lost - if it's reasonably well engineered.

Also, the client may respond to a failed long poll (due to NAT timeout)
simply by sending a new long poll - so that may also somewhat mask the
issue.

cheers