Subject: Re: [hybi] moving forward on HTTP extensions

From: Mark Nottingham <[email-masked]>
Date: Fri, 17 Jul 2009 12:23:11 +1000

7. Possible improvements / extensions to XMLHttpRequest to expose  
connection-handling details (e.g., use of pconns, pipelining, etc.)?

On 18/06/2009, at 3:10 AM, Peter Saint-Andre wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Some potential future work items for bidirectional HTTP are  
> discussed in
> draft-loreto-http-bidirectional-00 as follows:
>
> 1. An HTTP extension for long polling, including request tracking,
>   duplication, and retry methods.
>
> 2. A method for monitoring the state of multiple resources.
>
> 3. A request header to determine timeouts.
>
> 4. A request header to determine the longest acceptable polling
>   interval.
>
> 5. Improved rendezvous logic between the user agent, a proxy /
>   connection manager, and the backend application server.
>
> 6. Improved addressing for the entities involved in bidirectional
>   HTTP, possibly including the use of URI templates.
>
> Sal and I just had a chat about this topic and would like to ask for
> feedback from the list members:
>
> - - Which work items do folks think would be most productive to  
> focus on?
>
> - - Who will be available in Stockholm to join a bar BOF about this?
>
> - - Would it be helpful to schedule a text or voice chat before IETF  
> 75?
>
> Thanks!
>
> Peter
>
> - --
> Peter Saint-Andre
> https://stpeter.im/
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAko5I6IACgkQNL8k5A2w/vwpjgCcC4T2PL4OpKxx70bNxN4pVAZK
> o2oAnjHbiQfyrEUosImI/oB+7jg9yree
> =+IB+
> -----END PGP SIGNATURE-----
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

--
Mark Nottingham     http://www.mnot.net/

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 17 Jul 2009 10:47:18 +0300

Hi Mark,

do you mean extension to the XMLHttpRequest methods and attributes ?
it would be a good idea, but it would be just a wish, there is nothing 
IETF can do on this side. isn't it?

/Sal

Mark Nottingham wrote:
>> Some potential future work items for bidirectional HTTP are discussed in
>> - - Which work items do folks think would be most productive to focus 
>> on?
>>
>> - - Would it be helpful to schedule a text or voice chat before IETF 75?
>>
>> Thanks!
>>
>> Peter
>>
>> - --
>> Peter Saint-Andre
>> https://stpeter.im/
>>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 17 Jul 2009 17:55:18 +1000

Sure, IETF may not be appropriate, but there are other folks who can  
do this, and the subject has been brought up before.

Cheers,

On 17/07/2009, at 5:47 PM, Salvatore Loreto wrote:

> it would be a good idea, but it would be just a wish, there is  
> nothing IETF can do on this side. isn't it?
>
> /Sal
>
> Mark Nottingham wrote:

---

From: Yves Lafon <[email-masked]>
Date: Fri, 17 Jul 2009 08:38:00 -0400 (EDT)

On Fri, 17 Jul 2009, Mark Nottingham wrote:

> Sure, IETF may not be appropriate, but there are other folks who can do this, 
> and the subject has been brought up before.
>

>> it would be a good idea, but it would be just a wish, there is nothing IETF 
>> can do on this side. isn't it?

See http://www.w3.org/TR/XMLHttpRequest/
and http://www.w3.org/2006/webapi/
The document is still a WD, so there is room for additions.
Cheers,

>>>> - - Which work items do folks think would be most productive to focus on?
>>>> 

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 21 Jul 2009 10:59:24 +1000

> 4. A request header to determine the longest acceptable polling

I've been thinking about these a bit.

One thing that could be done would be to have each hop add an  
indication of how long it's able to keep a connection open to the  
request; e.g.,

LongPoll-Timeout: 30, 60, 0

However, there are a few downsides here:
    * It relies on every hop adding its information, otherwise the  
numbers are misleading
    * It actually requires two numbers for each intermediary, since  
they can treat client-side and server-side connections independently  
(and often do, in terms of timeouts)
    * The information is only available to the server; the client only  
knows its own preferences, and intermediaries only know preferences in  
the direction of their clients.

I'm starting to think that a reasonable alternative would be to simply  
have the client insert a flag in the request (e.g., a header) that  
indicates that it's interested in long-polling, without timeout or  
interval information. This way, intermediary implementations as well  
as servers can use that information to adjust their use of resources  
accordingly.

Another approach would be to run some sort of discovery protocol  
before polling begins, so that the client (and potentially  
intermediaries) can figure out timeouts and appropriate polling  
intervals beforehand. I'm inclined to think that this would be too  
complex for practical use, however; it would require significant  
implementation overhead (especially in intermediaries) and wouldn't be  
robust (e.g., in the face of more complex proxy topologies).

Thoughts?

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 21 Jul 2009 14:43:42 +1000

> IETF can do on this side. isn't it?

Well if the IETF do define some headers related to long-poll requests,
the it would be sensible for browsers to update their XHR handling
to look for those headers and to perhaps adjust their pipelining
logic accordingly.    This is not exactly "exposing" pipelining, but
it could certainly influence.

cheers

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 21 Jul 2009 09:52:42 +0300

Mark Nottingham wrote:
> * It relies on every hop adding its information, otherwise the numbers 
> are misleading
> * It actually requires two numbers for each intermediary, since they 
> can treat client-side and server-side connections independently (and 
> often do, in terms of timeouts)
Hi Mark,

some thoughts...

a compromise could be, as I have written in a different thread, a header 
(or another kind of flag) that lets just the client indicate that the 
request is a "long-polling" one;
and at the same time it gives the possibility to the user to set (e.g. 
longer or shorter it) the timeout on the server and then use different 
values (tune it) for different applications;
it also gives the advantage to understand that the response to that 
request can arrive any time until the timeout expires (value indicate in 
the header).
If the server or the proxies don't understand the header the application 
will just fall-back to the current situation.

e.g. LongPoll-Timeout: 60

In this way you do not have a way to discover how long each hop is able 
to keep a connection open to the request, but we give the possibility to 
the client to set the time-out
(e.g. an application wants just wait for 60s for a request, where the 
standard timeout on the server side is 300s, or the other way around).
It could be also used to figure out timeouts and appropriate polling 
intervals: the first request is set to the largest timeout the 
application is willing to wait for, then if it receives
an empty successful answer before the interval it has set, it can be 
almost sure one of the proxies or the server on the path as a lower timeout.

> robust (e.g., in the face of more complex proxy topologies).
talking about a discovery protocol, while we were writing the 
"draft-loreto-http-bidirectional-00", we had some ideas about the 
possibility to design
a discovery protocol to understand which technique would be possible to 
use (e.g. long-polling, HTTP streaming): e.g. you are willing to use 
HTTP streaming
as bidirectional communication technique, but you want check in advance 
if the proxies along the path to the server will support it.
This could be useful for protocol like Bayeux that can use both the 
techniques.

/Sal
www.sloreto.com

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 21 Jul 2009 17:35:24 +1000

Sal, Mark,

How about negotiating a long poll using the Accept-Xxx
pattern already established and using a 406 NotAcceptable response.

So it could look something like this:

  BROWSER                  PROXY               SERVER

  GET /mycometserver HTTP/1.1
  Host: acme.com
  Accept-Longpoll: 60

                           HTTP/1.1 406 Not_Acceptable
                           Accept-Longpoll: 45

  GET /mycometserver HTTP/1.1
  Host: acme.com
  Accept-Longpoll: 45

                                               HTTP/1.1 406 Not_Acceptable
                                               Accept-Longpoll: 30

  GET /mycometserver HTTP/1.1
  Host: acme.com
  Accept-Longpoll: 30

  .
  . waiting ...
  .
                                               HTTP/1.1 200 OK
                                               Content-Type: text/json
                                               Content-Length: 1234

                                               { blahblahblah : ... }

Perhaps Accept-Longpoll could be called Accept-Latency to keep a more neutral
phrase?

cheers

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 23 Jul 2009 11:32:24 +1000

On 21/07/2009, at 4:52 PM, Salvatore Loreto wrote:
>
> some thoughts...
>
> a compromise could be, as I have written in a different thread, a  
> header (or another kind of flag) that lets just the client indicate  
> that the request is a "long-polling" one;

Yep.

> and at the same time it gives the possibility to the user to set  
> (e.g. longer or shorter it) the timeout on the server and then use  
> different values (tune it) for different applications;
> request can arrive any time until the timeout expires (value  
> indicate in the header).
> If the server or the proxies don't understand the header the  
> application will just fall-back to the current situation.
>

> In this way you do not have a way to discover how long each hop is  
> able to keep a connection open to the request, but we give the  
> possibility to the client to set the time-out
> (e.g. an application wants just wait for 60s for a request, where  
> the standard timeout on the server side is 300s, or the other way  
> around).

I hadn't considered this use (at least in a hybi context).

My concern is that if the client uses it to set a *lower* bound for  
timeouts (e.g., "I want a response in 1 second or I'm not  
interested"), network transit times become an issue, and there's the  
potential for a race condition.

The most reliable way for a client to tell the server that they're no  
longer interested in the outstanding response is to close the  
connection.

> almost sure one of the proxies or the server on the path as a lower  

Sure, but the next request may go through a different path, or a  
different heuristic may be applied by a proxy or the server in the  
future.

I'm not in-principle against advertising the timeout in requests, I'm  
just having difficultly finding a use case where it actually adds  
value. In the scenario above, the client can estimate (keeping in mind  
the caveats) the polling interval whether or not it's told the server  
that it wants an interval of 60s.

Besides which, I think the common case is going to be "keep the  
connection open as long as you're willing to", isn't it?

Can you illustrate a scenario where the server (or an intermediary)  
actually uses the timeout value in the request header?

>> implementation overhead (especially in intermediaries) and wouldn't  
>> be robust (e.g., in the face of more complex proxy topologies).
> talking about a discovery protocol, while we were writing the "draft- 
> loreto-http-bidirectional-00", we had some ideas about the  
> possibility to design
> a discovery protocol to understand which technique would be possible  
> to use (e.g. long-polling, HTTP streaming): e.g. you are willing to  
> use HTTP streaming
> as bidirectional communication technique, but you want check in  
> advance if the proxies along the path to the server will support it.

Right, but realise that connections in HTTP aren't end-to-end. I.e.,  
just because you connect to your proxy and make two subsequent  
requests on that connection, doesn't mean that the proxy uses the same  
connection to forward them -- or even the same path.

Cheers,