Subject: [hybi] SPDY protocol from google frame

From: Greg Wilkins <[email-masked]>
Date: Fri, 13 Nov 2009 07:05:16 +0900

FYI Google have announced some work they are doing on a HTTP improvement

  http://dev.chromium.org/spdy/spdy-whitepaper

It appears to capture many of the same ideas I was attempting to with BWTP - but
they are binary encoded and have working code :)

cheers

---

From: Lloyd Wood <[email-masked]>
Date: Thu, 12 Nov 2009 23:05:01 +0000

How does this compare to BEEP?

On 12 Nov 2009, at 22:05, Greg Wilkins wrote:

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/

<http://info.ee.surrey.ac.uk/Personal/L.Wood/><[email-masked]>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 13 Nov 2009 08:24:38 +0900

>From my quick look:

 + It's simpler than beep in channel management
 + It's addresses the header compactness issues
 + It is definitely bound to uber-http semantics, while beep is more general
   purpose (but could have a http binding defined).

But in concept, it is not too far apart.  It's using the same grab
bag of techniques (multiplexed channels with fragmentation and orderly
channel management) that Beep uses (and BWTP is advocating).

cheers

---

From: Ted Goddard <[email-masked]>
Date: Thu, 12 Nov 2009 17:25:25 -0700

It appears to be a binary framing protocol carrying
an HTTP subset.  The goals of the protocol match the
BWTP "manifesto", so hopefully the SPDY authors will
join this BOF.

I still like the idea of text-based framing and metadata
because it's easy to debug with telnet and tcpdump.
For instance, BWTP header persistence is still human
readable, would be very effective in terms of
compression, and could be retrofitted for httpbis;
whereas SPDY headers are compressed with gzip, resulting
in effective compression but no savings in header
encoding and decoding, and a random looking byte stream
unless you have a specialized debug tool.

It would be interesting to take a poll to see if people
have a binary or a text-based protocol in mind.

Ted.

---

From: "Infinity Linden (Meadhbh Hamrick)" <[email-masked]>
Date: Thu, 12 Nov 2009 17:51:21 -0800

+1 ted

i have to admit i'm a fan of text based protocols that use ASCII for
headers, allow arbitrary length fields, and whose elements are delimited by
ASCII characters. in the early days of my career, i was a fan of fixed
length fields with binary elements. to be sure, there are definitely places
where they are useful. but in my experience, when you're building something
new, the benefits of extensibility and debuggability outweigh the drawbacks
of dealing with larger, variable length messages that may require a little
bit more coding to parse.

<soapbox>

but i am kind of curious.. if there are people on this list that prefer
binary formats, i wonder, what are the characteristics you're trying to
preserve?

a. messages should be easy to parse?
b. messages and message elements should be as small as possible?
c. something else?

the reason i ask it that i've had experience with building IIOP and SS7
implementations and debugging HTTP is waay more fun than IIOP. and by "fun"
i mean that i'm less likely to drink heavily after a 24 hour debugging
session.

that being said... if i was building something for the telco ecosystem, and
i needed to respond to messages in a small, deterministic amount of time, i
would be interested in investigating where i would need to use fixed length
fields in a binary message. (but i wouldn't like it.)

it just seems to me that we're talking about bidirectional HTTP here; not
BEEP or IIOP or SS7. HTTP is generally NOT a binary protocol. doesn't it
make sense that a bidirectional version of it should also be ASCII/text
based? you're going to have developers and tool chains that are probably
going to assume that headers flowing across port 80 will be human
comprehensible. and while i have no problem using binary encoding when it's
needed... but... is it really needed here?

do we really have use cases where the < 1% savings you get on message size
by adding binary headers and protocol flow markers makes or breaks a
service?

</soapbox>

-cheers
-meadhbh

--
  infinity linden (aka meadhbh hamrick)  *  it's pronounced "maeve"
        http://wiki.secondlife.com/wiki/User:Infinity_Linden

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 12 Nov 2009 18:00:20 -0800

The point of SPDY was to make the web faster. Billions of users requesting
pages, want to speed it up. Of those billions of users, perhaps 100 care
whether it's binary or ASCII. Using ASCII just so that it's easier for 100
people to look at the traffic on the wire without any "specialized tools"
while slowing it down for billions of users seems like a strange tradeoff.

2009/11/12 Infinity Linden (Meadhbh Hamrick) <[email-masked]>

> On Thu, Nov 12, 2009 at 16:25, Ted Goddard <[email-masked]>wrote:
>
>>

---

From: "Infinity Linden (Meadhbh Hamrick)" <[email-masked]>
Date: Thu, 12 Nov 2009 18:02:18 -0800

well.. you're already scanning byte by byte for unicode support, aren't you?
i mean, at some level you are.

i would argue that you're trading flexibility at the "just above HTTP" layer
for efficiency in coding at the HTTP layer.

still.. it's a totally valid response to say you hate it.
--
  infinity linden (aka meadhbh hamrick)  *  it's pronounced "maeve"
        http://wiki.secondlife.com/wiki/User:Infinity_Linden

On Thu, Nov 12, 2009 at 17:57, Roberto Peon <[email-masked]> wrote:

> Having implemented the HTTP framing library that Google uses (yes I'm one
> of the SPDY guys), I can say that I absolutely hate dealing with non-length
> prefixed protocols.
> With HTTP, I don't know how much to allocate, I have to scan byte-by-byte
> for boundaries, and when dealing with text.. and worse,  you get into all
> sorts of fun ambiguities about what is allowed (e.g. is a newline allowed in
> quoted text?)
> Some of that was addressed with HTTPbis, but it is no fun whatsoever
> dealing with header-injection where unknown people can change your framing.
>
> For my sanity, security, and for significantly better performing proxies, I
> prefer length prefixing! :)
> -=R
>
>
> On Thu, Nov 12, 2009 at 5:51 PM, Infinity Linden (Meadhbh Hamrick) <
> [email-masked]> wrote:
>
>> +1 ted
>>
>> that being said... if i was building something for the telco ecosystem,
>> and i needed to respond to messages in a small, deterministic amount of
>> time, i would be interested in investigating where i would need to use fixed
>> length fields in a binary message. (but i wouldn't like it.)
>>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Nov 2009 17:57:23 -0800

Having implemented the HTTP framing library that Google uses (yes I'm one of
the SPDY guys), I can say that I absolutely hate dealing with non-length
prefixed protocols.
With HTTP, I don't know how much to allocate, I have to scan byte-by-byte
for boundaries, and when dealing with text.. and worse,  you get into all
sorts of fun ambiguities about what is allowed (e.g. is a newline allowed in
quoted text?)
Some of that was addressed with HTTPbis, but it is no fun whatsoever dealing
with header-injection where unknown people can change your framing.

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 13 Nov 2009 02:10:59 +0000

Infinity Linden (Meadhbh Hamrick) wrote:
>    it just seems to me that we're talking about bidirectional HTTP here;
>    not BEEP or IIOP or SS7.

WebSocket is clearly part of the HyBi discussion, and WebSocket's spec
editor clearly sees uses cases which are more like the IIOP and SS7
end of things than bidirectional HTTP.

That is, very small control messages between an application and it's
server.

SO I don't think we can just casually assume we're talking about
bidirectional HTTP.

>    HTTP is generally NOT a binary protocol.

The HTTP headers and framing aren't, but the bulk of what's sent over
a HTTP connection are the bodies, which are often binary or compressed text.

>    doesn't it make sense that a bidirectional version of it should also
>    be ASCII/text based?

Probably, but there is no consensus that bidirectional HTTP is an
ideal thing to aim for.

>    do we really have use cases where the < 1% savings you get on message
>    size by adding binary headers and protocol flow markers makes or
>    breaks a service?

For applications which involve a flow of small control messages, as
envisaged by at least some WebSocket developers, each message is only
a short string and HTTP headers would take more space than the
messages themselves.  That means >50%.

I guess that's why Google use compression in SPDY :-)

Personally, when debugging HTTP used for applications tunnelling
two-way messaging over HTTP, the text HTTP headers are only so useful.
They don't even contain all the addressing information in such
applications - most of what's interesting is encoded in the message
bodies, often a proprietary protocol layered over HTTP (that being
something we can hopefully improve by standarding useful HyBi).

With such applications, it's far more useful to have a good tracing
tool that can parse the protocol(s) in use and spit out an expanded
textual representation of the relevant parts.

That applies to binary protocols too.  If you're spending too much
time looking at binary network traces and you'd prefer that it was
text for ease of debugging, perhaps what's missing is a little tool to
annotate the traces, rather than a change of protocol.

The real strength, IMHO, of text-based protocols is their potential
ease of implementation and ease of extension.  If a text-based
protocol doesn't have those qualities, the text aspect is not so useful.

-- Jamie

---

From: "Infinity Linden (Meadhbh Hamrick)" <[email-masked]>
Date: Thu, 12 Nov 2009 18:16:29 -0800

ah. i see the disconnect here.

a lot of the software i've made was not "consumer oriented." rather
it's all B2B or *shudder* B2G. for people way out on the long tail,
the benefit of making a page .0003% faster is insignificant compared
to the pain involved in donning the positive pressure biohazard suit
to enter the biodetector and attach a disposable laptop to the local
area network only to discover that the right software wasn't loaded on
your system and no, they don't have the wireless network turned on
cause it's a secure facility and sure, get the intern to slide a
thumbdrive with the blackberry development tools on it through the
airlock.

so yes. this is a classic trade off. on one hand you have upstream
programmer convenience. on the other you have downstream integrator
convenience.

can we please specify a protocol that reduces the likelihood that
someone will be sitting in a class 3 containment facility waiting idly
for someone to fetch the debugging tools 'cause another integrator
couldn't figure out what version of the software was deployed on the
system inside a potentially contaminated mail processing facility?

-cheers
-meadhbh

2009/11/12 Ian Fette (イアンフェッティ) <[email-masked]>
>
> The point of SPDY was to make the web faster. Billions of users requesting pages, want to speed it up. Of those billions of users, perhaps 100 care whether it's binary or ASCII. Using ASCII just so that it's easier for 100 people to look at the traffic on the wire without any "specialized tools" while slowing it down for billions of users seems like a strange tradeoff.
>
>> i have to admit i'm a fan of text based protocols that use ASCII for headers, allow arbitrary length fields, and whose elements are delimited by ASCII characters. in the early days of my career, i was a fan of fixed length fields with binary elements. to be sure, there are definitely places where they are useful. but in my experience, when you're building something new, the benefits of extensibility and debuggability outweigh the drawbacks of dealing with larger, variable length messages that may require a little bit more coding to parse.
>> <soapbox>
>> but i am kind of curious.. if there are people on this list that prefer binary formats, i wonder, what are the characteristics you're trying to preserve?
>> the reason i ask it that i've had experience with building IIOP and SS7 implementations and debugging HTTP is waay more fun than IIOP. and by "fun" i mean that i'm less likely to drink heavily after a 24 hour debugging session.
>> that being said... if i was building something for the telco ecosystem, and i needed to respond to messages in a small, deterministic amount of time, i would be interested in investigating where i would need to use fixed length fields in a binary message. (but i wouldn't like it.)
>> it just seems to me that we're talking about bidirectional HTTP here; not BEEP or IIOP or SS7. HTTP is generally NOT a binary protocol. doesn't it make sense that a bidirectional version of it should also be ASCII/text based? you're going to have developers and tool chains that are probably going to assume that headers flowing across port 80 will be human comprehensible. and while i have no problem using binary encoding when it's needed... but... is it really needed here?
>> do we really have use cases where the < 1% savings you get on message size by adding binary headers and protocol flow markers makes or breaks a service?
>> </soapbox>
>> -cheers
>> -meadhbh
>>
>> --

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 13 Nov 2009 10:45:56 +0800

It was interesting to hear the IAB opinions on this in relation to internationalization.  The thesis was that the bulk of the web is video or some other efficiently encoded binary data and that text makes up a very small proportion of the data that is moved around.  The purported costs of more verbose encodings (in this case of Unicode) were actually considered negligible when compared to the other benefits.

Thus, I’d question whether these speed benefits are of any tangible benefit.  Sure, we minify javascript and there are still bandwidth-limited devices that would benefit from some speed improvement, but I don’t see a binary protocol improving this situation significantly.  And when I consider that some folks are getting better bandwidth on their mobiles than I get over my ADSL2+ connection at home, I wonder why we waste so much time on saving bits.

Can anyone point to the studies on impact of HTTP header sizes on throughput, latency and overall bandwidth use?

I’m also increasingly of the view that channels-based protocols aren’t necessarily the right way to solve head-of-line blocking problems.  There are other methods that do not demand the same overheads.  For instance, a simple request identifier can be used for request-response correlation.

I’m actually more excited about the prospect of getting HTTP over SCTP.

--Martin

From: [email-masked] [mailto:[email-masked]] On Behalf Of Ian Fette (????????)
Sent: Friday, 13 November 2009 11:00 AM
To: Infinity Linden (Meadhbh Hamrick)
Cc: [email-masked]
Subject: Re: [hybi] SPDY protocol from google frame

The point of SPDY was to make the web faster. Billions of users requesting pages, want to speed it up. Of those billions of users, perhaps 100 care whether it's binary or ASCII. Using ASCII just so that it's easier for 100 people to look at the traffic on the wire without any "specialized tools" while slowing it down for billions of users seems like a strange tradeoff.
2009/11/12 Infinity Linden (Meadhbh Hamrick) <[email-masked]<mailto:[email-masked]>>
+1 ted

<soapbox>

</soapbox>

-cheers
-meadhbh

On Thu, Nov 12, 2009 at 16:25, Ted Goddard <[email-masked]<mailto:[email-masked]>> wrote:

Ted.

From my quick look:

cheers

cheers
_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

<http://info.ee.surrey.ac.uk/Personal/L.Wood/><[email-masked]<mailto:[email-masked]>>

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 13 Nov 2009 10:48:42 +0800

This is good WS feedback :)

Having implemented the protocol with the worst framing in wide deployment (apologies XMPP folks ;), I can say that length-prefixed protocols are the nicest, cleanest and sanest of them all.

--Martin

From: [email-masked] [mailto:[email-masked]] On Behalf Of Roberto Peon
Sent: Friday, 13 November 2009 10:57 AM
To: Infinity Linden (Meadhbh Hamrick)
Cc: [email-masked]
Subject: Re: [hybi] SPDY protocol from google frame

Having implemented the HTTP framing library that Google uses (yes I'm one of the SPDY guys), I can say that I absolutely hate dealing with non-length prefixed protocols.
With HTTP, I don't know how much to allocate, I have to scan byte-by-byte for boundaries, and when dealing with text.. and worse,  you get into all sorts of fun ambiguities about what is allowed (e.g. is a newline allowed in quoted text?)
Some of that was addressed with HTTPbis, but it is no fun whatsoever dealing with header-injection where unknown people can change your framing.

For my sanity, security, and for significantly better performing proxies, I prefer length prefixing! :)
-=R

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 13 Nov 2009 11:53:17 +0900

Firstly it is really great that the SPDY guys are reading this
list!   I have some questions for you below after my own little
rant (haven't had a rant for a week or two... withdrawal)

<rant>
  As somebody who has implement both HTTP and several ss7 related
  telecommunications protocols - I can definitely say that I can see
  the benefits of both points of view.

  Unfortunately websocket has been designed (IMNSHO) to capture
  the worse aspects of both.

  It does not have the ease of debugging of a text protocol, yet it
  does not have the predictability or security from injection that
  you get with binary protocols that have fixed header sizes.

  Instead we have this strange mix of two types of framing - both
  sentinel based.  One looks for a sentinel byte to mark the end
  of the frame and the other looks for a sentinel bit to mark the
  end of the frame length.
</rant>

However, to be more constructive:  The IETF is very likely
to form a WG that is focused on standardizing websocket.

One of the things I would very much like that WG to achieve
is a rationalization of the framing in Websocket to
a fixed length length prefix (double 'length' is deliberate).

IF (big IF) this was achieved - it was my intention to
redo the BWTP proposal as something that could be carried
over websocket framing.

Would the google SPDY folks be interested in doing the same
with their protocol.   If so, I'd be very much inclined to
combine the efforts as the efforts are very similar?

I think that having something with the advance semantics of
HTTP that can be carried over a bidirectional transport
with reduced header overheads would be absolutely awesome
(although I'd advocate having channel header defaults
as in BWTP rather than scary gzipped headers - but minor point).

Since there is some convergence of efforts around websocket,
it would be a real pity if a) we could not make it's framing
acceptable to more efforts; b) other efforts duplicated the
work needed to get browser/intermediaries/servers to accept
a new transport.

cheers

Roberto Peon wrote:
> Having implemented the HTTP framing library that Google uses (yes I'm
> one of the SPDY guys), I can say that I absolutely hate dealing with
> non-length prefixed protocols.
> With HTTP, I don't know how much to allocate, I have to scan
> byte-by-byte for boundaries, and when dealing with text.. and worse,
>  you get into all sorts of fun ambiguities about what is allowed (e.g.
> is a newline allowed in quoted text?)

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Nov 2009 19:00:00 -0800

http://sites.google.com/a/chromium.org/dev/spdy/spdy-whitepaper
<http://sites.google.com/a/chromium.org/dev/spdy/spdy-whitepaper>
Header compression resulted in an ~88% reduction in the size of request
headers and an ~85% reduction in the size of response headers. On the
lower-bandwidth DSL link, in which the upload link is only 375 Kbps, request
header compression in particular, led to significant page load time
improvements for certain sites (i.e. those that issued large number of
resource requests). We found a reduction of 45 - 1142 ms in page load time
simply due to header compression.

<http://sites.google.com/a/chromium.org/dev/spdy/spdy-whitepaper>In other
words, HTTP headers are so verbose that we get a large latency reduction
simply by gzipping them. If we did a delimiter-based framing, we'd have to
ensure that the compressed data was not including that delimiter, or we'd
have to give up on header compression entirely. Neither of these options is
particularly interesting.

Hopefully, we can publish some more data about this later-- We've done a lot
of experimentation with dummynet+chrome+SPDY server on a lot of sites. The
server assumed that every resource came back in 30ms, which is on the fast
side, but that should have biased in favor of HTTP... We're trying to be as
data driven as possible about this.

-=R

On Thu, Nov 12, 2009 at 6:45 PM, Thomson, Martin
<[email-masked]>wrote:

>  It was interesting to hear the IAB opinions on this in relation to
> internationalization.  The thesis was that the bulk of the web is video or
> some other efficiently encoded binary data and that text makes up a very
> small proportion of the data that is moved around.  The purported costs of
> more verbose encodings (in this case of Unicode) were actually considered
> negligible when compared to the other benefits.
>
>
>
> Thus, I’d question whether these speed benefits are of any tangible
> benefit.  Sure, we minify javascript and there are still bandwidth-limited
> devices that would benefit from some speed improvement, but I don’t see a
> binary protocol improving this situation significantly.  And when I consider
> that some folks are getting better bandwidth on their mobiles than I get
> over my ADSL2+ connection at home, I wonder why we waste so much time on
> saving bits.
>
>
>
> Can anyone point to the studies on impact of HTTP header sizes on
> throughput, latency and overall bandwidth use?
>
>
>
> I’m also increasingly of the view that channels-based protocols aren’t
> necessarily the right way to solve head-of-line blocking problems.  There
> are other methods that do not demand the same overheads.  For instance, a
> simple request identifier can be used for request-response correlation.
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Ian Fette (????????)
> *Sent:* Friday, 13 November 2009 11:00 AM
> *To:* Infinity Linden (Meadhbh Hamrick)
> *Cc:* [email-masked]
>
> *Subject:* Re: [hybi] SPDY protocol from google frame
>
>
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Nov 2009 19:01:50 -0800

Oh, and I'm excited about SCTP too.. I just don't think that it solves all
of the problems that the browsers have (you need browser-indicated priority,
flow control, etc. SCTP isn't a silver bullet).

-=R

On Thu, Nov 12, 2009 at 7:00 PM, Roberto Peon <[email-masked]> wrote:

> Hopefully, we can publish some more data about this later-- We've done a
> lot of experimentation with dummynet+chrome+SPDY server on a lot of sites.
> The server assumed that every resource came back in 30ms, which is on the
> fast side, but that should have biased in favor of HTTP... We're trying to
> be as data driven as possible about this.
>
> -=R
>
>
> On Thu, Nov 12, 2009 at 6:45 PM, Thomson, Martin <
> [email-masked]> wrote:
>
>> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
>> Of *Ian Fette (????????)

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 13 Nov 2009 04:37:19 +0000

Thomson, Martin wrote:
>    I'm also increasingly of the view that channels-based protocols aren't
>    necessarily the right way to solve head-of-line blocking problems.
>    There are other methods that do not demand the same overheads.  For
>    instance, a simple request identifier can be used for request-response
>    correlation.

Fwiw, when I've talked about multiplexing on the HyBi list before,
I've always had in mind request-response correlation identifiers, not
opening long-lived channels.

In general, request-response is a bit limiting: Some applications just
want messages, without responses, because they correlate in other ways
unknown to the transport.  They may not be 1:1, as with pubsub
architectures for example.  So let's say request-response is good, but
it's better if the response is optional because redundant responses
are overhead.

I've advocated that multiplexing protocols should handle small and
large message sizes equally well, for the same reason as HTTP:
Separation of concerns, and robust good performance over a range of uses.

To send *large* messages without head-of-line blocking, you need a way
to split them (to allow others to go through), and a message identifier
to correlate the fragments.

One bit of baggage associated with channels is that they need to be
"setup" before using them.  BEEP does this, and it is way too high
overhead (latency).  It should not require a round trip before using
new channels: just start using them, allocated by the sender!  There
are receiver overflow issues, but that is best handled by flow control.

When you can use new channels immediately, that's technically *the
same thing* as request or message identifiers with ability to split
messages.  The difference is just terminology.

It's been argued that a basic transport protocol doesn't need to split
large messages, because applications (or shared middle layers) can do
it themselves, over a basic transport optimised for short messages
only.  The argument is if it can be done at a higher level, keep the
transport simple.

That probably is good protocol design, to keep the problems separate.

However, for *implementation* there are technical arguments in favour
of doing splitting and multiplexing together close to transport
framing: Network performance, latency, fairness among message sources,
and flow control to different sinks, are likely to perform better with
knowledge fed back from the transport.  Things like aligning with TCP
MSS boundaries to reduce average message latency.

(Perhaps some of the debates/conflict over this are due to it being
good to specify them as separate protocol layers, but good to
implement them in a slightly more unified way.)

I hope I've been able to explain why "channels" means different things
and can be a *low overhead, performance enhancing* concept which arise
naturally from robust, multiplexed message transport.

-- Jamie

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 13 Nov 2009 14:44:59 +0900

On 2009/11/13 11:45, Thomson, Martin wrote:
> It was interesting to hear the IAB opinions on this in relation to internationalization.
> The thesis was that the bulk of the web is video or some other efficiently encoded binary
 > data and that text makes up a very small proportion of the data that 
is moved around.
> The purported costs of more verbose encodings (in this case of Unicode) were actually
 > considered negligible when compared to the other benefits.

UTF-8 (the Unicode encoding favored by the IETF) isn't particularly 
verbose in general. In particular, for ASCII text, it's exactly as 
verbose as ASCII. It's slightly more verbose than legacy encodings for 
Chinese and Japanese, a bit more verbose than legacy encodings for 
languages such as Russian, Greek, Arabic, Hebrew,..., and definitely 
more verbose for Indian languages and quite a few other not so well 
known languages.

Also, for the bulk of text (mail or http bodies), conventions are well 
established for labeling encodings. For identifiers, which were the main 
topic of the IAB talk, the overall data volume is indeed very low 
compared to video. So for the IAB message, in a very simplified form, of 
"let's just use only UTF-8 for identifiers", data volume is indeed not a 
major concern.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Nov 2009 08:05:02 +0100

Ian Fette (イアンフェッティ) wrote:
> The point of SPDY was to make the web faster. Billions of users 
> requesting pages, want to speed it up. Of those billions of users, 
> perhaps 100 care whether it's binary or ASCII. Using ASCII just so that 
> it's easier for 100 people to look at the traffic on the wire without 
> any "specialized tools" while slowing it down for billions of users 
> seems like a strange tradeoff.
> ...

Hm. Are you sure about the "100"?

BR, Julian

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 12 Nov 2009 23:20:07 -0800

Ok, I will give you a few thousand. Either way, it's many orders of
magnitude fewer people as compared to the general internet population, who
could care less how easy it is to debug data on the wire, and just want
their webpage to load fast.

2009/11/12 Julian Reschke <[email-masked]>

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 13 Nov 2009 16:07:37 +0800

Ian Fette (イアンフェッティ) wrote:
> Ok, I will give you a few thousand. Either way, it's many orders of magnitude fewer people as compared to the general internet population, who could care less how easy it is to debug data on the wire, and just want their webpage to load fast.

They care when their sites don’t work as expected.  They care when it costs more to develop a site.

Of course, this matters less for those who have decent development budgets and dedicated test teams and the like.

--Martin

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 13 Nov 2009 17:15:31 +0900

It's not only "look at", it's also occasionally "just type it in". 
HTTP/1.0 is one line minimum, HTTP/1.1 is two lines minimum.

Another aspect is teaching: It's so much easier to teach a text-based 
protocol than a binary protocol, because there is much less of a 
difference between what you show and what's actually going on.

Regards,    Martin.

On 2009/11/13 16:05, Julian Reschke wrote:
>> perhaps 100 care whether it's binary or ASCII. Using ASCII just so
>> that it's easier for 100 people to look at the traffic on the wire
>> without any "specialized tools" while slowing it down for billions of
>> users seems like a strange tradeoff.
>> ...
>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 13 Nov 2009 17:38:30 +0900

Another point is that these numbers apply to deployed protocols like 
HTTP. For new protocols, the numbers are quite a bit different. And it's 
new protocols where the competition is going on, and where easy 
debugability counts. I'm sure this was the case for HTTP when it was 
'young'. It may or may not apply to SPDY or other protocols proposed today.

Regards,   Martin.

On 2009/11/13 16:20, Ian Fette (イアンフェッティ) wrote:
> 2009/11/12 Julian Reschke<[email-masked]>
>
> ------------------------------------------------------------------------
>

---

From: Ted Goddard <[email-masked]>
Date: Fri, 13 Nov 2009 09:27:31 -0700

OK, let's say there are 1000 people debugging network
traffic at any time -- this is a dangerous subset of
people to ignore, though, because they're not just the
users of the protocol, they're the ones developing
the servers, clients, and applications for the users.

Ted.

On 2009-11-13, at 12:20 AM, Ian Fette (イアンフェッティ) wrote:

> magnitude fewer people as compared to the general internet  
> population, who could care less how easy it is to debug data on the  
> wire, and just want their webpage to load fast.
>
> without any "specialized tools" while slowing it down for billions  
> of users seems like a strange tradeoff.
> ...
>

---

From: Ted Goddard <[email-masked]>
Date: Fri, 13 Nov 2009 09:47:32 -0700

On 2009-11-12, at 7:45 PM, Thomson, Martin wrote:

> I’m also increasingly of the view that channels-based protocols  
> aren’t necessarily the right way to solve head-of-line blocking  
> problems.  There are other methods that do not demand the same  
> overheads.  For instance, a simple request identifier can be used  
> for request-response correlation.

A request identifier (say, as an HTTP header) appears to
have a number of useful properties:

   - it can be optionally specified, so only clients
     supporting the feature need be aware of it
   - it can be ignored by the server, so server implementation
     is also optional
   - there is no latency for channel setup; uniqueness of the
     identifier can be a client responsibility, so there is
     no negotiation cost
   - the same identifier can be applied to chunked responses,
     allowing large responses to be interleaved

Would an existing HTTP proxy be confused by interleaved,
"identified" chunks?

> I’m actually more excited about the prospect of getting HTTP over  

What if google began providing their current services over
SCTP?  This will need a critical mass to get started.  (There
is also the small problem of missing SCTP client stacks.)

Ted.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 13 Nov 2009 08:49:49 -0800

In the case if SPDY,  we have a simple tool that reads the binary protocol
and outputs ASCII in a correlated structured format.

We'll also have a wireshark plugin soon for "defense in depth." Debugging
isn't significantly more difficult than it is for HTTP (where i have to look
at the output in a hexdump format to see if the delimeters are correct, etc.
The traffic generated by users is truly surprisingly weird sometimes)
And, thanks to the SHOULDS, you have no idea whatsoever how the next hop is
going to construe the exact bytes that you've received. Will it be able ti
handle a message with mixed \r\n and other eol  terminals??

Debugging HTTP on google servers is a royal pain. It isnt easy to debug just
because it is text...

Would we implement IP or tcp in ASCII?

On Nov 13, 2009 8:28 AM, "Ted Goddard" <[email-masked]> wrote:

Ted.

On 2009-11-13, at 12:20 AM, Ian Fette (イアンフェッティ) wrote:

> magnitude fewer people as comp...
>
> > _______________________________________________ > hybi mailing list >
> [email-masked] > https://www....
>
_______________________________________________ hybi mailing list
[email-masked] https://www.ietf....

---

From: Roberto Peon <[email-masked]>
Date: Fri, 13 Nov 2009 09:09:05 -0800

Speaking as the person who would be responsible for much of the work of
sticking sctp in for google, the greatest challenges are seemingly the home
routers/nat devices. Both server and client-side changes are trivial when
using the sockets API.

Note that our research Haas found that multiplexing itself may actually be
harmful... it is multiplexing plus client assigned priorities that seems to
be the winning combination.

If all my wishes came true tomorrow, Sctp would be widely deployed, would
give userspace applications easy access to a representation of bytes in
flight, cwnd, and other important connection properties...and would support
channel bases priorities.

The connection properties are interesting for varying server and client
behavior to better serve the user. I really want to send data to someone
using a satellite uplink fir data differently than a mobile user, or someone
with a nearly lossless 100 Mb link!

On Nov 13, 2009 8:47 AM, "Ted Goddard" <[email-masked]> wrote:

Ted.

---

From: Ted Goddard <[email-masked]>
Date: Fri, 13 Nov 2009 10:22:09 -0700

On 2009-11-13, at 9:49 AM, Roberto Peon wrote:

> In the case if SPDY,  we have a simple tool that reads the binary  
> protocol and outputs ASCII in a correlated structured format.
>
> We'll also have a wireshark plugin soon for "defense in depth."  
> Debugging isn't significantly more difficult than it is for HTTP  
> (where i have to look at the output in a hexdump format to see if  
> the delimeters are correct, etc. The traffic generated by users is  
> truly surprisingly weird sometimes)

There will be malformed SPDY traffic if multiple libraries
implement the protocol.  Will that be easy to debug with the
wireshark plugin?  Of course, debugging tools can be
developed for any binary protocol; the question is whether
you have that debugging tool available in the field when
you are faced with a critical bug.

> And, thanks to the SHOULDS, you have no idea whatsoever how the next  
> hop is going to construe the exact bytes that you've received. Will  
> it be able ti handle a message with mixed \r\n and other eol   
> terminals??

Perhaps the use of newlines in these cases should simply
be clarified?

> Debugging HTTP on google servers is a royal pain. It isnt easy to  
> debug just because it is text...

That's a controlled, high-volume environment, and I agree,
advanced tools are possible to install and necessary even
for debugging text protocols.

> Would we implement IP or tcp in ASCII?

Only for the packets that travel via paper printouts ...

This is a good point; on the other hand, TCP is well
established, inherently fixed length (now that everybody
is using ipv6 we won't run out of space again), and
lower in the stack.

It's all a question of tradeoffs -- we could certainly
be successful with either a text or a binary protocol.

Ted.

>> > Ok, I will give you a few thousand. Either way, it's many orders  
>> of magnitude fewer people as comp...
>>
>> > _______________________________________________ > hybi mailing  
>> list > [email-masked] > https://www....
>>
>> _______________________________________________ hybi mailing list [email-masked] 
>>  https://www.ietf....
>>
>

---

From: Ted Goddard <[email-masked]>
Date: Fri, 13 Nov 2009 10:41:07 -0700

On 2009-11-13, at 10:09 AM, Roberto Peon wrote:

> Speaking as the person who would be responsible for much of the work  
> of sticking sctp in for google, the greatest challenges are  
> seemingly the home routers/nat devices. Both server and client-side  
> changes are trivial when using the sockets API.

Start by tunnelling SCTP over UDP?  Home users would
be willing to open a UDP port on their router if it
improved their browsing experience.

> Note that our research Haas found that multiplexing itself may  
> actually be harmful... it is multiplexing plus client assigned  
> priorities that seems to be the winning combination.

A priority header could also be added alongside an
HTTP request identifier.

> If all my wishes came true tomorrow, Sctp would be widely deployed,  
> would give userspace applications easy access to a representation of  
> bytes in flight, cwnd, and other important connection  
> properties...and would support channel bases priorities.
>
> The connection properties are interesting for varying server and  
> client behavior to better serve the user. I really want to send data  
> to someone using a satellite uplink fir data differently than a  
> mobile user, or someone with a nearly lossless 100 Mb link!

That's true, the application should react differently
depending on the capacity.  Discovering the capacity
quickly would seem to require a combination of client
and server hints together with feedback from the
connection.  The application-level API for this might
not be simple -- it may be analogous to the transition
from single threaded to multi-threaded APIs.  Do you
have a version of this in a SPDY API?

Ted.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 13 Nov 2009 10:04:13 -0800

> improved their browsing experience.

It'd be great if this would be deployed :). UDP certainly seems like one of
the most likely ways to deploy SCTP.
Given how much pain various browsers (including mobile browsers) have caused
me, though, I remain pessimistic about users doing anything en-mass unless
things completely cease to function without it. They don't know the
technology that makes things work, and they don't want to. They just
(understandably) want it to work, and leave the rest to the
specialists/experts!

> A priority header could also be added alongside an

Certainly that could work too.. so long as you always get priorities with
multiplexing, you get some of the benefits that we've exhibited
experimentally with SPDY. Note that while multiplexing+priorities helps
everyone, it helps high-bandwidth clients more (as they're more likely to be
affected by head-of-line blocking).
Header text compression is the big win for users on slower links.

> have a version of this in a SPDY API?

Unfortunately no-- that is too low level. SPDY looks like HTTP for the
application users.
As far as accessing that info, we have modified linux kernels that allow for
userspace access to some of that information. We hope that we can
communicate some of this back to the client (as servers are FAR more likely
to change to a new kernel, etc. than the majority of users out there) and
thus both sides can profit from the information... but it is just an idea
right now. No implementation yet exists (we're too busy trying to open
source and deploy what we have already!).

-=R

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 14 Nov 2009 09:03:20 +1100

During these discussions- I've changed my mind from being a big
supporter of text protocols, to now leaning towards binary
framing.

But for me, the biggest argument in favour for a binary framed
protocol is security.

It is the forgiving nature of text protocols that makes
them easy to interoperate, but also easy to exploit. If
it wasn't for the easy injectable nature of HTTP, the
current SSL vulnerability would not be so exploitable.

There have been exploits on HTTP servers due to
injection into:
  response reason field
  cookie names/values
  headers

Having a protocol that every frame starts with a fix header
that includes a definite frame length - and applying the same
approach to meta-data fields, is certainly a good way to avoid
user data ending up looking like protocol framing.

As much as I appreciate the ability to telnet to port 80
and debug by hand - that is something that I would be prepared
to give up for a sane secure binary framing protocol.

Note that for anybody that thinks the "sane" part applies
to WS, the only thing I have to say to that is :

  0x80 0x80 0x80 0x80 0x80 0x80 0x80 0x80 0x80 ... forever

The "secure" part does not apply to any packet starting with 0x00

cheers

---

From: Mike Dierken <[email-masked]>
Date: Mon, 16 Nov 2009 19:40:17 -0800

Was the reduced page load time due to fewer TCP packets needing to be
exchanged (due to smaller HTTP request and response headers)?

Mike

On Thu, Nov 12, 2009 at 7:00 PM, Roberto Peon <[email-masked]> wrote:
> In other words, HTTP headers are so verbose that we get a large latency
> reduction simply by gzipping them. If we did a delimiter-based framing, we'd
> have to ensure that the compressed data was not including that delimiter, or
> we'd have to give up on header compression entirely. Neither of these
> options is particularly interesting.
> On Thu, Nov 12, 2009 at 6:45 PM, Thomson, Martin <[email-masked]>
> wrote:
>>

---

From: Roberto Peon <[email-masked]>
Date: Mon, 16 Nov 2009 19:54:52 -0800

That was only part of the savings, though it was certainly the most
significant savings for users with thin pipes.

Much of the improvement comes from the multiplexing+prioritization-- since
the browser can trust the server to fill the pipe with the things the
browser wants the most, it is free to issue requests as soon as it knows
that it needs them. Today, browsers will artificially induce a delay in
requests because it is vastly important that the HTML and CSS arrive
unimpeded.

Also a factor is how quickly we open up the cwnd. Since the protocol is
multiplexing, it has more bytes to stuff down the pipe (and sooner that it
would with HTTP thanks to prioritization+multiplexing), and so it opens cwnd
more quickly. This is simplifying a bit.. but you get the picture :)

-=R

On Mon, Nov 16, 2009 at 7:40 PM, Mike Dierken <[email-masked]> wrote:

> lot
> There
> requesting
> not
> size
> >> It appears to capture many of the same ideas I was attempting to with
> BWTP
> >> - but

---

From: Mike Dierken <[email-masked]>
Date: Mon, 16 Nov 2009 20:13:44 -0800

Sounds great. I really like this pragmatic approach of doing things to
improve user's experience that have a good chance of wide deployment.
I'm also very interested in the goal to allow the server to initiate
communication with the client. Would this be accomplished via
bi-directional streams within the TCP connection - but  the connection
itself would still need to be initiated by the client? Meaning you
haven't figured out a magical way to put an httpd on the client?

Mike

---

From: Roberto Peon <[email-masked]>
Date: Mon, 16 Nov 2009 20:35:00 -0800

Correct. The client always initiates the connection, but once a connection
is established, the server may initiate a stream.
-=R

On Mon, Nov 16, 2009 at 8:13 PM, Mike Dierken <[email-masked]> wrote:

> >> > reduction simply by gzipping them. If we did a delimiter-based
> framing,
> >> > we'd
> >> > side, but that should have biased in favor of HTTP... We're trying to
> be
> >> > as
> >> >> internationalization.  The thesis was that the bulk of the web is
> video
> >> >> or
> >> >> small proportion of the data that is moved around.  The purported
> costs
> >> >> of
> >> >> devices that would benefit from some speed improvement, but I don’t
> see
> >> >> a
> aren’t
> >> >> are other methods that do not demand the same overheads.  For
> instance,
> >> >> a
> >> >> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of
> >> >> Ian Fette (????????)
> >> >> whether it's binary or ASCII. Using ASCII just so that it's easier
> for
> >> >> 100
> >> >> 2009/11/12 Infinity Linden (Meadhbh Hamrick) <[email-masked]
> >
> >> >>
> >> >> +1 ted
> >> >>
> >> >>
> >> >>
> >> >> time, i would be interested in investigating where i would need to
> use
> >> >> fixed
> >> >> comprehensible. and while i have no problem using binary encoding
> when
> >> >> it's

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 17 Nov 2009 06:41:19 +0000

Roberto Peon wrote:
>    Much of the improvement comes from the multiplexing+prioritization--
>    since the browser can trust the server to fill the pipe with the
>    things the browser wants the most, it is free to issue requests as
>    soon as it knows that it needs them. Today, browsers will artificially
>    induce a delay in requests because it is vastly important that the
>    HTML and CSS arrive unimpeded.
> 
>    Also a factor is how quickly we open up the cwnd. Since the protocol
>    is multiplexing, it has more bytes to stuff down the pipe (and sooner
>    that it would with HTTP thanks to prioritization+multiplexing), and so
>    it opens cwnd more quickly. This is simplifying a bit.. but you get
>    the picture :)

It's nice to see this confirmed.  This is a major reason why I've been
advocating multiplexing on the hybi list - it's HTTP pipelining done
properly, as it should have been in the first place, so that it's (a)
usable and (b) performs well for browser applications.

(The other major reason for multiplexing is to give the browser
(especially) and proxies/servers freedom to combine or separate
connections - for exactly the kind of efficiencies you've described,
when multiple pages and multiple separately-authored things per page
are involved.

It's very interesting that you found it's important for the server to
prioritise response order, and that not doing so risks being slower
than some browser heuristics with HTTP.

-- Jamie

---

From: Larry Masinter <[email-masked]>
Date: Tue, 17 Nov 2009 08:58:15 -0800

> usable and (b) performs well for browser applications.

HTTP pipelining was added to HTTP/1.1 with more stringent backward
compatibility constraints, and at a time where there was still some
hope for HTTP-NG, viz:

http://www.w3.org/Protocols/HTTP-NG/

I suppose this is another one of those bits of history which we are
bound to repeat, hopefully more successfully this time.

If we weren't so busy trashing XHTML, http://www.w3.org/XML/EXI/ would
also be useful in making the web more efficient.

Larry
--
http://larry.masinter.net

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 18 Nov 2009 18:23:45 +0800

The 11% speedup is probably more indicative of real world performance.  I note that 1% packet loss is quite high for the sorts of connections simulated in the tests.  TCP doesn't like those sorts of loss rates, especially for larger payloads.

> -----Original Message-----
> From: Mike Dierken [mailto:[email-masked]]
> Sent: Tuesday, 17 November 2009 12:40 PM
> To: Roberto Peon
> Cc: Thomson, Martin; [email-masked]

---

From: Roberto Peon <[email-masked]>
Date: Wed, 18 Nov 2009 07:59:51 -0800

All we know that, at Google, we see TCP retransmits which are seemingly
indicative of a 1% packet loss rate.

One of the things that we're trying to convince people to do is a
internet-wide study of packet loss. No-one really has a good model for
it. No-one really understands it beyond their little corner of the 'net..
and that isn't good enough. For instance, when packet loss occurs, is it
truly random? Or is it "correlated" (e.g. if you had 6 tcp connections to
the same host would they likely all drop packets at the same time). We
suspect that it is correlated.. but have no proof.

In any case, we'll start real world testing soon-- in the next few weeks
(certainly before the end of Q4) we'll have a Chromium dev-channel (i.e.
pre-beta) release out as well as some public facing machines at Google.
Thus, you'll be able to test against Google soon. Unfortunately, it will
likely only be Google for the immediate future :/, though I hope that others
will join in the experiment soon.

-=R

On Wed, Nov 18, 2009 at 2:23 AM, Thomson, Martin
<[email-masked]>wrote:

> The 11% speedup is probably more indicative of real world performance.  I
> note that 1% packet loss is quite high for the sorts of connections
> simulated in the tests.  TCP doesn't like those sorts of loss rates,
> especially for larger payloads.
>

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 18 Nov 2009 17:42:30 +0000

Roberto Peon wrote:
> 
>    All we know that, at Google, we see TCP retransmits which are
>    seemingly indicative of a 1% packet loss rate.
> 
>    it. No-one really understands it beyond their little corner of the
>    'net.. and that isn't good enough. For instance, when packet loss
>    occurs, is it truly random? Or is it "correlated" (e.g. if you had 6
>    tcp connections to the same host would they likely all drop packets at
>    the same time). We suspect that it is correlated.. but have no proof.
>    In any case, we'll start real world testing soon-- in the next few
>    weeks (certainly before the end of Q4) we'll have a Chromium
>    dev-channel (i.e. pre-beta) release out as well as some public facing
>    machines at Google. Thus, you'll be able to test against Google soon.
>    Unfortunately, it will likely only be Google for the immediate future
>    :/, though I hope that others will join in the experiment soon.

The places where I've seen the most "might be random" packet loss have
been over mobile IP, getting up to about 5% loss while still being
(barely) usable for web access and SSH sessions.  So don't forget to
run Chromium dev-channel on your phones for testing :-)

Other than that, it's been mostly links or routers going down then
coming up again 10 seconds later, which is of course highly correlated
among multiple connections using the link/route at the time :-)

10 years ago it was (for me) 56k modems over poor quality lines,
behaving with a similar loss and latency profile as what I see for
mobile IP now, but with a much lower ceiling on throughput of course
(56kbps vs. >2Mbps).

>    <[1][email-masked]> wrote:
> 
>      The 11% speedup is probably more indicative of real world
>      performance.  I note that 1% packet loss is quite high for the
>      sorts of connections simulated in the tests.  TCP doesn't like
>      those sorts of loss rates, especially for larger payloads.

I regularly see higher loss percentages over a link which is still
working.  On my experience of mobile IP, up to 5% is not uncommon.
They tend to come mixed between vaguely random-looking losses at <2%
loss, to short bursts of loss averaging over time at up to 5%.  (At
higher percentages it's obvious the link is faulty and isn't usable,
but at 5% it continues to be usable, although annoyingly slow,
perhaps due to TCP's behaviour at such high loss rates.)

When loss becomes noticably bursty, I notice that ping responses
following a burst of loss are returned in a burst, with the first
having a high latency - about 5 seconds - and dropping, as if the
responses (or maybe requests) had all been queued up behind a
temporary link blockage of a few seconds, and then sent in rapid
succession.  But there is always some loss when that happens too, it's
not just queuing.

The fact that I see >1% regularly and Martin Thomson does not, just
confirms what Roberto says about loss measurements being confined to
each person's corner of the net.

An important related issue is latency, and latency variability.  I
noticed in the discussions about WebSocket that some people seem to
assume that round-trip latency from clients to web servers is not
significant enough to optimise for, but in my experience it depends
greatly on the type of link being used.  I've measured round-trip
latency over links where I'm having a useful web browsing experience,
ranging from <0.1 milliseconds up to >50 seconds.

-- Jamie

---

From: "Kensaku KOMATSU" <[email-masked]>
Date: Thu, 19 Nov 2009 16:59:50 +0900

Roberto Peon wrote:
> (certainly before the end of Q4) we'll have a Chromium dev-channel 
> (i.e. pre-> beta) release out as well as some public facing machines at
Google. 
> Thus, you'll be able to test against Google soon. Unfortunately, it will
likely
> only be Google for the immediate future :/, though I hope that others will
join in the experiment soon.

I'm looking forward to :)

Will "Server tools: coming soon!" also be available? 
(Announced in http://dev.chromium.org/spdy )

-- Kensaku KOMATSU