Subject: Re: [hybi] summary of frame length options

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 03:32:44 -0400

I appreciate the effort put into this thread, and agree with most of what
was written. Important points follow.

> The framing is intended to be as in
> http://www.ietf.org/mail-archive/web/hybi/current/msg03371.html
> so the assumption is there are 3 reserved bits available in the header,
> and some of these proposals provide less (which may require extending
> the header to get the extensibility desired).

Incorrect.  The assumption is there are 4 reserved bits.

CMLP should _not_ be used.

The framing will _not_ require the CMLP bit, as explained here:

http://www.ietf.org/mail-archive/web/hybi/current/msg03685.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03694.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03525.html (Thomson,
Martin)

Discussion:
http://www.ietf.org/mail-archive/web/hybi/current/msg03583.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03551.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03553.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03567.html

> 3G wireless networks have a low power FACH mode which can only transmit
> from 128-256 bytes (depending on carrier settings), including the TCP/IP
> header
>
> Sample traces from Wave and GWT Quake indicate that once compression is
> enabled, more than 98% of all messages will fit in ~126 bytes.  GWT
> Quake payloads were highly compressible ...[snip]... likely to have
> highly compressible payloads, such as chat messages, tweets, JSON
> serialized objects, etc.

True, but:

1) FACH limits will likely increase in future.
2) App developers will design to maximum FACH limits (even as they increase)
3) Chat users often use photo sharing (Yahoo Messenger) and I for one will
be using WebSockets to send thumbnail jpg clips in a webchat to do a
simulated form of video chat (until HTML5 device is ready, I can build
these clips in Flash and send to JavaScript). I do plan on competing with
is social networking in asia. I have a domain coolpage to be re-deployed
for that. I expect other developers to emulate what I do.
4) jpg is already compressed.

My basic point is that the limits are going to be a moving target. That is
why I proposed the 15/63 as the optimal compromise as it is always 3 bytes
from 0-32766, whereas the others are 4 bytes above 125 or 255.
http://www.ietf.org/mail-archive/web/hybi/current/msg03639.html

BYTES  RANGE        OPTION      RESERVED BITS (including 'More')
----------------------------------------------------------------
2      0-125        7/16/63     4
4      126-65535    7/16/63     4
3      0-255        8/16/32/64  9
4      256-65535    8/16/32/64  9
3      0-32766      15/63       4

Also 15/63 has the lowest CPU load too and by an order-of-magnitude if you
are comparing the ratio of the execution time. Although CPU load may or
may not be a factor, if it costs us nothing to optimize for CPU load, then
why not be safe:

http://www.ietf.org/mail-archive/web/hybi/current/msg03677.html

I have explained in my prior posts the game theory risks for high
PROPORTIONAL cpu load. I am not talking about the nominal nanoseconds, I
am talking about the RATIO of comparative CPU loads. I know it seems
insignificant when compared to network load and other loads, but I
explained before some theoretical worries and I am just saying why not
keep it simple and safe.

There is one more point that keeps getting lost when ever a
resummarization is done. Others have already agreed that we must have a
maximum frame size negotiation on connection handshake. The maximum must
be a hard limit, and this is not a security risk as I explained to John (5
times):

http://www.ietf.org/mail-archive/web/hybi/current/msg03626.html

>>  If you do check, then you have the same conditions.  There are lots of
>> attacks that are present because differing levels of validation -- for
>> example, a typical HTTP Request Smuggling attack is to have two
>> content-length headers.  One entity uses one of them, and another uses the
>> other, which allows something that would have been filtered through to the
>> second.  This is a real problem, and you have consistently ignored it when
>> it has been brought up.  So, I think this spec should require that
>> everything be validated.
>
> That is nonsense, because what ever the intermediary passes through, that
> is what the next end will see.  You are conflating an HTTP issue which is
> unrelated.

Thus with 15/63-bit, if the receiver wants to optimize away the CPU load
of comparisons for the size field, then it can set the maximum frame size
to 32766, then it never has to check for the special 32767 value
(signaling an 8-byte size field). Whereas, if we choose the 7/16/63-bit,
then that optimization would be at a maximum size of 125. That is a very
low threshold. If a receiver was selfish in optimizing its internal load,
that could cause a sender to have to refragment a large message to 125
size frames, causing a huge increase in CPU load and network load
externally.

What we have to remember when we do engineering, is that the wild is like
real life, where nature will optimize according to game theory. If we
offer a capability, then it will probably be used, even if we do not
recommend it. Programmers are very clever at extracting every last ounce
of performance to impress their bosses on bench tests.

I am not saying that the above is likely, I am just saying that since
15/63-bit is optimal in every way, then why not choose it?

The only argument against 15/63-bit is that 98% of frames will be less
than 126 forever, and that FACH limits won't increase forever.  I don't
find that very re-assuring, do you?

Sorry this was all very clear in my mind 2+ days ago, I hope I have
succeeded this time to explain it clearly and concisely.

The hyperoptimized option is very clever.  Kudos to Brian. But given my
points above, I don't think optimizing for a very narrow range is wise.

It might be relevant to note that I have probably written about 100,000
lines of 8086, 68000, 80386, etc assembling code in my lifetime. I hand
optimized all the intricate grain, brush, opacity, etc models for Corel
Painter in assembly back in mid-1990s.

Please do not copy (Cc:) to me on any reply to list. I am unsubscribed.
Thank you.

---

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 08:49:27 -0400

[snip]

>> from 128-256 bytes (depending on carrier settings), including the TCP/IP

[snip]

> 4) jpg is already compressed.

Also it is possible to send multi-media and stay just below the FACH
limit, even as it increases in the (not very distant) future. For example,
can send the jpg quantitization tables once for all subsequent
thumbnail-size images, and/or data can be sent interleaved so that build
up the quality of the image over time. Fps rates are so slow here (0.5 fps
maybe avg), that nothing is gained from MPEG.

Can we declare an API to read the available FACH threshold in the browser?
Specifically what is remaining after TCP and any TLS overhead, for
outgoing frames. The client can communicate this to the server for
tailoring incoming frame sizes.

> BYTES  RANGE        OPTION      RESERVED BITS (including 'More')

[snip]

---

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 09:54:37 -0400

I assert that 7/16/63 may not be the most popular. The voting may be
skewed by having too many similar dilutive choices.

I think it may not be obvious to others, that the votes for 8/63, 8/16/63,
and 15/63 should be perhaps be combined, which would make it the most
popular choice (assuming the same people have not voted for both?):

https://www.google.com/moderator/#15/e=2809f&t=2809f.40

Since we no longer need the CMLP bit, then perhaps we don't need the extra
reserved bit that 8/63 consumes:

http://www.ietf.org/mail-archive/web/hybi/current/msg03707.html

Thus, perhaps those voting for 8/63 and 8/16/63, have the similar concern
that I do (why I proposed 15/63) about needing some margin above the 125
threshold of 7/16/63. One criticism I can make against 15/63, is that cost
of 4 bytes above 255 is not very significant (cost is linear since similar
to cost of 2 bytes at 127). So thus I could accept 8/16/63 as a reasonably
optimal resplacement for 15/63. I don't think 8/63 is as optimal, because
it jumps to 8 bytes cost at 256, but I could still accept it as superior
margin to 7/63 and 7/16/63.

I had favored 15/63, because I thought we needed the extra reserved bit.
But if we don't, then the 8/16/63 is fine by me and 8/63 is tolerable. CPU
cost was not my overriding concern, I was just saying that if 15/63 is
optimal in every other way of balance, then great to know it and 8/63 also
have lowest CPU cost.  I still prefer that 0-32767 base range. Just makes
me feel safer, for lousy byte cost and saves 1 byte for 125+ or 255+.

Hope that helps move closer to concensus.  If I am wrong, at least it was
worth presenting as a hypothesis of the concensus.

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Dave Cridland <[email-masked]>
Date: Thu, 26 Aug 2010 16:44:43 +0100

> tailoring incoming frame sizes.

A implementation-specific layer API? That would be a layer violation.  
;-)

Seriously, while the FACH threshold can be 128 octets up to 3k or  
higher, I've never found figures on what it's set to on any  
particular network, and from what I can tell, it's usually 128.  
Raising it has an impact on the performance of the network, since  
FACH uses a common channel on the cell, so there's an advantage in  
keeping it low.

Users do actively avoid applications and - most especially - phones  
with poor battery performance (although the seem inclined to ignore  
that for shininess), but there's no knowledge in the general public  
that the choice of network can potentially make a significant  
difference to battery life - as such, I don't see a real benefit to  
network operators to set usefully high limits.

This all said, I'm working heavily on hearsay on this - experts in  
this field should feel free to jump in and correct me.

My only care is that:

1) We use one or more fixed-width fields in network byte order.

2) If we use more than one, then include an option for 7 or 8 bits.  
(7 if this saves an octet).

3) There must exist at least one option which is 16 bits wide or  
more. I'd settles for 15 at a pinch.

I know that other people have suggested that a 63-bit length option  
is mandatory - I don't care if it is available or not.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: "Shelby Moore" <[email-masked]>
Date: Thu, 26 Aug 2010 15:57:50 -0400

> A implementation-specific layer API? That would be a layer violation.

Nice catch. :D
I wasn't thinking it be contained within the WS protocol layers we are
designing. I was proposing only it be added in the appropriate API of the
user agent. Would that be HTML5 <device>?

>From what you wrote below, it seems it might be helpful for the
application. Sounds like there will be advances in this area, both from a
user sophistication and the various parties to enable that understanding,
as we can contribute here.

[snip]