Subject: [hybi] I-D Action: draft-ietf-hybi-permessage-compression-13.txt

From: [email-masked]
Date: Mon, 07 Oct 2013 20:43:44 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.

	Title           : Compression Extensions for WebSocket
	Author(s)       : Takeshi Yoshino
	Filename        : draft-ietf-hybi-permessage-compression-13.txt
	Pages           : 29
	Date            : 2013-10-07

Abstract:
   This document specifies a framework for creating WebSocket extensions
   that add compression functionality to the WebSocket Protocol.  An
   extension based on this framework compresses the payload data portion
   of non-control WebSocket messages on a per-message basis using
   parameters negotiated during the opening handshake.  This framework
   provides a general method to apply a compression algorithm to the
   contents of WebSocket messages.  For each compression algorithm, an
   extension is defined by specifying parameter negotiation and
   compression algorithm in detail.  This document also specifies one
   specific compression extension using the DEFLATE algorithm.

   Please send feedback to the [email-masked] mailing list.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-hybi-permessage-compression

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-13

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-hybi-permessage-compression-13

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 8 Oct 2013 12:47:35 +0900

- explained why prefixes like s2c_ and c2s_ are needed
- added short note about adjusting ZLIB/Adler-32 field

On Tue, Oct 8, 2013 at 12:43 PM, <[email-masked]> wrote:

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Joakim Erdfelt <[email-masked]>
Date: Tue, 8 Oct 2013 20:14:22 -0700

Jetty's Commentary on draft-ietf-hybi-permessage-compression-13.txt

----------------------

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3

 > good
 > +1 for the removal of the PPP text

   2.  Conformance Requirements and Terminology . . . . . . . . . . .  4

 > standard language, good

   3.  Complementary Terminology  . . . . . . . . . . . . . . . . . .  5

 > sufficient, would providing links to RFC6455 sections be worthwhile?
 > example:

 - Non-control message means a message consists of non-control frames.
 + Non-control message means a message that consists of non-control
   frames (e.g., data frames) as defined in [Section 5.6] of [RFC6455]

 - Message payload (or payload of a message) means concatenation of the
 - payload data portion of all frames consisting the message.
 + Message payload (or payload of a message) means concatenation of the
 + payload data portion of all data frames consisting of a single
 + message, as defined in [Section 6] of [RFC6455]

 > no commentary on section for the paragraphs about Extension behavior
 > as the reference to [Section 9] of [RFC6455] is handled in the next
 > section of this doc

   4.  WebSocket Per-message Compression Extension  . . . . . . . . .  6

 > good

   5.  Extension Negotiation  . . . . . . . . . . . . . . . . . . . .  7

 > the section on "Agreed parameters" is confusing

   "Agreed parameters" MUST represent all parameters that can determine
   behavior of both the server and the client so that it's unnecessary
   to identify which offer has been accepted.

 > what is this trying to say?
 > starts out by saying that a new key phrase "agreed parameters" is
 > a mandatory representation of all parameters that can determine
 > the behavior of incoming vs outgoing frames.  Then it adds on the
 > confusing part about about the parameters being unnessary to use
 > to identify which offer has been accepted.  That's counter to
 > how this extension works later in the same document, even the
 > example in same paragraph.

   For example, if a client
   sends an offer with a parameter "enable_compression" and another
   without the parameter, the server accepts the former by sending back
   an element including a parameter that acknowledges
   "enable_compression".  The name of the acknowledging parameter
   doesn't need to be the same as the offer.

 > to put this another way, like the example hints at.
 > the client makes an offer, identifying 2 sets of parameters.
 > how it wants to talk to server, and how it wants the server to
 > talk to it.
 > the server will use the stated client side configuration/behavior,
 > if it can, (otherwise it doesn't negotiate the extension), and
 > then make a determination of its own configuration/behavior,
 > based on hints provided in the client request extension /
 > negotiation, and respond to the extension negotiation with
 > how it is expecting the client communications and how it
 > will communicate to the client.  If the client doesn't like
 > it, then the client closes with close code 1010.

 > The server is ultimately in control of its own
 > communications and configuration (this is normal behavior for
 > web communications, even http)
 > in http land, for example, the client can POST with compression
 > and state that it accepts compression, but the server does not
 > have to honor that, as ultimately the server, the server developer,
 > the server operations folks, and the networking folks will
 > want to change behavior of the server communications to suit
 > the demands that they have for the server.

 > the rest of this is not very strict about this, but this section on
 > "Agreed parameters" appears to makes this strict, and counter to existing
 > behavior for web technologies.

 > We see the new language for s2c_ and c2s_, but still think these are
 > awful choices, and will point out why this language just adds to the
 > confusion later in the spec.

 > The added language about the directionality is useful.

 > Since all websocket communications are initiated by client to server
 > these should just be s/s2c_/server_/g and s/c2s_/client_/g

 > This is one of the first extension specs, and will likely be used
 > as an example for other extension specs, don't propagate bad behavior
 > this early in the process.  Be clear and easy to follow.

   If a server gives an invalid response, such as accepting a PMCE that
   the client did not offer, the client MUST _Fail the WebSocket
   Connection_.

 > should we mention close code 1010 here?

     5.1.  Negotiation Examples . . . . . . . . . . . . . . . . . . .  8

   o  Offer the permessage-foo with a parameter use_y which enables a
      feature y as first choice, and the permessage-foo without the
      use_y parameter as a fallback plan.

          permessage-foo; use_y, permessage-foo

 > suggest eliminating this entirely.
 > we believe this to be an improper interpretation of the extension
 > rules in RFC6455.
 > Basically, Extensions should not be aware of each other, but rather
 > have some rules that apply equally to all extensions in a consistent
 > behavior.
 > If we have extensions that need to be aware of each other in order
 > to know if they can run or not, then you put the burden of all
 > future extensions to be aware of all other extensions.
 > In the above example, this would be invalid an invalid offer.
 > as both tokens are using RSV1 bit, causing the server to either
 > not use the extension or fail the connection as there are 2
 > extensions both requesting RSV1 bit.

 > An interpretation of the rules outlined in RFC6455:
 >  - extension might use a RSV bit (extension spec declares which one)
 >  - extension might need to refragment the frames
 >  - extension will only work if frame boundaries are preserved
 >  - extension might use extension data section

 > With this knowledge, the base RFC6455 extension negotiation can
 > determine what is possible, and what is in conflict on the
 > extension offer line.

 > Lets say, for example, we have 3 fictious extensions along with
 > permessage-deflate
 >   a) The metadata extension
 >      - uses RSV1
 >      - uses extension-data
 >      - doesn't care about frame boundaries
 >      this tacks on some arbitrary metadata about the message on each
frame
 >   b) The gpgsig extension
 >      - uses RSV2
 >      - does not use extension-data
 >      - doesn't care about frame boundaries
 >      this does a gpg cryptographic signature of each the frame
 >   c) The interleaving extension
 >      - uses RSV3
 >      - uses extension data
 >      - doesn't care about frame boundaries on input
 >      - will adjust fragmentation to suit its needs
 >      this interleaves multiple messages at the same time
 >   d) The permessage-deflate
 >      - uses RSV1
 >      - does not use extension-data
 >      - doesn't care about frame boundaries on input
 >      - will adjust fragmentation to suit its needs
 >      this does what this spec is all about

 > If an offer arrives with all of these ..
 >   Sec-WebSocket-Extension: interleaving, metadata, gpgsig,
permessage-deflate
 >
 > What should the base RFC6455 do to negotiate this?
 >   1) one path would be to not respond with metadata or permessage-deflate
 >      as both declare RSV1 bit use, and that's a conflict.  This is the
 >      behavior outlined in RFC6455, but the reasons for conflict are lost
 >      as theres no mechanism to communicate this conflict.
 >   2) another path would be to fail the handshake with an HTTP response
code 400
 >      with details about the conflict.
 >   3) a first-to-file approach could also be used, and if there is a RSV1
 >      conflict, the first one in the offer to declare it is the one that
 >      will be used, and all subsequent RSV1 declarations will not be used.
 >   4) extension spec specific fallback behavior (as outlined in the
 >      current permessage-deflate spec)
 >      This is a variation to the first-to-file approach, but instead of
 >      first to file, its first to negotiate successfully, causing all
 >      downstream conflicting extensions to never be negotiated.
 >
 > You might be leaning towards #3 or #4, but wait!  Mux changes the rules
again.
 > If during mux you see an offer like this ...
 >   Sec-WebSocket-Extension: metadata, mux, permessage-deflate
 > Then the physical connection will be using metadata, and the
 > all of the virtual connection (channels) will be using
permessage-deflate.

 > Now, to go back to the example in the spec documentation.
 >    permessage-foo; use_y, permessage-foo

   6.  Framing  . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

 > good

     6.1.  Compression  . . . . . . . . . . . . . . . . . . . . . . . 10

   2.  If this PMCE is the last extension to process outgoing messages,
       build frame(s) by using the compressed data instead of the
       original data for the message payload, and setting the

 > The condition "if this PCME is the last extension to process
 > outgoing messages" is meaningless, here and further down in this
 > section.  The process is the same for all PMCEs regardless if it is
 > the first, middle, or last extension.
 >
 > Also, with the current spec, we would argue that it is impossible
 > to have multiple active PMCEs anyway (due to the RSV1 bit requirement)

     6.2.  Decompression  . . . . . . . . . . . . . . . . . . . . . . 11

 > Again, same commentary about the "PCME is the last extension"
 > condition being irrelevant

   7.  Intermediaries . . . . . . . . . . . . . . . . . . . . . . . . 13

 > good, but could use some commentary that if the intermediaries are
 > not aware of the opening handshake, then they should make no
 > modifications to the frames.  See [Section 5.4] of [RFC6455]
 > 3rd paragraph for similar text.

   8.  permessage-deflate extension . . . . . . . . . . . . . . . . . 14

 > OK, some suggestions on how to clean this section up.
 >
 > Introduce the 4 extension parameters first.
 >
 >   Use the language found elsewhere in the spec now about the asymmetric
 >   nature of the extension parameter configuration.
 >
 >   The following 2 extension parameters control the behavior of the
 >   deflate configuration for Client-to-Server messages:
 >   (We took this notation from the title of [Section 5.3] of [RFC-6455],
 >    including the capitalization)
 >
 >     o  "c2s_no_context_takeover"
 >     o  "c2s_max_window_bits"
 >
 >   The following 2 extension parameters control the behavior of the
 >   deflate configuration for Server-to-Client messages:
 >
 >     o  "s2c_no_context_takeover"
 >     o  "s2c_max_window_bits"
 >
 > Then describe how the Offer works. (essentially the language you
 > have now in the doc, with careful attention to not mix your intro
 > and then bullets)
 >    Request: desired client behavior + desired server behavior
 >    Response: acknowledge client behavior + server chosen behavior
 >
 > Example of confusing language:

   For an offer for this extension, the following 4 extension parameters
   are defined.  If needed to distinguish from ones for a response,
   these parameters are called with a prefix "client-to-server".

   o  "s2c_no_context_takeover"

   o  "c2s_no_context_takeover"

   o  "s2c_max_window_bits"

   o  "c2s_max_window_bits"

 > This section introduces you to "client-to-server", then lists
 > s2c_*, why?  Same for the next section, but in reverse.
 > Don't mix the introduction of the extension parameters and
 > the behavior of the request and response together.
 >
 > Finally, go into the MUST rules for Client and Server like you
 > currently have in the spec documentation.
 >
 > Include wording for Client MUST pointing back to close code
 > 1010 from [RC6455] as a recommended behavior for dealing with
 > bad extension negotiation.
 > Even for things like a server saying s2c_max_window_bits=1024

     8.1.  Method Parameters  . . . . . . . . . . . . . . . . . . . . 15
       8.1.1.  Context Takeover Control . . . . . . . . . . . . . . . 15

 > Include explicit language indicating default behavior for
 > context takeover.

 > When documenting one side, for example "s2c_", do not make references
 > to the opposite direction.  Its confusing, noisy, and adds no value.
 >
 > For example, Section 8.1.2.2. c2s_max_window_bits, start out talking
 > about the c2s_max_window_bits, then has suddenly talks about
 > "server-to-client parameter", mentioning behavior and server limits.
 > This is confusing.
 > Fix this, clarify it, eliminate the noise, or simplify it.
 >
 > Eg of simplification:
 >
 >  See [Section 8.1.2.1.] "s2c_max_window_bits" for server side
 >  equivalent
 >
 > We suspect that the use of "c2s" and "client-to-server" to
 > interchangably refer to either the "extension parameter token" itself
 > or the phase of the extension negotiation (request or response) is
 > the cause of the confusion.
 >
 > Either choose new terminology for the extension parameter token, or
 > choose new terminology for the phase of the extension negotiation
 > (request side or response side)
 >

       8.1.2.  Limiting the LZ77 sliding window size  . . . . . . . . 16

 > Include explicit language indicating default size for
 > LZ77 sliding window size.
 > RFC1979 seems to indicate 32KB as the maximum value, but no obvious
 > word on what is the default behavior, assuming no window size is
 > specifically declared.
 >
 > The ABNF for the c2s_max_window_bits indicates that a digit is required,
 > but the text indicates that no value is possible.
 > What does no-value mean?

       8.1.3.  Example  . . . . . . . . . . . . . . . . . . . . . . . 18

 > See earlier commentary about no value c2s_max_window_bits extension
 > parameter token.
 > See earlier commentary about extension negotiation for example #3
 > (fallback behavior written into this spec)

     8.2.  Message Payload Transformation . . . . . . . . . . . . . . 19

 > good
 > (one note, the BFINAL logic needs to be validated in current
 > implementations.  as BFINAL does not behave the same in pywebsocket,
 > autobahn, chrome, and safari).  We had to hack BFINAL to always be
 > 0 for outgoing frames from our server in order for existing
 > client implementations to work.

       8.2.1.  Compression  . . . . . . . . . . . . . . . . . . . . . 19

 > good

       8.2.2.  Decompression  . . . . . . . . . . . . . . . . . . . . 20

 > good

       8.2.3.  Examples . . . . . . . . . . . . . . . . . . . . . . . 21

 > Include example of multiple message behavior with no_context_takeover
 > configured.

     8.3.  Implementation Notes . . . . . . . . . . . . . . . . . . . 24

 > This section's updates are good.
 > The LZ77 sliding window size recommendation is ok, but where's the
 > default LZ77 sliding window size?

     8.4.  Intermediaries . . . . . . . . . . . . . . . . . . . . . . 24

 > Isn't this section identical to [Section 7] in this same spec?

 > We have no comments for the remaining sections

   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 25
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 26
     10.1. Registration of the "permessage-deflate" WebSocket
           Extension Name . . . . . . . . . . . . . . . . . . . . . . 26
     10.2. Registration of the "Per-message Compressed" WebSocket
           Framing Header Bit . . . . . . . . . . . . . . . . . . . . 26
   11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 27
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 28
     12.2. Informative References . . . . . . . . . . . . . . . . . . 28
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 29

--
Joakim Erdfelt <[email-masked]>
webtide.com <http://www.webtide.com/> - intalio.com/jetty
Expert advice, services and support from from the Jetty & CometD experts
eclipse.org/jetty - cometd.org

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 9 Oct 2013 16:40:30 +0900

Thank you for careful review. I'll address them.

Some questions.

>  > example in same paragraph.

How about

"Agreed parameters" MUST represent how the requests and hints in the
client's offer have been handled in addition to the server's requests and
hints on the client's behavior, so that the client can configure its
behavior without identifying which PMCE offer has been accepted.

>  > it, then the client closes with close code 1010.

Thanks for the suggestion. Made some rephrasing. Please take a look:

----

Though how to handle parameters in detail will be specified in the
specifications for each PMCEs, general negotiation flow will be like this:

A client makes an offer including parameters identifying the followings:
- Hints about how the client is planning to compress data
- Requests about how the server compresses data
- Limitation of the client's compression functionality

The peer server uses these parameters, makes a determination of its
behavior based on them if it can and wants to proceed with this PMCE
enabled, and responds to the client with parameters identifying the
followings:
- Requests about how the client compresses data
- How the server will compress data

The client uses these parameters from the server make a determination of
its behavior based on them if it can and wants to proceed.
Otherwise, the client starts closing handshake with close code 1010.

----

>  > these should just be s/s2c_/server_/g and s/c2s_/client_/g

Initially, I thought like this: Each client does both compression and
decompression. Their compression parameters can differ. So, it'd be better
prefixing them with explicit words about direction (or together with words
like _trasmitted or _outgoing. I.e. server_transmitted, client_transmitted,
server_outgoing, server_incoming).

But looking at them again, I see that it's not so unclear that they're
about compressing operation.

I'm fine with that renaming.

>  > have some rules that apply equally to all extensions in a consistent

Are you suggesting that we drop the feature of multiple configuration
offering?

>  > future extensions to be aware of all other extensions.

I agree that without some O(n)-like rule, it's very hard to resolve
conflict of extensions. In pywebsocket, we experienced that, yes.

>  > all of the virtual connection (channels) will be using

Yes, handling of mux is complex...

I thought that it's not realistic to have generalize rule but get consensus
on how to handle special extensions such as mux e.g. mux needs to be
resolved first.

Except for such special things, basically resolution should happen based on
combination of (3) and (4) ((4) only inside subgroup like a set of PMCEs
listed in sequence) and the client should be aware of that to decide the
order of offers properly.

>  >    permessage-foo; use_y, permessage-foo

As far as PMCEs are grouped together, it should be easy to resolve between
them and choose one.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 15 Oct 2013 23:00:33 +0900

Some addressed in -14. The rest will be addressed in -15.

>  > +1 for the removal of the PPP text

Yes. It was broken after rendering HTML.

>    frames (e.g., data frames) as defined in [Section 5.6] of [RFC6455]
done

>  + message, as defined in [Section 6] of [RFC6455]

s/consisting of/representing/ ?

done

>    5.  Extension Negotiation  . . . . . . . . . . . . . . . . . . . .  7

Used texts I showed in my last reply.

>  > should we mention close code 1010 here?

I'm not sure if 1010 is for this kind of failure. The text in RFC 6455
sounds like it's used to reject an opening handshake response that doesn't
include extension(s) the client considered to be mandatory.

There isn't any extension with concept of mandatory introduced yet.
permessage-deflate is not mandatory extension. We just want to tell the
server that it sent back an invalid extension response.

>  > to have multiple active PMCEs anyway (due to the RSV1 bit requirement)

[TBA]

>  > condition being irrelevant

[TBA]

>  > 3rd paragraph for similar text.

[TBA]

>  > Even for things like a server saying s2c_max_window_bits=1024

[TBA]

>  > (request side or response side)

[TBA]

>  > LZ77 sliding window size.

Added texts.

>  > but the text indicates that no value is possible.

It means that only capability is announced with no hint.

Added texts.

>  > client implementations to work.

Sorry. Could you please elaborate this?

>  > Include example of multiple message behavior with no_context_takeover

In "Sharing LZ77 Sliding Window" section, the second example is identical
to the first one. This is the example of multiple message behavior with
no_context_takeover.

>   This is the payload of the first message the client has sent.  If the
>   "agreed parameters" contain the "c2s_no_context_takeover" extension
>   parameter, the client compresses the payload of the next message into
>   the same bytes (if the client uses the same "BTYPE" value and
>   "BFINAL" value).  So, the payload of the second message will be:
>
>       0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00

Added concatenated examples.

>  > default LZ77 sliding window size?

This sentence in "Compression" section

> An endpoint MUST NOT use an LZ77 sliding window longer than 32,768 bytes
to compress messages to send.

and these sentences in "Decompression" section are intended to define
default LZ77 sliding window size.

> Otherwise, the client MUST use a 32,768 byte LZ77 sliding window to
decompress received messages.

> Otherwise, the server MUST use a 32,768 byte LZ77 sliding window to
decompress received messages.

>  > Isn't this section identical to [Section 7] in this same spec?

Yes. Section 7 is talking about interaction with intermediaries in general,
and this section is saying almost the same thing but using
permessage-deflate specific language for example.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 16 Oct 2013 18:13:28 +0900

Addressed the rest for -15.

>>  > to have multiple active PMCEs anyway (due to the RSV1 bit requirement)

This sentence corresponds to the next sentence with "otherwise". It
explains how to pass the result of compression to the next entity in the
chain. If it's another extension, the sentence starting with "Otherwise"
applies while if there's no more extension, we should follow the first
sentence to pass the result to the RFC 6455 send/receive algorithm.

>>  > condition being irrelevant

Ditto.

>>  > 3rd paragraph for similar text.

Done.

done

used Peter's texts.

> replied in another post.

done

>>  > to the opposite direction.  Its confusing, noisy, and adds no value.
done. removed "in the same ..." sentences

>>  > (request side or response side)

done i think...