Subject: Re: [hybi] I-D Action: draft-ietf-hybi-permessage-compression-09.txt

From: Tobias Oberstein <[email-masked]>
Date: Sat, 1 Jun 2013 15:29:52 -0700

> Please post your opinions and experience (implemented, faced any difficulties, found issues, etc.).

"""
   A client MAY attach the "c2s_max_window_bits" extension parameter if
   the client can adjust LZ77 sliding window size based on the
   "c2s_max_window_bits" sent by the server.  This parameter has no
   value.
"""

Any reason there is no "c2s_no_context_takeover" parameter that allows the client to _announce_ it supports the feature?

Proposal (new text):

"""
   A client MAY attach the "c2s_no_context_takeover" extension parameter if
   the client can disable the takeover of context based on the
 "c2s_no_context_takeover" sent by the server.  This parameter has no
   value.
"""

A client that fully supports all features and requests none from the server would then offer

"permessage-deflate; c2s_max_window_bits; c2s_no_context_takeover"

and the server would accept the offer requesting no features by responding with

"permessage-deflate"

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 20:00:28 +0900

On Fri, May 31, 2013 at 7:51 PM, Tobias Oberstein <
[email-masked]> wrote:

> Hi Takeshi,
> We are implementing the extension now in Autobahn - testing against latest
> Chrome Canary which provides the extension.
>
> Does Chrome have any (user accessible) switch/options to
>
> 1. force automatic message fragmentation
>

We don't have it yet. Unless it's required to be implemented, e.g. for mux
or partial send API in the future, we won't.

> 2. set compress window sizes
>

Not planned. Chrome always accept data compressed w/ 32k window.

> 3. context takeover

No.

>
> ?
>
> Rgd 1.: I am asking since Autobahn provides APIs for message, frame and
> streaming processing and we want to make sure permessage-deflate works for
> those also.
>

We could help you by extending pywebsocket's echo_client.py.

Sorry that Chrome won't.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 20:17:36 +0900

On Sat, Jun 1, 2013 at 5:54 AM, Tobias Oberstein <
[email-masked]> wrote:

> 1) Some first testing ..
> http://code.google.com/p/chromium/issues/detail?id=245732
> http://code.google.com/p/chromium/issues/detail?id=245719
>
>
Thanks so much.

> 2)
> If client offers multiple PMCE choices to the server by including multiple
> elements in the "Sec-WebSocket-Extensions" header and the server supports
> more than one of the PMCEs offered, is there any preference which PMCE the
> server SHOULD choose? E.g. the first PMCE in the header that is supported
> by the server?
>
> The description of the examples in "5.1. Negotiation Examples" seem to
> suggest that the _ordering_ of alternative PMCE offered designates
> _preference_.
>
>
Yes. Order means preference. The server may choose any even if it supports
all of them.

Do you think we need any text to make it clear?

> 3)
> """
>    A server MUST decline a "permessage-deflate" offer if any of the
>    following conditions is met:
>
>    o  The offer has any extension parameter not defined for use in an
>       offer.
>
>    o  The offer has any extension parameter with an invalid value.
>
>    o  The offer has multiple extension parameters with the same name.
>
>    o  The server doesn't support the offered configuration.
> """
>
> So if multiple permessage-deflate PMCEs are offered by the client, the
> server silently skips any invalid until the first valid or none if there is
> no single valid one?
>
> The server does NOT fail the WS connection upon encountering the first
> invalid PMCE in the list of offers?
>

Yes. As far as there's no WebSocket-core-protocol-level grammar error
(extension-param ABNF in RFC 6455), the server doesn't have to fail the
connection.

>
> 4)
> """
>   A client MUST _Fail the WebSocket Connection_ if the server accepted
>    a "permessage-deflate" offer with a response meeting any of the
>    following condition:
>
>    o  The response has any extension parameter not defined for use in a
>       response.
>
>    o  The response has any extension parameter with an invalid value.
>
>    o  The response has multiple extension parameters with the same name.
>
>    o  The client doesn't support the configuration the response
>       represents.
> """
>
> This list does not include: "The response corresponds to a PMCE
> configuration not offered by the client originally".
> Is this on purpose?
> (A client may support a configuration which it did not offer .. for
> whatever reasons)
>
>
Yes. To allow multiple offers but make it unnecessary to identify accepted
request (e.g. we need to attach request ID or something to each extension
entry to do that), I made permessage-deflate responses self-contained (both
client side parameters and server side parameters are listed in a response).

Each extension has sole discretion to consider what response to be
invalid/unsupported.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 20:35:20 +0900

On Sun, Jun 2, 2013 at 7:29 AM, Tobias Oberstein <
[email-masked]> wrote:

> Any reason there is no "c2s_no_context_takeover" parameter that allows the
> client to _announce_ it supports the feature?
>

I assumed that there's no platform/language where it's difficult to
implement c2s_no_context_takeover. It means that DEFLATE context is
singleton and can never be reset (or reset operation is very heavy).

Unless there's any existing difficulty, c2s_no_context_takeover should be
supported widely, I think.

Your suggestion allows a client to force a server which is reluctant to but
can enable c2s_context_takeover to enable it. Do you think it's important
functionality?

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Jun 2013 04:57:43 -0700

>>Rgd 1.: I am asking since Autobahn provides APIs for message, frame and streaming processing and we want to make sure permessage-deflate works for those also.

No problem. Thanks!

Autobahn now implements both server and client, and has all features controllable.

***
To everyone:

I am looking for other implementations of permessage-deflate (client and/or server) .. including ones that allow to control features .. to test interoperability.
***

Note: We are also working on extending Autobahn testsuite to test permessage-deflate .. but that will take a litte time ..

/Tobias

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Jun 2013 05:20:48 -0700

>     http://code.google.com/p/chromium/issues/detail?id=245719

Thanks for awesome speed with fixing the first!

> 
>     2)
>     If client offers multiple PMCE choices to the server by including
>     multiple elements in the "Sec-WebSocket-Extensions" header and the
>     server supports more than one of the PMCEs offered, is there any
>     preference which PMCE the server SHOULD choose? E.g. the first PMCE
>     in the header that is supported by the server?
> 
>     The description of the examples in "5.1. Negotiation Examples" seem
>     to suggest that the _ordering_ of alternative PMCE offered
>     designates _preference_.
> 
> 
> Yes. Order means preference. The server may choose any even if it 
> supports all of them.
> 

I think it would clarify things by adding text like

"""
A server MAY choose any PMCE offered by the client and supported by the server.
The order of PMCE offered by the client designates preference by the client and a server SHOULD accept the first offer in the list (by order) that it supports.
"""

This IMHO makes sense and brings the abstract description in line with the text in "5.1. Negotiation Examples".

>     So if multiple permessage-deflate PMCEs are offered by the client,
>     the server silently skips any invalid until the first valid or none
>     if there is no single valid one?
> 
>     The server does NOT fail the WS connection upon encountering the
>     first invalid PMCE in the list of offers?
> 
> 

"doesn't have to fail" ..

So a server MAY fail the handshake instead of just skipping bad offers? Confused.

At least for me I have troubles with the word "decline" in "A server MUST decline .."

a) decline = silently skip

OR

b) decline = fail handshake

?

I'd prefer b) - why be too liberal here - , but am ok with a) also .. but please consider defining "decline" in the text.

Sorry: maybe it's just me, but I was really spending time with above question ..

> Yes. To allow multiple offers but make it unnecessary to identify 
> accepted request (e.g. we need to attach request ID or something to each 
> extension entry to do that), I made permessage-deflate responses 
> self-contained (both client side parameters and server side parameters 
> are listed in a response).
> 

Lets say a client supports all features, but offers

permessage-deflate; s2c_max_window_bits=10

(hence, no fallback)

and the server responds:

permessage-deflate.

Autobahn client will FAIL that, since the PMCE response from the server does not match any offer, despite the reponse PMCE is supported.

If it would accept that, why would there be a need to offer explicit fallbacks like

permessage-deflate; s2c_max_window_bits=10, permessage-deflate

?

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Jun 2013 05:31:49 -0700

Am 04.06.2013 13:35, schrieb Takeshi Yoshino:> On Sun, Jun 2, 2013 at 7:29 AM, Tobias Oberstein 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Any reason there is no "c2s_no_context_takeover" parameter that
>     allows the client to _announce_ it supports the feature?
> 
> 
> Unless there's any existing difficulty, c2s_no_context_takeover should 
> be supported widely, I think.

The current draft reads:

" It is RECOMMENDED that clients implement the
   "c2s_no_context_takeover" parameter."

It's not "MUST implement" .. so there might be client which won't, and then a server has no chance to adapt it's response to the client and maximize likelihood of connection success given it ideally wants to activate "c2s_no_context_takeover".

> 
> Your suggestion allows a client to force a server which is reluctant to 
> but can enable c2s_context_takeover to enable it. Do you think it's 
> important functionality?

Why is that?

The presence of "c2s_no_context_takeover" in the client's offer would just signal to the server that the client supports the feature and the server MAY enable it.

Why signal feature presence for "c2s_max_window_bits", but not "c2s_no_context_takeover"?

Maybe it's just me, but the asymmetry was a source of confusion to me.

I think it would make the spec more consequent, and the change is minimal, e.g. it's a 1 liner for Chrome:

String CompressionMessageExtensionProcessor::handshakeString()
{
    return ASCIILiteral("permessage-deflate; c2s_max_window_bits");
}

=>

String CompressionMessageExtensionProcessor::handshakeString()
{
    return ASCIILiteral("permessage-deflate; c2s_max_window_bits; c2s_no_context_takeover;");
}

/Tobias

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 14 Jun 2013 01:22:33 +0900

> b) decline = fail handshake
I meant a). As far as the Sec-WebSocket-Extensions header value conforms to
the grammar for it and the server doesn't include a reply for it, there
should be no problem.

But I'm also fine with b).

> I'd prefer b) - why be too liberal here - , but am ok with a) also .. but
> please consider defining "decline" in the text.
>
Sorry about that.

> > Each extension has sole discretion to consider what response to be

Sorry. Let me correct this:
s/Each extension/The client/.

> permessage-deflate; s2c_max_window_bits=10

This is bad response. Servers MUST not generate such a response.

But if the client can and wants to accept the response, we don't have to
stop it.

I remember the rationale based on which we made RFC 6455 to enforce servers
to reject frames w/o masking. Enforcing receiver to ban bad-mannered data
can work as some pressure to server implementors to conform to the server
requirements.

>From this point of view, I agree with you.

>
> Autobahn client will FAIL that, since the PMCE response from the server
> does not match any offer, despite the reponse PMCE is supported.
>
A server conforms to the spec (requirements for servers) responds with
"permessage-deflate" to this request, but doesn't for "permessage-deflate;
s2c_max_window_bits=10".

So, the client needs to send this.

Sorry that my last response was unclear and misleading.

---

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 14 Jun 2013 02:23:46 +0900

On Tue, Jun 4, 2013 at 9:31 PM, Tobias Oberstein <
[email-masked]> wrote:

> It's not "MUST implement" .. so there might be client which won't, and
> then a server has no chance to adapt it's response to the client and
> maximize likelihood of connection success given it ideally wants to
> activate "c2s_no_context_takeover".
>

Oh, this is a bug. This should be MUST.

> The presence of "c2s_no_context_takeover" in the client's offer would just
> signal to the server that the client supports the feature and the server
> MAY enable it.
>

This makes sense. But as I said above I wanted to mean
c2s_no_context_takeover implementation is mandatory for clients. Once the
fix is made on the spec, it would be rather confusing that we have
announcement about mandatory feature.

> e.g. it's a 1 liner for Chrome:

Yes

---

From: Peter Thorson <[email-masked]>
Date: Thu, 13 Jun 2013 16:33:39 -0500

> Oh, this is a bug. This should be MUST.

+1

> This IMHO makes sense and brings the abstract description in line with the text in "5.1. Negotiation Examples".

Technically RFC6455 explicitly states that extensions are offered in order of preference, but I agree that there should be some text here (beyond the hints in the examples) that either references that or just reminds implementers that this is the case. 

Just to confirm, am I correct in assuming the following operations shouldn't cause problems:
1. Either endpoint may unilaterally send messages compressed with a smaller window size than the (implicit or explicit) negotiated maximum window size.
2. Either endpoint may unilaterally reset its own context for outgoing messages without informing the peer that it is doing so.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 19 Jun 2013 17:48:37 +0900

On Fri, Jun 14, 2013 at 6:33 AM, Peter Thorson <[email-masked]>wrote:

> 1. Either endpoint may unilaterally send messages compressed with a
> smaller window size than the (implicit or explicit) negotiated maximum
> window size.
>

Y. Decompressor with larger window works for data compressed with smaller
window.

> 2. Either endpoint may unilaterally reset its own context for outgoing
> messages without informing the peer that it is doing so.
>

Y. Just data after reset doesn't have back reference to the old context.
Decompressor still holds old history which new frames never refer, but
works for them.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 19 Jun 2013 17:53:55 +0900

I've fixed bugs reported in this thread in -10 draft.

It's still ignoring bad offers. It does not fail.
Anyone else has opinion on this?