Subject: [hybi] Changing meaning of 1007 status code ("frame" => "message")

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 6 Sep 2011 17:37:25 +0200

As said in other mail I suggest changing the meaning of 1007 status code from:

      1007 indicates that an endpoint is terminating the connection
      because it has received data that was supposed to be UTF-8 (such
      as in a text frame) that was in fact not valid UTF-8 [RFC3629].

to:

      1007 indicates that an endpoint is terminating the connection
      because it has received a message that was supposed to be UTF-8
      that was in fact not valid UTF-8 [RFC3629].

WS message format/encoding/content MUST be validated when receiving
all the frames belonging to such message, as message inspection/usage
belongs to the WS application, rather than the WS "transport" layer
(framing stuf).

Of couse, a WS endpoint COULD validate each text frame by inspecting
whether it contains valid UTF-8 bytes (completed or splitted bytes).
That's could be an optimization out of the scope of the protocol
specification (as many others).

I insist: this specification SHOULD define logic layers, basically:

* WS transport layer: framing stuff and control frames.
* WS application layer: sends and receives *full* WS messages (binary or text).

- The WS transport layer receives control frames and send control
frames to the other endpoint.
- The WS transport layer gives received *messages* to the WS
application layer (it doesn't matter giving the full buffered message
or using straming).
- The WS application layer sends *messages* to the WS transport layer.
The transport layer decides whether to split, or not, the message into
N frames.
- The WS application layer notifies the WS transport layer to close
the connection with status 1007 upon receipt of an invalid UTF-8 text
message.
- The WS application layer notifies the WS transport layer to close
the connection due to other causes (normal termination and so).
- The WS transport layer notifies the WS application layer when the
connection has been closed (locally or remotely) and the exact cause.

How the WS transport layer and the WS application layer sends messages
or notifies events to each other is out of the scope of this
specification (of course).

Validation of a WS message should be done by the WS application layer,
so 1007 code should refer to "message" rather than "frame".

IMHO this is obvious, and it's also obvious that defining logic layers
when creating a protocol specification is *good* for the implementor.
If not, we end with these kinds of anarchic discussions (see other
threads) which have difficult answer due to the protocol specification
ambiguity.

Please consider it. Any protocol in Internet is designed with logical
layers, all but WebSocket.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 6 Sep 2011 08:54:12 -0700

>       that was in fact not valid UTF-8 [RFC3629].

Does not make a difference, since a message that consists of
100 frames and contains invalid UTF-8 in the first frame is
as invalid as the same message containing the invalid octets
only in the last frame.

Why should I read the subsequent 99 frames after having
received the first frame with invalid UTF-8 before failing
the connection?

> WS message format/encoding/content MUST be validated when receiving all
> the frames belonging to such message, as message inspection/usage belongs
> to the WS application, rather than the WS "transport" layer (framing stuf).

No, since that would imply that you can't have a message of arbitrary
number of frames, or a message frame with 2^63 octets. No one could
buffer that, and there is no need to do so anyway, when you have a frame-based
or a streaming API in your implementation.

The question of API is orthogonal to the protocol.

> That's could be an optimization out of the scope of the protocol specification
> (as many others).
> 

No, thats not WS. WS itself is not a message based, but framed streaming
protocol.

> 
> - The WS transport layer receives control frames and send control frames to
> the other endpoint.
> - The WS transport layer gives received *messages* to the WS application
> layer (it doesn't matter giving the full buffered message or using straming).

No, I have 3 types of API: message-based, frame-based and streaming.

> The transport layer decides whether to split, or not, the message into N
> frames.

No, there may be valid scenarios when the application wants to control
fragmentation. Think about multi-channel media streaming.

> - The WS application layer notifies the WS transport layer to close the
> connection with status 1007 upon receipt of an invalid UTF-8 text message.

No, the WS implementation fails the connection and fires a connectionClose()
handler on the application. See the JS WS API.

> Please consider it. Any protocol in Internet is designed with logical layers, all
> but WebSocket.

Your premise is false, and thus any conclusion void.
 
> 
> Regards.
> 
> --
> Iñaki Baz Castillo
> <[email-masked]>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Philipp Serafin <[email-masked]>
Date: Tue, 06 Sep 2011 18:25:08 +0200

Am 06.09.2011 17:54, schrieb Tobias Oberstein:
> No, thats not WS. WS itself is not a message based, but framed
> streaming protocol.
I can understand the rationale for a streaming-based API (though I don't
see why already established multimedia protocols like SIP, RTSP or RTMP
wouldn't be a better choice here) but why would we both need
message-based *and* frame-based APIs?

Frames may be arbitrary buffered by intermediaries, the current
javascript API hides frames completely and, as previously discussed, you
can't be sure that a frame belonging to a text message actually contains
valid UTF-8. All of that seem to make them a decidedly bad choice for me
to base your API on. What additional benefits compared to a
message-based API would that give you?

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 6 Sep 2011 09:32:32 -0700

> >> application layer (it doesn't matter giving the full buffered message or
> using straming).
> I can understand the rationale for a streaming-based API (though I don't see
> why already established multimedia protocols like SIP, RTSP or RTMP
> wouldn't be a better choice here) but why would we both need message-

None of these has the (promised) attraction of being "compatible with
restricted network infrastructures".

> based *and* frame-based APIs?

A multimedia application that wants to stream over WS may want to control
fragmentation size when sending. Intermediaries MAY coalesce/fragment,
but only when not extension is in place, which semantics is not understood
by the interm.
 
> Frames may be arbitrary buffered by intermediaries, the current javascript
> API hides frames completely and, as previously discussed, you can't be sure

The JS API is just that: an API for a single environment in which WS is
implemented.

Other environment may provide frame-based/streaming APIs, and even
JS may provide such things in a 2nd revision.
 
> that a frame belonging to a text message actually contains valid UTF-8. All of
> that seem to make them a decidedly bad choice for me to base your API on.

Of course a frame-based API may be more useful for binary messages. But
even for text: what if I write received streams immediately to a temp file.
When the connection fails because of invalid UTF-8, I just delete the temp file.
When everythings fine till the message end, I rename the temp file to final file.

> What additional benefits compared to a message-based API would that give
> you?

As a library author, I want to give choice to users, since I don't know their
requirements.
I would not use a mixture of APIs in _one_ application.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 6 Sep 2011 18:33:40 +0200

> received the first frame with invalid UTF-8 before failing

IMHO you are mixing a specification with an implementation. I already

> That's could be an optimization out of the scope of the protocol specification

But what you are doing is subordinating the protocol specification to
the implementation you have in mind.

All the rest of your responses are based in that, IMHO, wrong assumption.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 6 Sep 2011 18:36:46 +0200

> restricted network infrastructures".

Please, again the assumtion of an Internet based on HTTP and WebSocket
no. There si life out of port 80, really.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Richard L. Barnes" <[email-masked]>
Date: Tue, 6 Sep 2011 12:59:38 -0400

As suggested in the other thread:
"
  1007 indicates that an endpoint is terminating the connection
  because it has received a text message that was not valid UTF-8 [RFC3629].
"
Text frames are the only UTF-8 things that are currently specified, and future UTF-8 things should have their own error codes.

---

From: "Richard L. Barnes" <[email-masked]>
Date: Tue, 6 Sep 2011 13:01:48 -0400

> received the first frame with invalid UTF-8 before failing

Because frames don't have to contain valid UTF-8.  Didn't we just have this argument?  :)

--Richard

---

From: John Tamplin <[email-masked]>
Date: Tue, 6 Sep 2011 13:03:37 -0400

On Tue, Sep 6, 2011 at 12:59 PM, Richard L. Barnes <[email-masked]> wrote:

> Text frames are the only UTF-8 things that are currently specified, and
> future UTF-8 things should have their own error codes.

I suggest it would be more useful for 1007 to mean that the payload contents
were invalid for the frame type/negotiated extensions/etc.  For text
messages, that would mean invalid UTF8 sequences.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: "Richard L. Barnes" <[email-masked]>
Date: Tue, 6 Sep 2011 13:10:18 -0400

Seems sensible enough.
"
1007 indicates that an endpoint is terminating the connection because it has received data within a message that was not consistent with the type of the message (e.g., non-UTF-8 data within a text message).
"
--Richard

On Sep 6, 2011, at 1:03 PM, John Tamplin wrote:

> I suggest it would be more useful for 1007 to mean that the payload contents were invalid for the frame type/negotiated extensions/etc.  For text messages, that would mean invalid UTF8 sequences.
> 
> -- 
> John A. Tamplin

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 06 Sep 2011 11:13:52 -0600

<hat type='individual'/>

> 1007 indicates that an endpoint is terminating the connection because it has received data within a message that was not consistent with the type of the message (e.g., non-UTF-8 data within a text…

That seems good.

/psa

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 6 Sep 2011 19:15:32 +0200

>> 1007 indicates that an endpoint is terminating the connection because it has received data within a message that was not consistent with the type of the message (e.g., non-UTF-8 data within a text…

+1. It is a flexible definition which fits well with current purpose
(invalidating a text message due to invalid UTF-8 data).

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 06 Sep 2011 22:23:39 +0100

>(invalidating a text message due to invalid UTF-8 data).
Ok, changed in my copy.