Subject: Re: [hybi] Call for interest: multiplexing dedicated for WebSocket

From: Tobias Oberstein <[email-masked]>
Date: Sat, 1 Jun 2013 02:55:54 -0700

> On Mon, May 27, 2013 at 10:25 AM, Takeshi Yoshino <[email-masked]>
> wrote:
> > Isn't it in-app priority? Can't script and server address that
> > problem? If only the client knows current priority, the client can
> > tell the priority to the peer servlet using a WebSocket message. If
> > the server know priority for each position, it can just control transmission
> speed.
> >
> > Even if we need help by the server, the servlet would call some
> > special function to tell the server about priority.
> >
> > Different from HTTP/2.0's case, we don't need to provide a special
> > field in mux protocol to pass priority, I think.

If intermediaries are no preserve a per-channel priority based scheduling, then they might need to know the channel priority.

> 
> Sure, but then you force every application to reinvent this mechanism on top of
> WebSocket mux.
> At that point, one could argue that application messages could have their own
> "channel_id" in their messages, and applications can do mux themselves, if
> they need it.

Yep, agree.

Scheduling integrated with MUX has another advantage:

e.g. when doing per-channel priority based scheduling the scheduling could not only consider channel priority, but also the quota.

A high priority channel with data outstanding to be sent but not quota to send would otherwise block a lower priority channel with data _and_ quota.

Another issue: implementing s.th. like priority based scheduling at app level would might require to reinvent some kind of fragmentation at the app level to be able to schedule according to priority (otherwise once a very large message on a low prio channel is started to be sent, it'll block everything else).

One way to look at this is:

WS fragmentation < WS multiplexing < WS scheduling | APP

where "<" means "builds on", and everything up to "|" is provided by an WS implementation, hence does not need to be reinvented at app level, and WS aware infrastructure can optimize / do their thing.

===

It seems that HTTP/2 likely will have "request priorization" (I don't know about "response priorization").

If we (in a pure WS land) don't have priorization, that's a potential disadvantage and source of discussion.

===

A per-channel scheduling could work according to different policies:

1. Round-Robin scheduling
2. Priority scheduling
3. Fair Bandwidth scheduling
4. First-Come, First-Serve scheduling

The policy mechnism could be selected for the whole physical WS (and communicated for intermediaries), e.g.

Sec-WebSocket-Extensions: mux; scheduler="round-robin"

Only with 2. there is a need to select and communicate a per-channel info (during channel creation), e.g.

Sec-WebSocket-Extensions: mux; scheduler="priority"

and then

AddChannelRequest with the 3 RSV bits (currently unassigned) containing a priority 0-7 for the channel. 

[This does not yet answer how to re-prioritize an existing channel]

Hence I'd like to propose to

1) add an (optional) extension parameter "scheduler" as above
2) use the 3 currently unassigned RSV bits in AddChannelRequest for channel prio (when scheduler="priority")
3) define appropriate MUST semantics (scheduling should not follow advisory semantics .. apps should be able to rely on it)
4) probably add a new PriorityControl message to change prio of existing channels

===

Above is a kind of per-channel scheduling. One might argue that it lacks in at least 2 respects:

- an app is not able to control e.g. prio on a per-message basis
- there are no ordering guarantees between messages on different channels

Example:
An app creates an empty file on some cloud service. Subsequently the app wants to append to that file.

The first could be a high-prio message, the latter messages low-prio.

The app does not care whether those messages travel over 1 physical, or multiple different logical WS channels.

However, the app does care that the first "create file" message arrives before any subsequent "append to file" messages.

So a different design involving per-message prios on a single WS channel (physical or logical) might allow higher-prio messages to overtake lower prio-messages, but not otherwise round. 

/Tobias

---

From: Roberto Peon <[email-masked]>
Date: Mon, 3 Jun 2013 11:12:50 -0700

Well, more precisely, the compression context must be shared between
material sourced by the attacker and also for the purpose of using the
site, and the context must be probe-able in some reasonable amount of time.
A gzip context which was reused between messages would be a no-no, for
instance. Another compression context (which makes probing difficult) might
be acceptable.

-=R

On Fri, May 31, 2013 at 1:27 AM, Tobias Oberstein <
[email-masked]> wrote:

> The way CRIME works is to have a 3rd party include links to the
>> site-to-attack. Thus it doesn't matter if the schemes are http, https,
>> ws, wss, whatever.
>>
>
> Does above refer to the general requirement for CRIME: the attacker must
> be able to inject data into the source material before it is compressed and
> then encrypted?
>
> If so, how would that be possible with WebSocket and the proposed
> WebSocket per-message compression?
>
>
> random thoughts:
>
> a)
> permessage-deflate only compresses WS data messages, not control frames.
>
> So the attacker would need to inject data into the app-level payload.
>
> The only way to send app-level payload in browsers if via the JavaScript
> WebSocket API (socket.send()).
>
> So the attacker would need to somehow modify this JS code. However, the
> origin of the JS running is (at least with browsers) sent during the WS
> opening handshake as a HTTP header, and a WS server can check and decline
> an incoming connection.
>
> b)
> With WS HTTP headers are only sent _once_: during the initial WS opening
> handshake, and by the client to the server.
>
> CRIME requires repeated injection of attacker data.
>
> More so: permessage-deflate does not touch the opening handshake, which
> remains uncompressed.
>
> Side Q: Do browsers reuse an established TLS/TCP connection for multiple
> WS connections?
>
> /Tobias
>
>

---

From: Roberto Peon <[email-masked]>
Date: Mon, 3 Jun 2013 11:15:38 -0700

imho, we're reinventing SPDY/HTTP/2, and I'd rather implement the
complexity of muxing, security, prioritization, flow-control,
liveness-checking, stream initiation, and compression, only once.
-=R

On Sat, Jun 1, 2013 at 2:55 AM, Tobias Oberstein <
[email-masked]> wrote:

> If intermediaries are no preserve a per-channel priority based scheduling,
> then they might need to know the channel priority.
>
> >
> e.g. when doing per-channel priority based scheduling the scheduling could
> not only consider channel priority, but also the quota.
>
> A high priority channel with data outstanding to be sent but not quota to
> send would otherwise block a lower priority channel with data _and_ quota.
>
> Another issue: implementing s.th. like priority based scheduling at app
> level would might require to reinvent some kind of fragmentation at the app
> level to be able to schedule according to priority (otherwise once a very
> large message on a low prio channel is started to be sent, it'll block
> everything else).
>
> where "<" means "builds on", and everything up to "|" is provided by an WS
> implementation, hence does not need to be reinvented at app level, and WS
> aware infrastructure can optimize / do their thing.
>
> ===
>
> 3) define appropriate MUST semantics (scheduling should not follow
> advisory semantics .. apps should be able to rely on it)
> However, the app does care that the first "create file" message arrives
> before any subsequent "append to file" messages.
>
> So a different design involving per-message prios on a single WS channel
> (physical or logical) might allow higher-prio messages to overtake lower
> prio-messages, but not otherwise round.
>
> /Tobias
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 3 Jun 2013 14:33:27 -0700

>imho, we're reinventing SPDY/HTTP/2, and I'd rather implement the complexity of muxing, security, prioritization, flow-control, liveness-checking, stream initiation, and compression, only once.
>-=R

First, lets admit: we are both biased;)

What about HTTP/2 over WS?

http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility-02

I prefer native WS. I do single-page web apps with app manifests.

The attractiveness of SPDY(TM) (be it native or over WS) in this situation is limited. Any static resources are retrieved only once anyway, and any communication after that can be done via WS.

- native WS is widely implemented today
- native WS has lower per-message overhead than "WS over SPDY(TM)"
- WS (now) has _payload_ compression
- liveness checking => WS pings/pongs
- security => WS masking, WS origin

So I'd say a WS-MUX with flow-control (draft there) plus priorization/scheduling (not there) is "the last missing piece". Lets finish it.

/Tobias

Yep, agree.

===

===

and then

===

Example:

/Tobias

---

From: Roberto Peon <[email-masked]>
Date: Mon, 3 Jun 2013 15:07:53 -0700

Noone is unbiased, but I'm a lot less biased than you'd think given my name
is within the WS doc too and I was pretty active in attempting (and sadly
failing) to get mux into the base WS functionality. :)

Out of curiosity, and on a different topic, how large have the messages
been in your applications?
-=R

On Mon, Jun 3, 2013 at 2:33 PM, Tobias Oberstein <
[email-masked]> wrote:

> The attractiveness of SPDY(TM) (be it native or over WS) in this situation
> is limited. Any static resources are retrieved only once anyway, and any
> communication after that can be done via WS.
>
> So I'd say a WS-MUX with flow-control (draft there) plus
> priorization/scheduling (not there) is "the last missing piece". Lets
> finish it.
>
> /Tobias
>
>

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 3 Jun 2013 21:41:08 -0700

> Noone is unbiased, but I'm a lot less biased than you'd think given my name is within the WS doc too and I was pretty active in attempting (and sadly failing) to get mux into the base WS functionality. :)

I think the extensibility built into WS (extensions and subprotocols) is a strength, and having a relatively simple base functionality is good. I don't want to implement MUX on an embedded device like Arduino, and I likely don't need it there anyway. Also: having implemented an automated WS protocol test suite I can say that getting good coverage of the "relatively simple base functionality" already requires a lot. Even the >300 test cases we have today don't cover 100%. I guess writing automated tests for WS-MUX / flow-control / scheduling might be a challenge;)

>Out of curiosity, and on a different topic, how large have the messages been in your applications?

I haven't don't systematic measurements / stats, but - today - probably between 20 - 20k of JSON payload with a fair amount of URIs sprinkled in.

Now that we've implemented "permessage-deflate", I'll take the chance and add some traffic stats tracking ..

/Tobias

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 18:04:13 +0900

On Thu, May 30, 2013 at 8:28 PM, Tobias Oberstein <
[email-masked]> wrote:

> b) the endpoints need to cut mass data into smaller chunks to reserve
>>
> slots for inserting chatty traffic. e.g. if acceptable delay for chatty
>> traffic is x ms, set the maximum send unit of mass traffic to (current
>> through put) * x.
>>
>> To allow one to control the peer's message scheduling, both per-socket
>> buffer size and ws-mux flow control are insufficient, I think. Needs to
>> exchange ToS info.
>>
>
> I now can see that flow control and priorization are somewhat orthogonal
> issues.
>
> Priorization and ToS is not goal of WS-MUX at all?
>
>
It's again, as I said in reply to Simone's comment above, it depends on
frequency and usage if we should have WebSocket protocol level ToS or
should just do it in JS.

...

>> Traffic for various services would be multiplexed into one TCP. We wanna
>> demultiplex it and forward to designated backends. This means that
>> there're again lots of TCP connections between LB-backends, but we can
>> make any optimization there as we want.
>>
>
> I see. So the load-balancer needs to be a layer-7 one, and not only be
> WS-MUX enabled, but also app/service aware? Something like this:
>
> E.g. user has 2 tabs open which (logically) connect to
>
> wss://google.com/gmail
> wss://google.com/gplus
>
> Both would be transported over 1 physical WS (via WS-MUX).
>
> The LB would demux traffic for logical channel
>
> wss://google.com/gmail
>
> to backend nodes of cluster "GMAIL", while it would demux traffic for
>
> wss://google.com/gplus
>
> to backend nodes of cluster "GPLUS"?
>
>
Yes.

>
>
>>      >Basically multiplexing should be done as close to network as
>>     possible. If it's done in application layer, for example when
>>     compression is used, such a load balancer need to decode the
>>     compression to see the >multiplexing information. That doesn't make
>>     them happy.
>>
>>     This seems to assume that
>>
>>     - either the LB is done above layer 4 and/or
>>     - the logical WS connections contained in the physical WS would need
>>     to be balanced to _different_ backend nodes.
>>
>>
>> Yes. Sorry that we haven't told the story in our mind in detail.
>>
>
> I think it might be helpful to add text to the MUX draft that describe the
> use cases WS-MUX addresses and how it improves thing over alternatives.
> Motivation is always good and helps by providing context and examples.

OK.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 18:08:16 +0900

Considering CRIME attack, we have a note in the current multiplexing spec
saying that history based compression should not be used over chunks
containing handshake and ones containing script generated data keeping
compression context.

E.g.
Sec-WebSocket-Extensions: permessage-deflate, mux
is allowed but
Sec-WebSocket-Extensions: mux, permessage-deflate
is prohibited.

I listed CRIME just as an example of security consideration around
multiplexing. For now, it's already addressed in the mux spec.

---

From: Simone Bordet <[email-masked]>
Date: Tue, 4 Jun 2013 11:21:15 +0200

Hi,

>>imho, we're reinventing SPDY/HTTP/2, and I'd rather implement the complexity of muxing, security, prioritization, flow-control, liveness-checking, stream initiation, and compression, only once.

FWIW, I am with Roberto here.

> - security => WS masking, WS origin

Sure. But I fail to see why you need MUX for your use case. One duplex
connection seems enough to me.

> So I'd say a WS-MUX with flow-control (draft there) plus priorization/scheduling (not there) is "the last missing piece". Lets finish it.

The problem I see is that the 2 features you mention here are quite
complicated to specify and implement correctly, i.e. we duplicate the
larger effort undertaken by HTTP/2.

--
Simone Bordet
----
http://cometd.org
http://webtide.com
http://intalio.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Intalio, the modern way to build business applications.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 4 Jun 2013 18:31:15 +0900

> then they might need to know the channel priority.

Good point.

> send would otherwise block a lower priority channel with data _and_ quota.

Yes, if the algorithm is designed to do so.

> large message on a low prio channel is started to be sent, it'll block
Yes. The app must be able to send a fragment.

But it's a matter of WebSocket API. A server can do that, but if a client
want to, it can't for now due to limitation of the API. Hence, it needs
application level fragmentation.

> disadvantage and source of discussion.
Yes

> 4) probably add a new PriorityControl message to change prio of existing

WebSocket API is not ready for such a sophisticated ToS mechanism.

> - there are no ordering guarantees between messages on different channels

I think it's ok.

> prio-messages, but not otherwise round.

I see.

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Jun 2013 02:33:42 -0700

> connection seems enough to me.

Example: Say I want to write a single-page browser app that has both low-latency chatty traffic and mass-data traffic (e.g. chat + file transfer).

I want the chatty traffic maintain it's low-latency, while mass-data transfers are concurrently happening.

This could be done by muxing 2 logical WS over a physical WS with the first WS having high-prio vs the 2nd WS.

So a small auto-fragment size plus flow-control and scheduling would be helpful.

Doing this in JS seems suboptimal the least ..

> priorization/scheduling (not there) is "the last missing piece". Lets finish it.
> 
> complicated to specify and implement correctly, i.e. we duplicate the larger

I agree .. those are tricky. FWIW I am willing to invest time and effort to extend Autobahn testsuite for WS extensions. We are working now on permessage-deflate.

> effort undertaken by HTTP/2.

It all depends on perspective, goals and use cases .. but I can follow your view .. guess Jetty wants to have both WS and SPDY.

/Tobias

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 4 Jun 2013 03:04:31 -0700

>     Another issue: implementing s.th <http://s.th>. like priority based
>     scheduling at app level would might require to reinvent some kind of
>     fragmentation at the app level to be able to schedule according to
>     priority (otherwise once a very large message on a low prio channel
>     is started to be sent, it'll block everything else).
> 
> 
> But it's a matter of WebSocket API. A server can do that, but if a 
> client want to, it can't for now due to limitation of the API. Hence, it 
> needs application level fragmentation.

The WS client implementation could do auto-fragmentation. The fragment size
could be chosen by upstream bandwidth and latency target. A WS server could do
the same. Auto-fragment based on downstream bandwidth and latency target.

The app isn't bothered with fragmentation then.

Yes.

Apart from extending the JS WS browser API (which currently only has
WS URL and protocol for open) with ToS:

var sock = new WebSocket("ws://example.com ", [], {scheduler: "priority", priority: 7});

This would assume that logical WS channels eligible to be shared over 1 physical WS
not only share same host:port, but also scheduler.

[this API does not provide for 4) - dynamically changing channel prio]

Another option would be to let the server choose based on URL requested

URL 1 : ws://example.com/chat
URL 2 : ws://example.com/file

If the client connects and speaks MUX, the server can answer (in opening handshake)

mux; scheduler=priority; priority=7

for URL 1 e.g.. For the first connection, that would happen on the physical WS. For subsequent
connections, it would happen on logical WS (scheduler needs to be the same): MUX is not nested,
but only the "priority" parameter used by the client WS implementation.

Yet another option would be to "use" URL parameters to hint

URL 1 : ws://example.com?scheduler=priority&priority=1
URL 2 : ws://example.com?scheduler=priority&priority=7

and let the server make final decision.

>     The app does not care whether those messages travel over 1 physical,
>     or multiple different logical WS channels.
> 
>     However, the app does care that the first "create file" message
>     arrives before any subsequent "append to file" messages.
> 
>     So a different design involving per-message prios on a single WS
>     channel (physical or logical) might allow higher-prio messages to
>     overtake lower prio-messages, but not otherwise round.
> 
> 
> I see.

This would definitely require API change

sock.send("foobar", 7); // send message with priority 7

It would provide even richer semantics to an app though ..

---

From: Adam Rice <[email-masked]>
Date: Mon, 10 Jun 2013 16:12:07 +0900

I believe our priority is to finalise a multiplexing specification quickly.
So we should focus on what is essential for multiplexing to be useful.

I think we have general consensus that flow control is essential.

I have not been convinced that prioritisation is essential. In the event
that a single application has both low-latency and high-throughput
channels, the lack of prioritisation in the protocol can be worked around
by using different endpoints (hence different physical channels) for
different traffic types. To make this easier to deploy, I am leaning
towards the idea that wss://a.example.com/ and wss://b.example.com/ should
be considered distinct endpoints, even if the IP address is the same.

This is not ideal, and the low-latency and high-throughput channels will
still fight for bandwidth at the network level, but it is still a win to
have one each of latency-sensitive and throughput-sensitive TCP/IP
connections rather than a dozen of each.

I am not saying that prioritisation is not desirable, just that I don't
think the complexity versus benefit tradeoff makes sense right now.

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 11 Jun 2013 02:08:30 -0700

Am 10.06.2013 09:12, schrieb Adam Rice:> I believe our priority is to finalise a multiplexing specification 
> quickly. So we should focus on what is essential for multiplexing to be 
> useful.

I agree.

I'd like to note however that "useful" from my POV depends on use-case. 

Reducing the number of physical TCPs in scenarios like

1. multiple web apps of a single vendor open in multiple browser tabs
2. web widget from a vendor embedded into pages of many sites

might already be well addressed with MUX+flow-control and don't necessarily need priorization.

Scenarios like e.g.

3. single app with multiple communication requirements with different  TOS (low-latency/mass..)

might need MUX+flow-control+priorization.

> I think we have general consensus that flow control is essential.

Yep, seems nobody stepped up for multiplexing _without_ flow control.

> I have not been convinced that prioritisation is essential. In the event 

It's not essential for use case 1. + 2. above, I agree. But it is for use case 3.

> for different traffic types. To make this easier to deploy, I am leaning 
> towards the idea that wss://a.example.com/ <http://a.example.com/> and 
> wss://b.example.com/ <http://b.example.com/> should be considered 
> distinct endpoints, even if the IP address is the same.

+1 for making those "different" (cannot be mux'ed over same physical WS).

> think the complexity versus benefit tradeoff makes sense right now.

Ok. I agree that specifying different scheduling schemes etc in detail now
would delay the draft.

However, even the current draft contains text

"""
13. Fairness

Sender side

   o  The sender MUST use a fair mechanism for selecting which logical
      channel's data to send in the next WebSocket message.  Simple
      implementations may choose a round-robin scheduler, while more
      advanced implementations may adjust priority based on the amount
      or frequency of data sent by each logical channel.
"""

Here is an idea: could we just include minimal additions which would allow
describing specific scheduling algorithms later or in separate documents?

Like e.g. the "permessage-compression" draft does: it provides a _framework_
for compression schemes, and only specs 1 algo: deflate.

Here is a proposal that only adds to the opening handshake on physical and logical WS:

************************

1) Handshake on physical WS

Example 1.1

client-to-server: "mux; scheduler=roundrobin"

=> client announces MUX, and states that it implements only "roundrobin" scheduler.

server-to-client: "mux; scheduler=roundrobin"

=> server agrees to speak MUX and round-robin scheduling

Only "roundrobin" would be described in the current draft.

Example 1.2

client-to-server: "mux; scheduler=priority, mux; scheduler=roundrobin"

=> client annouces MUX, preferring "priority" based scheduler, but may fallback to roundrobin

server-to-client: "mux; scheduler=priority"

=> server agrees to speak MUX and select priority-based scheduling

Detailing "priority" scheduling would be postponed.

2) Handshake on logical WS

For a future priority scheduler, a client should be able to _hint_ which priority
a new (logical) WS connections should have.

The only way to do that without changing the WS (browser) API would be
to include the "hint" within the WS URL opened by the client:

Example 2.1

Example 2.2

URL 1 : ws://example.com?priority=7
URL 2 : ws://example.com?priority=1

A server receiving such a new logical WS connection may then respect the hint,
and send back header in logical handshake:

Sec-WebSocket-Extensions: mux; scheduler=priority; priority=7

if the scheduler is "priority" on the physical WS. Otherwise it ignores the hint.

The client then applies the parameters sent by the server in the logical handshake to
the scheduler for the physical WS.

************************

In effect, the scheduling scheme in use is negotiated during the opening handshake on 
the physical WS, while the per-channel scheduler parameters can be hinted by the client,
but are always (finally) decided by server (for both client and server side) and determined
during the opening handshake on the logical WS for the channel.

Pros:

+ only uses WS extensions headers of physical and logical WS
+ no additional MUX control frame opcodes
+ no use of MUX control frame RSV bits
+ no browser API change

Cons:

- scheduler parameters for logical WS cannot be changed dynamically
- TOS parameters cannot be chosen "per WS message"

I think with a little more work we could prepare for more advanced scheduling
schemes in the future and allow implementations to experiment today without
breaking interoperability.

/Tobias

---

From: "John A. Tamplin" <[email-masked]>
Date: Tue, 11 Jun 2013 11:06:40 -0400

> might need MUX+flow-control+priorization.

In the case of a single app, it can always do its own prioritization, so I
do not think it essential for this case.

> breaking interoperability.

I don't see much value in providing the extension mechanism in this draft
without actually requiring an implementation, and requiring a priority
implementation means adding a lot of extra work.  The only way I would
support this would be if the text were basically "the application can
provide an indication of priority, and the server is free to use or ignore
this information as it sees fit".  In that case, would anyone bother to use
it?

-- 
John A. Tamplin

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 11 Jun 2013 08:35:53 -0700

> In the case of a single app, it can always do its own prioritization, so 

The app then needs to implement app-level fragmentation, where the
draft already requires fragmentation for mux.

> I do not think it essential for this case.

For a single instance (tab) of the app, the can reinvent fragmentation
and implement it's own priority scheduling.

This seems to be highly inefficient.

For multiple instances (tabs) of the app, it's essential, since the app cannot
implement itself.

> I don't see much value in providing the extension mechanism in this 
> draft without actually requiring an implementation, and requiring a 

The draft already requires the implementation to use a "fair scheduler", but
leaves the exact mechnism open.

> priority implementation means adding a lot of extra work.  The only way 
> I would support this would be if the text were basically "the 
> application can provide an indication of priority, and the server is 
> free to use or ignore this information as it sees fit".  In that case, 
> would anyone bother to use it?

A client can only hint, but _if_ the server sets priority on a channel, the
client _must_ schedule accordingly.

For a implementations not supporting priority scheduling, the proposal
essentially only relaxes the allowed parameters in Sec-WebSocket-Extensions
header for client-to-server:

priority-enabled client => non-supporting server:

c2s: mux; priority=scheduler; mux
s2c: mux

Change: skip mux configurations not supported instead of failing the handshake.

non-supporting client => priority-enabled server:

c2s: mux
s2c: mux

No change at all.