Subject: Unidirectional streams PR

From: Martin Thomson <[email-masked]>
Date: Wed, 21 Jun 2017 17:41:36 +1000

I've created a pull request unidirectional streams.

https://github.com/quicwg/base-drafts/pull/643

I won't go into detail about this here, other than to point out that
there is extensive rationale for the choices I made in the PR summary,
a copy of which I will include for your convenience.

## Transport Changes

Streams are unidirectional.  Each has three states: idle, open, and
closed.  I separated the transitions for sending and receiving because
that turned out to be easier to explain.  The reordering thing makes
them different in subtle ways.

That's all.  The changes in transport are relatively small and they
simplify streams a lot.  That it also makes the transport more generic
is a nice bonus.

## HTTP Changes

This is where the bulk of the changes are.

### Stream Correlation

Previously request and response were implicitly correlated, as was the
data correlated with the request or response headers.  With this
change, that correlation each stream has a header that explicitly
correlates these.

There are 5 types of stream: connection control, request, response,
data, and push.  The first two have a simple header that has a type.
The next two reference another stream in their header, which ties
request to response and data to request or response.  Push streams
each reference a PUSH_PROMISE using a newly minted push ID (I decided
that was cleaner than what I proposed at the interim).

I chose backward references rather than forward references for two
reasons.  First, request and response correlation can't use forward
references because that would mean the client would be exerting
(uncoordinated) control over server streams.  Second, that gives the
server the most flexibility in terms of how it answers requests (see
#281).  The cost of using backward references is that endpoints have
to check that the backward references aren't bad, either referencing
the wrong type of stream, or with multiple references to the same
stream.

The presence or absence of a message body is signaled after the
initial header block using a HAS_BODY frame.  This empty frame
indicates that another stream will include the body of the message -
or a promise for a body.  I would like to eliminate this stream split.
See #245 and #557 for more details on that, though we need to fix #176
first, which brings us back to QPACK/QCRAM again.

### Prioritization

This changes prioritization so that it identifies requests.  I only
made the minimal changes here, which means that this doesn't fix #441
at the same time, I've left that for later (see below).

### Cancelling Pushes

Because server push doesn't create streams with PUSH_PROMISE, I had to
create a way to cancel them between the time that the PUSH_PROMISE is
sent and when the push stream is created.  That's called RST_PUSH.

## Things That Need Improvement

I haven't based this on #171.  I think that functionality is good, but
the last time I looked at the PR I didn't like some of the changes
Mike made there.  (That's a taste thing.)  This really assumes that we
accept something very much like #171.

This really needs QPACK/QCRAM.  Right now, it is impossible to cancel
some types of request using RST_STREAM because not all messages will
have a body (#176 again).  On balance, given that bodies are what you
really want to kill, it's not disastrous, but it's a problem
nonetheless.  I think that we all agree that this is something that we
need to fix, but we haven't reached the point where we agree on the
details of the fix.

## Things That Want Improvement

This also really wants headers and data on the same stream .  It
doesn't exactly need that, but merging the streams would make things a
lot easy, both conceptually and structurally.

I chose the simplest possible encoding for all of the fields that I
touched.  That means that stream IDs are all 32 bits in size and the
HAS_BODY frame takes an entire 9 octets.  For things that are that
common, there are many ways in which byte efficiency could be
improved.

The push ID changes might allow us to trivially fix #441.  Use of an
as-yet-not-created push ID as a node in the priority tree would allow
for prioritization to use "empty" nodes.  We'd need to explicitly
allow that though.

# Fixes

Closes #515, #240, #281, #175.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 21 Jun 2017 10:48:58 +0100

I have come to strongly support unidirectional streams. I'm writing because
I think we can take this just a tiny bit further and get a pretty big
architectural win.

the PR adds a stream-header to the http mapping.. a key feature of that is
a few bytes indicating what stream # a reply (or a push, etc..) is
associated with because its not implicit in the stream # anymore. I love
the removal of some transport IDs from the HTTP mapping.

If we take that just a touch further and include this label explicitly in
the request as well then there is no reason to require the use of transport
stream ids  for the labeling at all (though you could!) - and now you can
imagine quic http being implemented on a quic transport api that looks a
lot like existing transport apis without having to resort to inventing
something like quic_getpeerstreamid().

Relatedly the mapping defines a very short stream header to identify the
control channel  so I don't think there is any reason to lock it onto
stream 1 (and let the transport-id leak into the application mapping in the
process). Its not like putting it on stream 1 guarantees it will arrive
first anyhow.

-Patrick

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 21 Jun 2017 16:48:07 -0700

Thanks for writing up the PR, Martin. It does help to see the details laid
down. I have a few thoughts.

First, there are issues that unidirectional streams raises, some of which
we can articulate, and some which we don't know yet simply because we have
no production experience with them.

In the former category, here's an issue that isn't handled right now.
Google maps uses map-tiles for displaying parts of mapped region. As a user
moves to a different region, the client cancels all pending requests, since
those map-tiles are no longer relevant and those responses are no longer
needed. Let's say that the requests were completely sent (FIN and all), but
the responses hadn't started yet. In the bidirectional world, a client
would achieve this by sending a RST_STREAM on all streams that have
responses pending, causing the server to send a RST_STREAM back. In the new
world, the client would have no way of canceling a pending request until
the server started responding. (And right now in your PR, I don't think
there's a way for an endpoint to RST a peer-initiated stream.)

Of the issues we don't know about yet, we'll only know with deployment
experience. I'm wary of standardizing something where we have no deployment
experience, no production experience. (Server Push is an overused but
classic example of this: it was standardized before there was any
significant deployment experience, and we continue to struggle with
actually deploying it. Also, see HTTP/2 Priorities.)

Second, your key premise is that unidirectional streams makes transport
logic simpler. I agree with that, and the PR clearly simplifies the
transport stream machinery. However, HTTP requires bidirectional streams,
and this PR moves that complexity to HTTP. This is useful if HTTP were the
only application that did request-response, but request-response is an
incredibly common design pattern, and one that is naturally facilitated by
bidirectional streams. Making every application reimplement bidirectional
streams atop QUIC's unidirectional streams *adds* complexity overall. My
design sense says to optimize for the common case and to allow for the
exceptions. Given that our scope is to really get things rolling for HTTP
ASAP, I'd argue that request-response is the common case that we need to
optimize for.

This PR makes it even clearer to me that we need deployment experience with
unidirectional streams before going down this path. We have that with
bidirectional streams, which makes it worth standardizing.

To be clear, I'm not saying that unidirectional streams aren't interesting.
FWIW, the original input draft
<https://www.ietf.org/archive/id/draft-hamilton-quic-transport-protocol-01.txt>,
which is still how GQUIC works right now, has app read_close and
write_close transitions in the stream state machine which would allow a
stream to become unidirectional, but the application endpoints have to know
that these streams are unidirectional. We can explicitly specify a separate
bit in the STREAM frame that indicates uni/bi-directional stream, allowing
for unidirectional streams as "always half-closed" (bidirectional) streams.
Implementations can obviously optimize code for uni streams separately, but
the spec would be straightforward. I'm happy to write this up as an
alternative, since I think this is valuable.

- jana

On Wed, Jun 21, 2017 at 2:48 AM, Patrick McManus <[email-masked]>
wrote:

> I have come to strongly support unidirectional streams. I'm writing
> because I think we can take this just a tiny bit further and get a pretty
> big architectural win.
>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 22 Jun 2017 10:02:08 +1000

On 22 June 2017 at 09:48, Jana Iyengar <[email-masked]> wrote:
> (And right now in your PR, I don't think there's a way for an endpoint to
> RST a peer-initiated stream.)

Yes, that's clearly an oversight.  It's trivial to fix; I'll make the
necessary changes for that.

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 21 Jun 2017 17:08:01 -0700

(Adding a link to previous discussion on unidirectional streams: Issue #175
<https://github.com/quicwg/base-drafts/issues/175>)

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 22 Jun 2017 11:26:53 +1000

[ co-chair hat on ]

Everyone --

To add some context -- this is (obviously) a big change, so we've asked Martin to prioritise it so that people can give it due consideration early in the process, before people start implementing streams broadly.

As our charter says, "consensus is required both for changes to the current protocol mechanisms and retention of current mechanisms. In particular, because something is in the initial document set does not imply that there is consensus around the feature or around how it is specified." 

To get to that consensus in *either* direction, we need broad input on this, so please have a look through and express your support, concerns, etc. on-list. 

I'd like to see a substantial amount of on-list discussion before Prague, so that we can make a decision in that time frame.

Cheers,

> On 21 Jun 2017, at 5:41 pm, Martin Thomson <[email-masked]> wrote:
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Thu, 22 Jun 2017 11:50:09 +1000

On 22 June 2017 at 09:48, Jana Iyengar <[email-masked]> wrote:
> My design sense says to optimize for the common case and to allow for the

I think that this is a fair criticism, and I wanted to address it.

My concern is exactly the same as yours: I want get HTTP working.  And
unfortunately, HTTP/2 is not a simple request/response protocol.  This
is why HTTP/2 and now QUIC allow the server to initiate streams.  And
it's that exact gap that motivated me to propose this change.  In my
view, this change makes it *easier* to finish HTTP over QUIC.

The cost is that HTTP is marginally more complex.  My view is that the
overall protocol (HTTP+QUIC) is no more or less complex than before,
there are big savings in transport, and some minor savings in HTTP
that counteract the added cost: having to include explicit correlation
for request and response, and having to add a way to cancel requests
before the response starts. [1]

In terms of generality, we've already heard from Christian that this
is basically neutral for DNS, possibly with a tiny reduction in
complexity related to not having to check that redundant identifiers
in a response are consistent.

Ted suggested that we aim for a design that supports BOTH
unidirectional and bidirectional. Given how trivial it is to add an
identifier - as demonstrated by this PR - I don't see how building
that facility into the base transport is a net win.  Some protocols
can still use the stream identifiers as a correlator, for instance
those protocols that are properly request/response don't need explicit
correlators.  That choice - as with your proposal to retain
bidirectionality - creates a potential for head-of-line blocking under
certain circumstances, something that an explicit correlator at the
application layer neatly avoids.

[1] I don't consider HAS_BODY to be a cost, I'm now firmly of the
opinion that merging headers and body back into a single stream is the
right solution and that what I wrote up here is only temporary.

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 21 Jun 2017 19:18:17 -0700

> I think that this is a fair criticism, and I wanted to address it.

I'd like to hear your thoughts on the rest of it too (I think all my points
are fair :-))

> before the response starts. [1]

Supporting bidirectional streams is either complex or not: it doesn't
matter where you support it. If the complexity reduction is substantive in
transport, it can't be that the complexity increase in HTTP is marginal.
Your motivation is exactly my concern -- Server Push is a part of HTTP/2
for sure, but it is by no means the common case. Even though Server Push is
a part of the design, request-response dominates usage in HTTP/2.

We need to accomodate Server Push and make it work, which, as I suggested
in my earlier email, was supported in the early input draft (and in GQUIC).
The application can "half close" streams that are known to be
unidirectional (app read_close, app write_close). These transitions were
then removed during revisions as extraneous, since they were local to an
endpoint, but it seems that the core idea was lost as well. In the case of
HTTP/2, since server-initiated streams are always unidirectional (Push),
the application (HTTP) would always half-close them at both endpoints, not
requiring any special signaling on the wire.

If we want to make this more general, we could have a single
"uni/bi-directional" bit in every stream frame, so that we would not
require application knowledge of which streams are expected to be
uni/bi-directional.

> application layer neatly avoids.

Creating a new identifier in every application to correlate
request/response pairs adds complexity when you start adding applications.
Every app has to now explicitly design and implement this incredibly common
design pattern. Building common design patterns into the transport _is_ the
role of the transport. Otherwise, we could be building HTTP directly over
UDP, with no QUIC in the middle. The correlator is a feature, and one that
is required even in your design (although you have it in HTTP). I don't see
how it introduces HoL blocking, but I don't want to rathole on a minor
point.

I've made other points as well in my email which I think are important.

---

From: Mike Bishop <[email-masked]>
Date: Thu, 22 Jun 2017 06:32:04 +0000

As we've discussed, I highly approve moving in this direction, and this PR looks like a solid way of doing it.  Review with a few comments, but nothing I'd scream about.

I'll second Martin's opinion -- this doesn't make HTTP/QUIC noticeably more complex.  What it does, and what I suspect gives some people heartburn, is make it *less like HTTP/2*.  The more code reuse you thought you were going to be able to do, the more this makes you unhappy.  This also makes it less like TCP; in a way that's simple for a mapping to deal with, but it means the mapping actually needs thought.  (That's not entirely a bad thing.)

I've said it before, and I'll probably keep saying it:  HTTP/QUIC is not just HTTP/2 over QUIC.  It's similar in spirit, but QUIC is fundamentally different and HTTP/QUIC has to follow.  I'm all for code reuse where feasible, and concept reuse as broadly as possible, but let's not get stuck on net-positive changes because they're different from our existing code (QUIC *or* HTTP/2).

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Martin Thomson
Sent: Wednesday, June 21, 2017 6:50 PM
To: Jana Iyengar <[email-masked]>
Cc: QUIC WG <[email-masked]>; Patrick McManus <[email-masked]>
Subject: Re: Unidirectional streams PR

On 22 June 2017 at 09:48, Jana Iyengar <[email-masked]> wrote:
> My design sense says to optimize for the common case and to allow for 
> the exceptions. Given that our scope is to really get things rolling 
> for HTTP ASAP, I'd argue that request-response is the common case that 
> we need to optimize for.

My concern is exactly the same as yours: I want get HTTP working.  And unfortunately, HTTP/2 is not a simple request/response protocol.  This is why HTTP/2 and now QUIC allow the server to initiate streams.  And it's that exact gap that motivated me to propose this change.  In my view, this change makes it *easier* to finish HTTP over QUIC.

The cost is that HTTP is marginally more complex.  My view is that the overall protocol (HTTP+QUIC) is no more or less complex than before, there are big savings in transport, and some minor savings in HTTP that counteract the added cost: having to include explicit correlation for request and response, and having to add a way to cancel requests before the response starts. [1]

In terms of generality, we've already heard from Christian that this is basically neutral for DNS, possibly with a tiny reduction in complexity related to not having to check that redundant identifiers in a response are consistent.

Ted suggested that we aim for a design that supports BOTH unidirectional and bidirectional. Given how trivial it is to add an identifier - as demonstrated by this PR - I don't see how building that facility into the base transport is a net win.  Some protocols can still use the stream identifiers as a correlator, for instance those protocols that are properly request/response don't need explicit correlators.  That choice - as with your proposal to retain bidirectionality - creates a potential for head-of-line blocking under certain circumstances, something that an explicit correlator at the application layer neatly avoids.

[1] I don't consider HAS_BODY to be a cost, I'm now firmly of the opinion that merging headers and body back into a single stream is the right solution and that what I wrote up here is only temporary.

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 22 Jun 2017 10:58:35 +0000

We are interested in unidirectional streams and like the concept, I have reviewed the PR and made some editorial or nit comments.

This is a really interesting discussion, which I don’t have much to add to at the moment. To add a spectators opinion

On Wed, Jun 21, 2017 Jana Iyengar wrote:

> Creating a new identifier in every application to correlate request/response pairs adds complexity when you start adding applications. Every app has to now explicitly design and implement this incredibly common design pattern. Building common design patterns into the transport _is_ the role of the transport. Otherwise, we could be building HTTP directly over UDP, with no QUIC in the middle. The correlator is a feature, and one that is required even in your design (although you have it in HTTP). I don't see how it introduces HoL blocking, but I don't want to rathole on a minor point.

This sounds somewhat like a library/framework developer’s dilemma. How much “hand holding” should the transport do for the applications atop (the number of which is unbounded?). Sometimes providing too much “implementation” works counter to the specific neefd of the application. Therefore, would it suffice to describe the design pattern in the transport, perhaps in terms of considerations that an application mapping should make?

Regards
Lucas

---

From: Jo Kulik <[email-masked]>
Date: Thu, 22 Jun 2017 10:32:24 -0400

>
> This sounds somewhat like a library/framework developer’s dilemma. How
> much “hand holding” should the transport do for the applications atop (the
> number of which is unbounded?). Sometimes providing too much
> “implementation” works counter to the specific neefd of the application.
> Therefore, would it suffice to describe the design pattern in the
> transport, perhaps in terms of considerations that an application mapping
> should make?

I think this is an interesting idea.

I'm personally in favor of the unidirectional streams proposal.  (Emphasis
on personally, this is not some institutional opinion).  But this is
because I have previous experience with non-request-response protocols, and
I think they fit awkwardly over a conventional request-response-modelled
transport.  If there's a chance that some very simple changes could make
QUIC more viable for those protocols, then I think it's worth at least
discussing.

I feel cautious about the, "Our current goal is HTTP, design for the common
case" argument.  HTTP is the first goal.  But it's not the only goal, is
it?

To Jana's point about not designing a hammer with no specific nail to speak
of, gosh it would be helpful if we had, ideally more than one, non-HTTP
protocol in mind that we could discuss concretely.  I know that Martin
mentioned a few on his interim group slides.  But I have no idea whether
there were any stakeholders who are planning to invest in those protocols.
It would be helpful, when examining the costs/benefit to say, "This is how
this protocol fits over the current model, and this is how this protocol
would fit over the unidirectional model."

Right now, the only concrete protocols that we're discussing are H2 and
maybe DNS, is that connect?

On Thu, Jun 22, 2017 at 6:58 AM, Lucas Pardue <[email-masked]>
wrote:

> We are interested in unidirectional streams and like the concept, I have
> reviewed the PR and made some editorial or nit comments.
>
>
>
> This is a really interesting discussion, which I don’t have much to add to
> at the moment. To add a spectators opinion
>
>
>
> This sounds somewhat like a library/framework developer’s dilemma. How
> much “hand holding” should the transport do for the applications atop (the
> number of which is unbounded?). Sometimes providing too much
> “implementation” works counter to the specific neefd of the application.
> Therefore, would it suffice to describe the design pattern in the
> transport, perhaps in terms of considerations that an application mapping
> should make?
>
>
>
> Regards
>
> Lucas
>

---

From: Ian Swett <[email-masked]>
Date: Thu, 22 Jun 2017 12:16:24 -0400

This proposal does two things, which I don't believe need to be conflated:
 1) Adds support for unidirectional streams
 2) Removes support for bidirectional streams

I want better support in QUIC for unidirectional streams, for the reasons
laid out above and quite a few others(ie: RTP).  I believe this was the
original motivation behind this proposal, but after thinking about it a lot
and looking at the PR, I think removing support for bidirectional streams
is a huge error, because it just moves work from the transport layer to
each application layer.  I agree with Ted, that we should support both, and
I believe there are simple ways to support both, so I'll send out a PR that
proposes one later today.

If it's easy in QUIC to open a stream in either bidirectional or
unidirectional mode, then it should be easy and efficient to map virtually
every existing application written for TCP or UDP onto QUIC.  The extra
text in the RFC and extra code to support bidirectional streams vs
unidirectional is quite small.

As evidence that this doesn't result in a net simplification, even if we
only care about HTTP over QUIC, the PR adds 212 lines of text.  I'm sure
that could be minimized some, but I think it's going to be very difficult
to end up with a pure unidirectional design that's simpler than one which
supports both unidirectional and bidirectional streams.

On Thu, Jun 22, 2017 at 10:32 AM, Jo Kulik <[email-masked]> wrote:

> I feel cautious about the, "Our current goal is HTTP, design for the
> common case" argument.  HTTP is the first goal.  But it's not the only
> goal, is it?
>
> To Jana's point about not designing a hammer with no specific nail to
> speak of, gosh it would be helpful if we had, ideally more than one,
> non-HTTP protocol in mind that we could discuss concretely.  I know that
> Martin mentioned a few on his interim group slides.  But I have no idea
> whether there were any stakeholders who are planning to invest in those
> protocols.  It would be helpful, when examining the costs/benefit to say,
> "This is how this protocol fits over the current model, and this is how
> this protocol would fit over the unidirectional model."
>
>> This is a really interesting discussion, which I don’t have much to add
>> to at the moment. To add a spectators opinion
>>
>>
>>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 22 Jun 2017 12:14:03 -0700

This seems like it's going in the right direction. I've left a bunch of
extensive comments
on the PR for minor issues. Substantive comments below.

> simplify streams a lot.  That it also makes the transport more generic

As I noted, I think it would be good to present the FIN/RST states
separately.
Now that you have made the state machine simpler, there's room to surface
some of the substates upwared.

> or a promise for a body.  I would like to eliminate this stream split.

Yes, I think that would be a good idea. It seems oddly artificial.

With that said, if you're not going to do that, I think I would eliminate
HAS_BODY
in favor of a pseudo-header.

> need to fix, but we haven't reached the point where we agree on the

It seems like there are two semantics here:

1. I stop sending me data
2. I don't care about this and I'm not responding.

As you say, for headers #1 isn't much of an issue and for #2, it seems like
an HTTP response that says "no" is just as efficient.

-Ekr

---

From: Ted Hardie <[email-masked]>
Date: Thu, 22 Jun 2017 12:15:14 -0700

On Wed, Jun 21, 2017 at 6:50 PM, Martin Thomson <[email-masked]>
wrote:

> that facility into the base transport is a net win.

If you have a protocol that has some unidirectional streams and some
bidirectional streams, it provides a simple facility for indicating which
streams have which characteristic.

In general, you can make a bidrectional stream model look like a
unidirectional stream model (the half-closed method is an example of
this).  You can make a unidirectional stream model look like a
bidirectional stream model (correlators are an example of this). Both mean
that some of the core characteristics of the base model aren't present in
some cases.  The key question is:  how do you identify when that is the
case?

My personal take on that right answer there is to support both, with an
explicit signal of when each model is in use, even for application
protocols that use only bidirectional or unidirectional stream types.  It's
not like we haven't seen app protocols grow to incorporate bidirectional
flows in previously unidirectional stream models (arguably RTCP with RTP)
or vice versa (one way to look at server push in HTTP/2).  This is a one
bit signal of a very basic property, there are lots of better places to
optimize than avoiding that spend.  It's moderately obvious, that we will
use both models in applications running on QUIC; what's th harm to an
explicit signal and support?

My personal opinion only,

Ted

> Some protocols

---

From: Wenbo Zhu <[email-masked]>
Date: Thu, 22 Jun 2017 14:59:18 -0700

My personal opinions.

1) is much harder to add later. QUIC will definitely be used by non-RPC
protocols.

It would be useful to see a concrete example on how to map something like
HTTP request/response to QUIC uni-streams. Could Mike publish two reference
versions, before/after the PR is Incorporated to the HTTP mapping spec?

Most L7 (application) protocols will keep using HTTP/* as the transport.
Other L7 protocols will have their own state machines on top of the QUIC
streams to manage the client-server conversation, inc. flow-control. The
less state the QUIC layer maintains, the better IMO unless other
transport-level features may benefit from the bidi model. I don't have any
specific feature in mind.

On Thu, Jun 22, 2017 at 9:16 AM, Ian Swett <[email-masked]> wrote:

>> I'm personally in favor of the unidirectional streams proposal.
>>  (Emphasis on personally, this is not some institutional opinion).  But
>> this is because I have previous experience with non-request-response
>> protocols, and I think they fit awkwardly over a conventional
>> request-response-modelled transport.  If there's a chance that some very
>> simple changes could make QUIC more viable for those protocols, then I
>> think it's worth at least discussing.
>>

---

From: Martin Thomson <[email-masked]>
Date: Fri, 23 Jun 2017 08:32:25 +1000

On 23 June 2017 at 05:15, Ted Hardie <[email-masked]> wrote:
> explicit signal of when each model is in use,

I have heard this request now from several folks at Google.  I don't
know how to make this work without increasing complexity considerably
.  I'd be willing to be wrong about this, but would prefer to see a
proposal.  I don't need a fully-worked proposal with text, just a
sketch of how the pieces work in enough detail to understand how this
might interact with other parts of the protocol.  I think that Jana
offered to do this, so maybe I can wait for that (c.f., Mark's earlier
statement about priority/urgency).

---

From: Ted Hardie <[email-masked]>
Date: Thu, 22 Jun 2017 15:46:47 -0700

On Thu, Jun 22, 2017 at 3:32 PM, Martin Thomson <[email-masked]>
wrote:

> I have heard this request now from several folks at Google.

Please be careful withyour attributions.  Several people replied with very
explicit statements that the were putting forward personal opinions.  Some
of those people, including those with different opinions, have day jobs at
the same place, but we were not speaking for our employer nor with the same
experience (either at that employer or elsewhere).

thanks,

Ted

> I don't

---

From: Mike Bishop <[email-masked]>
Date: Thu, 22 Jun 2017 22:47:43 +0000

I suppose you could drop the stream type header from HTTP into QUIC and make the types "bidirectional stream," "unidirectional stream," and "mate of bidirectional stream"; the last would further need to identify which peer stream it corresponded to.

But making them the first couple bytes of the stream data feels kind of kludgy if this is fundamentally built into the transport, which means they really belong in a dedicated frame.  Maybe a dedicated OPEN_STREAM frame, which blocks any data received on that stream until you know what type it is?

Actually, if we're planning on introducing other per-stream properties like partial reliability, there might be value in an OPEN_STREAM frame where we can describe a stream's desired properties before any data flows....

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Martin Thomson
Sent: Thursday, June 22, 2017 3:32 PM
To: Ted Hardie <[email-masked]>
Cc: Jana Iyengar <[email-masked]>; QUIC WG <[email-masked]>; Patrick McManus <[email-masked]>
Subject: Re: Unidirectional streams PR

On 23 June 2017 at 05:15, Ted Hardie <[email-masked]> wrote:
> My personal take on that right answer there is to support both, with 
> an explicit signal of when each model is in use,

I have heard this request now from several folks at Google.  I don't know how to make this work without increasing complexity considerably .  I'd be willing to be wrong about this, but would prefer to see a proposal.  I don't need a fully-worked proposal with text, just a sketch of how the pieces work in enough detail to understand how this might interact with other parts of the protocol.  I think that Jana offered to do this, so maybe I can wait for that (c.f., Mark's earlier statement about priority/urgency).

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 22 Jun 2017 16:54:41 -0700

> experience (either at that employer or elsewhere).

+1.
(To be clear, the reason I'm agreeing with Ted here is not because I'm at
Google.)

thanks,

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 22 Jun 2017 17:04:23 -0700

> statement about priority/urgency).

Ian offered to do this in an earlier email, so I'd like to wait for him.
But I already outlined some of my thinking earlier. Copying and pasting
from earlier email in this thread:

"We need to accomodate Server Push and make it work, which, as I suggested
in my earlier email, was supported in the early input draft
<https://tools.ietf.org/html/draft-hamilton-quic-transport-protocol-01#section-8.1>
(and in GQUIC). The application can "half close" streams that are known to
be unidirectional (app read_close, app write_close). These transitions were
then removed during revisions as extraneous, since they were local to an
endpoint, but it seems that the core idea was lost as well. In the case of
HTTP/2, since server-initiated streams are always unidirectional (Push),
the application (HTTP) would always half-close them at both endpoints, not
requiring any special signaling on the wire.

uni/bi-directional."

I'd like to wait for Ian's PR and discuss details there, but I believe his
core idea is the same.

---

From: Ryan Hamilton <[email-masked]>
Date: Thu, 22 Jun 2017 17:14:52 -0700

Thanks for writing this up!

I'm concerned that while this appears to simplify the transport draft, it
actually ends up a net increase in complexity. By moving the burden of
associating the "request stream" and the "response stream" up to the
application, say HTTP, we require that each application implement an
association mechanism independently. This means that the
logic/code/complexity that we remove from transport draft simply moves to
the HTTP draft and is then going to be *duplicated* in every other
app-over-QUIC mapping draft which want to do bidirectional streaming.

I am also concerned that there is no deployment experience with a
unidirectional-stream-only API. While I have no doubt that it would be
possible to build such a system, without experience to guide us we are
likely to be in the dark as to what dragons lurk in the corners. I would be
happy to see unidirectional streams added in addition to bidirectional
streams but am not supportive of removing bidirectional streams.

That being said, I do really like the idea of HTTP PUSH_PROMISE promising a
"push ID" instead of a stream ID so as to remove the complexity around
stream limits/reserved states. I would like to see this added to the
mapping independent of the unidirectional stream change as I think it's an
elegant solution to a thorny problem. (QUIC used to have a reserved state
which promised streams entered before they were open which avoided hitting
the open stream limits and allowed servers to promise more streams than it
could currently create (say to push all of the thumbnails on a page of
image search results. When the reserved state was removed from the spec, we
lost this ability, which as we know is a bummer)

Cheers,

Ryan

On Jun 21, 2017 12:42 AM, "Martin Thomson" <[email-masked]> wrote:

## Transport Changes

## HTTP Changes

### Stream Correlation

### Prioritization

### Cancelling Pushes

# Fixes

---

From: Martin Thomson <[email-masked]>
Date: Fri, 23 Jun 2017 10:20:22 +1000

On 23 June 2017 at 10:14, Ryan Hamilton <[email-masked]> wrote:
> When the reserved state was removed from the spec, we lost this ability

Removing reserved didn't change anything here, unless I missed
something.  The server still needs to commit to a stream ID for
pushing.  The stream limits still applied when the server wanted to
use the stream.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 01:06:40 +0000

I like the idea of unidirectional streams, if they make life easier for apps -- both "traditional" apps like H2 as well as less usual apps like ssh (one stream in one direction, two associated streams in the opposite direction).

I also like Mike's idea of OPEN_STREAM frame.  The way I'd describe OPEN_STREAM frame is that it is implicitly a STREAM frame with 0-offset and a set of "per-stream properties".

An important point is that when Transport is aware of the association of two unidirectional streams, the Transport API would be able to implement a bidirectional socket-like abstraction for applications that desire one.

OPEN_STREAM frame
The type byte for a OPEN_STREAM frame contains embedded flags, and is formatted as 10FSSPPD. The F, SS, and D bits are parsed as per STREAM frame.
* The PP bits encode the size of the Params Bitmap field. The values 00, 01, 02, and 03 indicate lengths of 8, 16, 24, and 32 bits long respectively.

The two Least Significant bits of the Params Bitmap indicate the length of the Associated Stream ID. The LSB values 00, 01, 02, and 03 indicate lengths of 0 (none), 8, 16, and 32 bits long respectively.
We can define up to 30 additional bits to be meaningful to the transport.  (I can imagine Partial-Reliability bit, Flow-Control-Exempt bit, Priority-Delivery (aka "DontBuffer") bit, etc).

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Stream ID (8/16/24/32)                   ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                Params Bitmap  (8/16/24/32)                   ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|            Associated Stream ID (0/8/16/32)                    ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|       [Data Length (16)]      |        Stream Data (*)      ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note that if Params Bitmap is 0x0, this OPEN_STREAM frame is equivalent to STREAM frame with OO=0.

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 22 Jun 2017 18:25:40 -0700

On Thu, Jun 22, 2017 at 6:06 PM, Lubashev, Igor <[email-masked]> wrote:

> I like the idea of unidirectional streams, if they make life easier for
> apps -- both "traditional" apps like H2 as well as less usual apps like ssh
> (one stream in one direction, two associated streams in the opposite
> direction).
>
> I also like Mike's idea of OPEN_STREAM frame.  The way I'd describe
> OPEN_STREAM frame is that it is implicitly a STREAM frame with 0-offset and
> a set of "per-stream properties".
>
> An important point is that when Transport is aware of the association of
> two unidirectional streams, the Transport API would be able to implement a
> bidirectional socket-like abstraction for applications that desire one.
>

I'm not sure I follow your thinking here: are you suggesting that QUIC
implement bidirectional streams in addition to unidirectional streams?

- jana

>
> OPEN_STREAM frame
> The type byte for a OPEN_STREAM frame contains embedded flags, and is
> formatted as 10FSSPPD. The F, SS, and D bits are parsed as per STREAM frame.
> * The PP bits encode the size of the Params Bitmap field. The values 00,
> 01, 02, and 03 indicate lengths of 8, 16, 24, and 32 bits long respectively.
>
> The two Least Significant bits of the Params Bitmap indicate the length of
> the Associated Stream ID. The LSB values 00, 01, 02, and 03 indicate
> lengths of 0 (none), 8, 16, and 32 bits long respectively.
> We can define up to 30 additional bits to be meaningful to the transport.
> (I can imagine Partial-Reliability bit, Flow-Control-Exempt bit,
> Priority-Delivery (aka "DontBuffer") bit, etc).
>
> 0                   1                   2                   3
> I suppose you could drop the stream type header from HTTP into QUIC and
> make the types "bidirectional stream," "unidirectional stream," and "mate
> of bidirectional stream"; the last would further need to identify which
> peer stream it corresponded to.
>
> But making them the first couple bytes of the stream data feels kind of
> kludgy if this is fundamentally built into the transport, which means they
> really belong in a dedicated frame.  Maybe a dedicated OPEN_STREAM frame,
> which blocks any data received on that stream until you know what type it
> is?
>
> Actually, if we're planning on introducing other per-stream properties
> like partial reliability, there might be value in an OPEN_STREAM frame
> where we can describe a stream's desired properties before any data
> flows....
>
> I have heard this request now from several folks at Google.  I don't know
> how to make this work without increasing complexity considerably .  I'd be
> willing to be wrong about this, but would prefer to see a proposal.  I
> don't need a fully-worked proposal with text, just a sketch of how the
> pieces work in enough detail to understand how this might interact with
> other parts of the protocol.  I think that Jana offered to do this, so
> maybe I can wait for that (c.f., Mark's earlier statement about
> priority/urgency).
>
>

---

From: Ian Swett <[email-masked]>
Date: Thu, 22 Jun 2017 21:43:06 -0400

As promised, here is my 1 bit extension to the existing QUIC STREAM frame
to support unidirectional streams in addition to bidirectional streams.  It
uses an extra bit in the type byte to indicate a STREAM frame is
unidirectional, and hence the stream is to be opened in a
half-closed(unidirectional) state.

There are many ways to frame this, which I'm happy to discuss, but the key
idea is that adding unidirectional streams to the existing design does not
add much complexity, and provides a more flexible API to use cases
including server push and RTP.

https://github.com/quicwg/base-drafts/pull/656

PS: This does not solve the issue Ryan identified above with PUSH_PROMISE
transitioning to a push id to avoid using up open stream limits.  I think
that should be separable and easy to put on top of this PR as well.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 01:49:51 +0000

I think of three layers of abstraction:

1.       QUIC Wire Protocol (the thing described by the QUIC Transport RFC)

2.       QUIC Library API (a library exposing some useful abstractions -- such as blocking/non-blocking unidirectional streams and bidirectional “sockets” -- and implementing them using QUIC Wire Protocol)

3.       Application (something that uses QUIC Library APIs)

What I am saying is that Martin’s unidirectional streams proposal does not have a way to implement generic bidirectional socket-like abstractions on the QUIC Library API layer.  However, if we follow Mike’s suggestion and allow an Associated Stream ID in OPEN_STREAM frames, this would make it easy for QUICK Libraries to provide generic bidirectional socket-like abstractions.

So what I am suggesting is that we keep QUIC Wire Protocol state machine simple and unchanged.  However, with the OPEN_STREAM frame, we would allow QUIC Libraries to implement both unidirectional and bidirectional streams.

-          Igor

From: Jana Iyengar [mailto:[email-masked]]
Sent: Thursday, June 22, 2017 9:26 PM
To: Lubashev, Igor <[email-masked]>
Cc: Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>; Patrick McManus <[email-masked]>
Subject: Re: Unidirectional streams PR

On Thu, Jun 22, 2017 at 6:06 PM, Lubashev, Igor <[email-masked]<mailto:[email-masked]>> wrote:
I like the idea of unidirectional streams, if they make life easier for apps -- both "traditional" apps like H2 as well as less usual apps like ssh (one stream in one direction, two associated streams in the opposite direction).

I'm not sure I follow your thinking here: are you suggesting that QUIC implement bidirectional streams in addition to unidirectional streams?

- jana

OPEN_STREAM frame

0                   1                   2                   3

---

From: Ian Swett <[email-masked]>
Date: Thu, 22 Jun 2017 22:09:49 -0400

On Thu, Jun 22, 2017 at 9:49 PM, Lubashev, Igor <[email-masked]> wrote:

> 2.       QUIC Library API (a library exposing some useful abstractions --
> such as blocking/non-blocking unidirectional streams and bidirectional
> “sockets” -- and implementing them using QUIC Wire Protocol)
>
> What I am saying is that Martin’s unidirectional streams proposal does not
> have a way to implement generic bidirectional socket-like abstractions on
> the QUIC Library API layer.  However, if we follow Mike’s suggestion and
> allow an Associated Stream ID in OPEN_STREAM frames, this would make it
> easy for QUICK Libraries to provide generic bidirectional socket-like
> abstractions.
>
>
>
> So what I am suggesting is that we keep QUIC Wire Protocol state machine
> simple and unchanged.  However, with the OPEN_STREAM frame, we would allow
> QUIC Libraries to implement both unidirectional and bidirectional streams.
>
>
>
> -          Igor
>
> Igor, I think you and I are in agreement that we both want a relatively
easy way of doing both unidirectional streams as well as bidirectional
streams?  Given that, it comes down to what to standardize at the transport
layer and some framing.

In regards to the unidirectional only model:
 1) The QUIC stream state machine is really not that complex, and the
current documentation does a nice job of explaining it.  Changing to
unidirectional only streams only saves 2 states, from 5 to 3, and a small
bit of text.
 2) I'm very concerned that if we don't standardize a way to do
bidirectional streams, different 'QUIC Library API's will internally
standardize on different approaches for recreating bidirectional streams.
I fear this may create more work and interoperability challenges down the
road.

So if we think literally no one will have any use for the bidirectional
stream model, then I agree we should remove it.  But even today, the HTTP
mapping is simpler with the bidirectional stream model than with only
unidirectional streams, which means the total amount of work of
implementing HTTP + QUIC is about the same either way, and I believe if
bidirectional streams and unidirectional streams do exist in QUIC(as in my
PR above), then the HTTP would/should use both.

>
>
>
>
> *From:* Jana Iyengar [mailto:[email-masked]]
> *Sent:* Thursday, June 22, 2017 9:26 PM
> *To:* Lubashev, Igor <[email-masked]>
> *Cc:* Mike Bishop <[email-masked]>; Ted Hardie <
> [email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <
> [email-masked]>; Patrick McManus <[email-masked]>
>
> *Subject:* Re: Unidirectional streams PR
>
>
>

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 02:43:22 +0000

Yes, we are in an agreement that we want “QUIC Libraries” to provide an easy way of doing both unidirectional streams as well as bidirectional streams.

I do not have a strong preference as to the details of framing. On one hand, I am not worried about libraries implementing bidirectional streams differently with OPEN_STREAM – there is really just one sane way of doing it, and the OPEN_STREAM model is more flexible.  On the other hand, OPEN_STREAM comes at a cost of a few extra bytes for a simple bidirectional stream implementation.

The UNI does complicate the state machine a bit by adding an extra state, as ekr pointed out.  You need a special state for implicitly opened streams.  They are not “idle” (they count toward the max), and they are not “open” (you cannot write to them), and they are not “local half-closed” (they may transition to “open”).  I do not think this is a huge deal, though – TCP has a lot more states!

From: Ian Swett [mailto:[email-masked]]
Sent: Thursday, June 22, 2017 10:10 PM
To: Lubashev, Igor <[email-masked]>
Cc: Jana Iyengar <[email-masked]>; Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>; Patrick McManus <[email-masked]>
Subject: Re: Unidirectional streams PR

On Thu, Jun 22, 2017 at 9:49 PM, Lubashev, Igor <[email-masked]<mailto:[email-masked]>> wrote:
I think of three layers of abstraction:

-          Igor
Igor, I think you and I are in agreement that we both want a relatively easy way of doing both unidirectional streams as well as bidirectional streams?  Given that, it comes down to what to standardize at the transport layer and some framing.

In regards to the unidirectional only model:
 1) The QUIC stream state machine is really not that complex, and the current documentation does a nice job of explaining it.  Changing to unidirectional only streams only saves 2 states, from 5 to 3, and a small bit of text.
 2) I'm very concerned that if we don't standardize a way to do bidirectional streams, different 'QUIC Library API's will internally standardize on different approaches for recreating bidirectional streams.  I fear this may create more work and interoperability challenges down the road.

So if we think literally no one will have any use for the bidirectional stream model, then I agree we should remove it.  But even today, the HTTP mapping is simpler with the bidirectional stream model than with only unidirectional streams, which means the total amount of work of implementing HTTP + QUIC is about the same either way, and I believe if bidirectional streams and unidirectional streams do exist in QUIC(as in my PR above), then the HTTP would/should use both.

---

From: Mike Bishop <[email-masked]>
Date: Fri, 23 Jun 2017 03:18:29 +0000

Interesting design.  I was trying to avoid doubling the number of frames consumed by the STREAM frame by consuming just one.  But it's a good insight that offset is only absent on the first STREAM frame, so we could leverage that if we wanted to.  Maybe OO=0 indicates the presence of some fixed-size params, and one of those bits is directionality?

I wasn't convinced this is simpler, because HTTP/QUIC would still need a stream header.  But if we used bidirectional streams for request/response, then all unidirectional streams are PUSH_PROMISE, and they're the only ones that need a stream header.

However, let me point out that this is fundamentally still unidirectional streams with the ability to declare (as metadata) that a particular stream is "affiliated" with a stream in the opposite direction.  (And perhaps the ability for the stream initiator to declare an expectation that there will be an affiliation coming.)  This does change the request on N / response on N model.  We're just talking about moving the correlator into the QUIC framing, so that each mapping doesn't have to reconstruct how to do correlations.

That means we get (or can get) the same state simplifications as in the PR by considering the streams to have independent lifecycles -- the correlator is simply made available to the application as information about the stream.  That also means that adding the correlator can be done in a separate PR, if we prefer.

-----Original Message-----
From: Lubashev, Igor [mailto:[email-masked]] 
Sent: Thursday, June 22, 2017 6:07 PM
To: Mike Bishop <[email-masked]>; Martin Thomson <[email-masked]>; Ted Hardie <[email-masked]>
Cc: Jana Iyengar <[email-masked]>; QUIC WG <[email-masked]>; Patrick McManus <[email-masked]>
Subject: RE: Unidirectional streams PR

OPEN_STREAM frame

0                   1                   2                   3

---

From: Ryan Hamilton <[email-masked]>
Date: Thu, 22 Jun 2017 20:46:09 -0700

On Thu, Jun 22, 2017 at 8:18 PM, Mike Bishop <[email-masked]>
wrote:

> Interesting design.  I was trying to avoid doubling the number of frames
> consumed by the STREAM frame by consuming just one.  But it's a good
> insight that offset is only absent on the first STREAM frame, so we could
> leverage that if we wanted to.  Maybe OO=0 indicates the presence of some
> fixed-size params, and one of those bits is directionality?
>
> I wasn't convinced this is simpler, because HTTP/QUIC would still need a
> stream header.  But if we used bidirectional streams for request/response,
> then all unidirectional streams are PUSH_PROMISE, and they're the only ones
> that need a stream header.
>
> However, let me point out that this is fundamentally still unidirectional
> streams with the ability to declare (as metadata) that a particular stream
> is "affiliated" with a stream in the opposite direction.  (And perhaps the
> ability for the stream initiator to declare an expectation that there will
> be an affiliation coming.)  This does change the request on N / response on
> N model.  We're just talking about moving the correlator into the QUIC
> framing, so that each mapping doesn't have to reconstruct how to do
> correlations.
>

​That's a clever observation. I think there's one slight difference though,
as it applies to stream limits. In the bidi case, a peer can open a bidi
stream if it has not yet exceeded the peer's limit, and the peer is allowed
to respond, even if the peer could not otherwise open a new stream because
of stream limits. In addition, with the bidi steam case, the stream is
considered open until both halves are closed, which would not be the case
with uni streams. (Though with the changes to stream limits this might not
actually be a terribly important distinction)

> That means we get (or can get) the same state simplifications as in the PR
> by considering the streams to have independent lifecycles -- the correlator
> is simply made available to the application as information about the
> stream.  That also means that adding the correlator can be done in a
> separate PR, if we prefer.
>
> To: Mike Bishop <[email-masked]>; Martin Thomson <
> [email-masked]>; Ted Hardie <[email-masked]>

---

From: Christian Huitema <[email-masked]>
Date: Thu, 22 Jun 2017 21:30:18 -0700

On 6/22/2017 7:09 PM, Ian Swett wrote:

> unidirectional only streams only saves 2 states, from 5 to 3, and a
> small bit of text.
> standardize on different approaches for recreating bidirectional
> streams.  I fear this may create more work and interoperability
> challenges down the road.

I review both PR (#643 and #656). I think they are pretty much
equivalent, and will result in the same number of messages on the wire.
I have a slight preference for Ian's design because it uses a single
stream number space for both client and server originated streams. I am
concerned that two independent number spaces will lead to ambiguities
and implementation errors. This is visible in the current text around
RESET_STREAM. If we have two number spaces, we need to split that in two
frames, RESET_MY_STREAM and TAKE_YOUR_STREAM_AWAY.

Of course, Martin could also update #643 to use a single stream number...

-- 
Christian Huitema

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 05:08:46 +0000

If you want to add information only to the initial STREAM frame without doubling the number of frames for STREAM, you have some options (each with a tradeoff).

1. As you said, initial STREAM frame (OO=0) can have an additional fixed field.

2. If you need to attach a single bit of information for the initial STREAM frame only occasionally, you can pay 1 byte when that bit of information is present.  (Use the fact that OO=0 and OO=01,Offset=0 are equivalent, so you can overload OO=01,Offset=0 with additional semantics.)  That bit can indicate "UNI" or it can indicate the presence of a fixed field after the Stream ID.

-----Original Message-----
From: Mike Bishop [mailto:[email-masked]] 
Sent: Thursday, June 22, 2017 11:18 PM
To: Lubashev, Igor <[email-masked]>; Martin Thomson <[email-masked]>; Ted Hardie <[email-masked]>
Cc: Jana Iyengar <[email-masked]>; QUIC WG <[email-masked]>; Patrick McManus <[email-masked]>
Subject: RE: Unidirectional streams PR

..

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 05:31:38 +0000

Ian, I am not in love with a possibility of having UNI flag clear on some initial STREAM frames and then UNI set on subsequent STREAM frames.  The PR does not prohibit this.  I read the PR (state diagram) to require the stream to go immediately into “half-closed (local)” state on receipt of such STREAM+UNI frame.  This is a problem, since some packets from the receiver to the sender could be in flight, and absent a STREAM+FIN or RST_STREAM from the receiver, the sender cannot do flow control accounting.

It seems best if a stream is UNI or BI from birth, and that’s indicated by the initial STREAM frame only.  If STREAM frames get reordered, until the receiver receives the initial STREAM frame (Offset=0), it should consider the stream to be in “implicit” state (do accounting for max stream count and flow control but cannot send data to it).

From: Lubashev, Igor [mailto:[email-masked]]
Sent: Thursday, June 22, 2017 10:43 PM
To: Ian Swett <[email-masked]>
Cc: Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; Patrick McManus <[email-masked]>; Jana Iyengar <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>
Subject: RE: Unidirectional streams PR

From: Ian Swett [mailto:[email-masked]]
Sent: Thursday, June 22, 2017 10:10 PM
To: Lubashev, Igor <[email-masked]<mailto:[email-masked]>>
Cc: Jana Iyengar <[email-masked]<mailto:[email-masked]>>; Mike Bishop <[email-masked]<mailto:[email-masked]>>; Ted Hardie <[email-masked]<mailto:[email-masked]>>; QUIC WG <[email-masked]<mailto:[email-masked]>>; Martin Thomson <[email-masked]<mailto:[email-masked]>>; Patrick McManus <[email-masked]<mailto:[email-masked]>>
Subject: Re: Unidirectional streams PR

-          Igor

---

From: Martin Thomson <[email-masked]>
Date: Fri, 23 Jun 2017 16:43:43 +1000

On 23 June 2017 at 14:30, Christian Huitema <[email-masked]> wrote:
> This is visible in the current text around

Note that I think we have some agreement to do this anyway, I just
decided to leave that to #171 or something that replaces it.

> Of course, Martin could also update #643 to use a single stream number...

I considered that, and could easily do so.  I would prefer to have two
spaces, but it's not a central point.

---

From: "Philipp S. Tiesel" <[email-masked]>
Date: Fri, 23 Jun 2017 12:16:46 +0200

> On 22. Jun 2017, at 03:50, Martin Thomson <[email-masked]> wrote:
> 

> right solution and that what I wrote up here is only temporary.

I have some concerns here, as separating the body into a different stream
allows more flexible design later on: If we added unreliable transport to
quick, having the header sent reliably and the body unreliably seems useful.

Canceling the stream is still possible if a) the stream ID is known by
convention or b) the stream ID is provided in a forward-reference.

> application layer neatly avoids.

I guess having a OPEN_STREAM frame that allows to associate a new steam
to a stream in reverse direction and thus building a a bi-directional
stream pair is a good compromise as it allows bi-directinal streams in
a quic API without introducing additional state space in the base protocol.

AVE!
  Philipp S. Tiesel / phils…

---

From: Ian Swett <[email-masked]>
Date: Fri, 23 Jun 2017 15:59:37 -0400

Igor,

In terms of the extra state, I believe EKR's right that this either creates
a need to change some text or add another state, which I hadn't
anticipated, but also isn't the end of the world.  However, Buck Krasic
pointed out we can now get rid of implicitly opened streams, because we've
transitioned from stream limits to max stream ID, which fixes this problem
in a simple way.  See issue #662
<https://github.com/quicwg/base-drafts/issues/662>.

I attempted to add text to specify that if a STREAM frame specifies UNI on
one packet, it must do so on all of them, but the text wasn't written
correctly, so I'll fix that in the PR.  I did consider whether to mark all
STREAM frames or only the first.  In the end, I proposed to mark all of
them because it's more resilient to reordering and it's easy to document
and understand.  I'm certainly open to other framings.

On Fri, Jun 23, 2017 at 1:31 AM, Lubashev, Igor <[email-masked]> wrote:

> Ian, I am not in love with a possibility of having UNI flag clear on some
> initial STREAM frames and then UNI set on subsequent STREAM frames.  The PR
> does not prohibit this.  I read the PR (state diagram) to require the
> stream to go immediately into “half-closed (local)” state on receipt of
> such STREAM+UNI frame.  This is a problem, since some packets from the
> receiver to the sender could be in flight, and absent a STREAM+FIN or
> RST_STREAM from the receiver, the sender cannot do flow control accounting.
>
>
>
> It seems best if a stream is UNI or BI from birth, and that’s indicated by
> the initial STREAM frame only.  If STREAM frames get reordered, until the
> receiver receives the initial STREAM frame (Offset=0), it should consider
> the stream to be in “implicit” state (do accounting for max stream count
> and flow control but cannot send data to it).
>
>
>
>
>
> *From:* Lubashev, Igor [mailto:[email-masked]]
> *Sent:* Thursday, June 22, 2017 10:43 PM
> *To:* Ian Swett <[email-masked]>
> [email-masked]>; Patrick McManus <[email-masked]>; Jana Iyengar
> <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <
> [email-masked]>
> *Subject:* RE: Unidirectional streams PR
>
>
>
> Yes, we are in an agreement that we want “QUIC Libraries” to provide an
> streams.
> I do not have a strong preference as to the details of framing. On one
> hand, I am not worried about libraries implementing bidirectional streams
> differently with OPEN_STREAM – there is really just one sane way of doing
> it, and the OPEN_STREAM model is more flexible.  On the other hand,
> OPEN_STREAM comes at a cost of a few extra bytes for a simple bidirectional
> stream implementation.
>
>
>
> The UNI does complicate the state machine a bit by adding an extra state,
> as ekr pointed out.  You need a special state for implicitly opened
> streams.  They are not “idle” (they count toward the max), and they are not
> “open” (you cannot write to them), and they are not “local half-closed”
> (they may transition to “open”).  I do not think this is a huge deal,
> though – TCP has a lot more states!
>
>
>
>
>
> *From:* Ian Swett [mailto:[email-masked] <[email-masked]>]
> *Sent:* Thursday, June 22, 2017 10:10 PM
> *Cc:* Jana Iyengar <[email-masked]>; Mike Bishop <
> [email-masked]>; Ted Hardie <[email-masked]>; QUIC WG <
> [email-masked]>; Martin Thomson <[email-masked]>; Patrick
> McManus <[email-masked]>

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Jun 2017 21:59:30 +0000

Yes, getting rid of implicitly open streams can work.  To get rid of “implicit” state entirely with your design, you would need to keep UNI bit on all packets.  Otherwise, reordering of the initial STREAM and a subsequent STREAM would require the receiver to enter “implicit” state for the stream (stream is no longer idle, but one cannot assume it is bidirectional).

Sent: Friday, June 23, 2017 4:00 PM

Igor,

In terms of the extra state, I believe EKR's right that this either creates a need to change some text or add another state, which I hadn't anticipated, but also isn't the end of the world.  However, Buck Krasic pointed out we can now get rid of implicitly opened streams, because we've transitioned from stream limits to max stream ID, which fixes this problem in a simple way.  See issue #662<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_662&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=gv2nW8KsHS6Lu2KcNFaw91ojaVQERu816iviQ20Vwkc&s=6kSCQxFc0DF2n1FyODQcprN8J8KATca2FHRLEm02adc&e=>.

I attempted to add text to specify that if a STREAM frame specifies UNI on one packet, it must do so on all of them, but the text wasn't written correctly, so I'll fix that in the PR.  I did consider whether to mark all STREAM frames or only the first.  In the end, I proposed to mark all of them because it's more resilient to reordering and it's easy to document and understand.  I'm certainly open to other framings.

On Fri, Jun 23, 2017 at 1:31 AM, Lubashev, Igor <[email-masked]<mailto:[email-masked]>> wrote:
Ian, I am not in love with a possibility of having UNI flag clear on some initial STREAM frames and then UNI set on subsequent STREAM frames.  The PR does not prohibit this.  I read the PR (state diagram) to require the stream to go immediately into “half-closed (local)” state on receipt of such STREAM+UNI frame.  This is a problem, since some packets from the receiver to the sender could be in flight, and absent a STREAM+FIN or RST_STREAM from the receiver, the sender cannot do flow control accounting.

---

From: Ian Swett <[email-masked]>
Date: Fri, 23 Jun 2017 20:40:41 -0400

ekr and I were chatting about the high level design options here.  There
are a lot of specific details to both PRs which may not be ideal, but no
matter what, the HTTP mapping(and future apps) are going to need both
bidirectional and unidirectional streams, so it's just a matter of where
that functionality is defined and implemented.

We came up with 4 options(I'm sure there are sub-options here, but bear
with me):
  1) Do nothing, let the application(ie: HTTP) create unidirectional
streams by closing one half of the bidi stream.
  2) Add unidirectional streams as an option to existing bidirectional
streams. (my PR is a proof of concept of that)
  3) Change the transport text to describe unidirectional streams and add a
mechanism in the transport doc to pair them into bidi streams.
  4) Replace bidirectional streams with unidirectional streams in the
transport doc and make transports implement bidi streams if they need them.
(Martin's PR is an example of this)

I think Martin and my PR's demonstrate that #2 and #4 are possible options,
and I suspect it would be straightforward to draft an example PR for #3 if
someone desired.

I want to keep bidirectional streaming in the transport doc and wrote up 2
because I felt it was simpler to implement and reason about than 3.  I
don't believe anyone is a strong supporter of #1 (and if you are, now would
be an excellent time to speak up), which leaves the WG choosing between 2,
3 and 4 as architectural directions.

I'm going to update my PR to try to fix the issues identified in the
comments when I can this weekend, but I think the critical question for the
WG is which direction(s) we'd like to spend time refining, and are people
clamoring for an example PR representing option 3?

On Fri, Jun 23, 2017 at 5:59 PM, Lubashev, Igor <[email-masked]> wrote:

> Yes, getting rid of implicitly open streams can work.  To get rid of
> “implicit” state entirely with your design, you would need to keep UNI bit
> on all packets.  Otherwise, reordering of the initial STREAM and a
> subsequent STREAM would require the receiver to enter “implicit” state for
> the stream (stream is no longer idle, but one cannot assume it is
> bidirectional).
>
>
>
>
>
>
>
>
>
> *From:* Ian Swett [mailto:[email-masked]]
> *Sent:* Friday, June 23, 2017 4:00 PM
> In terms of the extra state, I believe EKR's right that this either
> creates a need to change some text or add another state, which I hadn't
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_662&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=gv2nW8KsHS6Lu2KcNFaw91ojaVQERu816iviQ20Vwkc&s=6kSCQxFc0DF2n1FyODQcprN8J8KATca2FHRLEm02adc&e=>
> .
>
>
>

---

From: Mike Bishop <[email-masked]>
Date: Sat, 24 Jun 2017 00:42:40 +0000

I think my and Igor’s e-mails illustrate that path; I can put a PR together if desired.

Sent: Friday, June 23, 2017 5:41 PM

ekr and I were chatting about the high level design options here.  There are a lot of specific details to both PRs which may not be ideal, but no matter what, the HTTP mapping(and future apps) are going to need both bidirectional and unidirectional streams, so it's just a matter of where that functionality is defined and implemented.

We came up with 4 options(I'm sure there are sub-options here, but bear with me):
  1) Do nothing, let the application(ie: HTTP) create unidirectional streams by closing one half of the bidi stream.
  2) Add unidirectional streams as an option to existing bidirectional streams. (my PR is a proof of concept of that)
  3) Change the transport text to describe unidirectional streams and add a mechanism in the transport doc to pair them into bidi streams.
  4) Replace bidirectional streams with unidirectional streams in the transport doc and make transports implement bidi streams if they need them. (Martin's PR is an example of this)

I think Martin and my PR's demonstrate that #2 and #4 are possible options, and I suspect it would be straightforward to draft an example PR for #3 if someone desired.

I want to keep bidirectional streaming in the transport doc and wrote up 2 because I felt it was simpler to implement and reason about than 3.  I don't believe anyone is a strong supporter of #1 (and if you are, now would be an excellent time to speak up), which leaves the WG choosing between 2, 3 and 4 as architectural directions.

I'm going to update my PR to try to fix the issues identified in the comments when I can this weekend, but I think the critical question for the WG is which direction(s) we'd like to spend time refining, and are people clamoring for an example PR representing option 3?

On Fri, Jun 23, 2017 at 5:59 PM, Lubashev, Igor <[email-masked]<mailto:[email-masked]>> wrote:
Yes, getting rid of implicitly open streams can work.  To get rid of “implicit” state entirely with your design, you would need to keep UNI bit on all packets.  Otherwise, reordering of the initial STREAM and a subsequent STREAM would require the receiver to enter “implicit” state for the stream (stream is no longer idle, but one cannot assume it is bidirectional).

From: Ian Swett [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Friday, June 23, 2017 4:00 PM
To: Lubashev, Igor <[email-masked]<mailto:[email-masked]>>
Cc: Mike Bishop <[email-masked]<mailto:[email-masked]>>; Ted Hardie <[email-masked]<mailto:[email-masked]>>; Patrick McManus <[email-masked]<mailto:[email-masked]>>; Jana Iyengar <[email-masked]<mailto:[email-masked]>>; QUIC WG <[email-masked]<mailto:[email-masked]>>; Martin Thomson <[email-masked]<mailto:[email-masked]>>
Subject: Re: Unidirectional streams PR

Igor,

In terms of the extra state, I believe EKR's right that this either creates a need to change some text or add another state, which I hadn't anticipated, but also isn't the end of the world.  However, Buck Krasic pointed out we can now get rid of implicitly opened streams, because we've transitioned from stream limits to max stream ID, which fixes this problem in a simple way.  See issue #662<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__github.com_quicwg_base-2Ddrafts_issues_662%26d%3DDwMFaQ%26c%3D96ZbZZcaMF4w0F4jpN6LZg%26r%3DDjn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo%26m%3Dgv2nW8KsHS6Lu2KcNFaw91ojaVQERu816iviQ20Vwkc%26s%3D6kSCQxFc0DF2n1FyODQcprN8J8KATca2FHRLEm02adc%26e%3D&data=02%7C01%7CMichael.Bishop%40microsoft.com%7C7aa1c512b7014b576db908d4ba99b144%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636338616647826049&sdata=MmRi%2BdVb6NHkcS3Fj5avK0AtY3UGspmsJkSxHAapOJ0%3D&reserved=0>.