Subject: Core drafts -02 out

From: Martin Thomson <[email-masked]>
Date: Tue, 14 Mar 2017 10:57:55 +1100

The editors have submitted -02 versions of the base set of QUIC drafts.

https://tools.ietf.org/html/draft-ietf-quic-transport-02
https://tools.ietf.org/html/draft-ietf-quic-tls-02
https://tools.ietf.org/html/draft-ietf-quic-recovery-02
https://tools.ietf.org/html/draft-ietf-quic-http-02

The list of changes is LONG.  I won't even attempt to summarize the
changes.  There's a list of changes in an appendix in each draft.

---

From: Stefan Eissing <[email-masked]>
Date: Wed, 15 Mar 2017 14:22:23 +0100

> Am 14.03.2017 um 00:57 schrieb Martin Thomson <[email-masked]>:
> 
[...]
> https://tools.ietf.org/html/draft-ietf-quic-http-02

Thanks, Martin! I assume that was announced to get feedback from the lurkers here. ;-)

I'll give this a try. All mistakes and misunderstandings are mine alone.

-----------------------------------------------------------------------------------------

Very well written spec. Easy to understand for someone having read rfc 7540 a bit. 

I have some comments to the proposed HTTP mapping approach. Where the wg has already discussed and exhausted alternatives, please excuse my ignorance and ignore my comments. I had not the time to follow all discussions ongoing on this topic. Feel free to cherry-pick what seems helpful.

> 4.  Stream Mapping and Usage

+1 to directly using quic stream ids instead of virtual h2 stream identifier

However, using 2 quic streams for a single request/response does not sit well with me. I assume that stems from the wish to get rid of DATA frames. Which sounds nice, but is it worth it? By doubling the # of streams for a client, how much overhead does that introduce (I speak of a server holding >10k quic "connections")?

Also, the server needs to buffer data on quic streams 7, 11, 15, etc. because HEADERs might arrive some time in the future on streams 5, 9, 13, etc. or not. There is no way to route this data somewhere, because the meta information is still missing.

And there is still Holb on:

> 4.2.1.  Header Compression
> ...
> DISCUSS:  Keep HPACK with HOLB?  Redesign HPACK to be order-
>       invariant?  How much do we need to retain compatibility with
>       HTTP/2's HPACK?

Using a counter in HEADERS is a crutch:
- it is a highly specific solution to a common problem in http/quic: synchronicity in connection level state changes. SETTINGS (see below) has the same problem, as does have PRIORITY in HEADERS. It seems that performance wise, all HEADERS could as well be sent on stream 3. 

Now, solving Hol blocking for HEADERS would be a fine achievement. 

> 5.  HTTP Framing Layer
> Frames are used only on the connection (stream 3) and message
>    (streams 5, 9, etc.) control streams. 

And streams 4, 8, 12, etc. I assume.

> 5.2.3.  SETTINGS
> ...
> SETTINGS frames always apply to a connection, never a single stream.
>  A SETTINGS frame MUST be sent as the first frame of the connection
>  control stream (see Section 4) by each peer, and MUST NOT be sent
>  subsequently or on any other stream.  If an endpoint receives an
>  SETTINGS frame on a different stream, the endpoint MUST respond with
>  a connection error of type HTTP_SETTINGS_ON_WRONG_STREAM.  If an
>  endpoint receives a second SETTINGS frame, the endpoint MUST respond
>  with a connection error of type HTTP_MULTIPLE_SETTINGS.

What about HPACK state? The connection state change problem again visible.

This is a severe restriction on extensions mechanisms that want to affect a connection. Because if the http/quic does not solve this problem, how are they expected to do it? They either announce themselves on the first SETTINGS or remain silent forever, it seems. How would an extension handshake work then? Own stream 3 handshake frames?

> 5.2.3.3.  Usage in 0-RTT

What about HPACK state? Does it need to be kept or is it reset?

> HTTP_PUSH_ALREADY_IN_CACHE (0x03):  The server has attempted to push
>      content which the client has cached.
> ...
> HTTP_REQUEST_CANCELLED (0x04):  The client no longer needs the
>     requested data.

Nitpick: seems redundant. The first could be replaced by the second. Stream number will suffice.

----------------------------------------------------------

Taking two steps back:

I think the main difficulty comes from the lack of a "hq connection state" concept and how changes to that state can be managed. Evidence:
- The 0-RTT mentions certain SETTINGS that need to be remembered by a client. How would an extension add to this? Will every extension have to come up with its own solution?
- The HEADERs sequence number is a highly specific fix for the missing state change
- The SETTINGS-ONCE restriction simply avoids the problem by killing a h2 mechanism

If hq defines stream 3 as the place where connection state changes happen *AND* synchronizes OPEN/CLOSE/RST of other streams on it, client and server can have shareable concept of the connection state.

I am no HPACK expert. The basic problem looks like concurrent editing against a repository. 
Both client and server start with connection state zero (CS-0) and the predefined HPACK dictionary (HP-0). After SETTINGS exchange, client is in CS-1 and server is in CS-2 for its side. Let's call the  CS-1 HPACK state HP-1.

Client sends new HEADERS on 5 and keeps the HPACK delta around (HP-1.5). The HEADERS carries the connection state number it is based on (CS-1). Client sends new HEADERS on stream 9, also based on CS-1. Client keeps that delta around (HP-1.9). 

Client then decides to announce a new connection state (CS-3) by sending how it applied the deltas HP-1.5 and HP-1.9 to come up with HP-3. The description allows the server to update its HP-1 to HP-3 as well.

Response HEADERS are also based on a server connection state, explicitly, so the client knows which HP-X to use when decoding them. At some time, the server sends an announcment of CS-4 with HPACK data on stream 3 back to the client.

For a 0-RTT, client and server could exchange the connection state id in the initial SETTINGS, to make sure they have at least the same name remembered as the other side. Client: "I was in CS-19 and you were in CS-8." Server: "Yep."

By implicitly adding all SETTINGS changes to connection states, the problem is also solved for extensions.

If one side receives HEADERS with an unknown connection state:
- if the state id is greater than any known one: set a stream timeout and wait for changes on stream 3 to arrive
- state id less than max(known conn state): STREAM_RST_UNKNOWN_CONN_STATE

New SETTINGS value: MAX_CONN_STATE number of maximum connection state the client/server is willing to keep, exchanged initially. Announcing a new connection state allows the other side to drop the lowest one, if MAX_CONN_STATE are used.

To get optimal HPACK size compression, every HEADERs would also announce a new connections state. To have less potential HOLB, connection states do not change during request bursts.

Something like that.

-Stefan

---

From: Mike Bishop <[email-masked]>
Date: Wed, 15 Mar 2017 17:37:20 +0000

Thanks for the feedback!

Yes, you've run straight into the big quandary with HPACK.  I don't think anyone expects that we will ship this way; I have a proposal in https://tools.ietf.org/html/draft-bishop-quic-http-and-qpack for an HPACK-replacement that would solve many of these.  Buck Krasic has a proposal which he has sketched in e-mail but not submitted as a draft.  The main point of the sequence number was to get us off the "everything on stream 3" model and let us sort out the problems of HPACK later.  #228 tracks fixing HPACK, in whatever form that takes.

We could solve it in the same way that we did PRIORITY, by adding an "affected stream number" field and moving the HEADERS/PUSH_PROMISE frames to Stream 3 as well.  However, that is just as blocking as the current approach, so not really an improvement.  Worse, the reason we can tolerate large header frames is because they occur on their own streams and don't block arrival of data from other streams.  If all headers occur on a single stream, that's not true -- you *are* blocking muxing of other streams again.

I like the comparison to concurrent edits.  We've discussed having rolling deltas in various mechanisms; the problem is that it requires reaching into the transport for ACK state to figure out when you can discard old deltas for good on the receiver side.  Buck's proposal is similar, essentially requiring the receiver to echo back the point up to which it has received all frames, and the sender shouldn't reference state that the receiver hasn't fully assimilated yet.  It feels like adding application-level ACKs to me, which I'd like to avoid.

HPACK is also one of the reasons for the two-stream-per-request approach.  There are two reasons for this -- one is that HPACK frames (in the current design) can't be lost when a stream is RST, so the draft forbids resetting control streams.  Since we still need to be able to RST requests, we add the semantic that killing the data stream implies the same thing about the control stream.  It's messy, and hopefully we can remove that once HPACK is fixed.  The second, as you guessed, is not dealing with DATA frames.  #245 notes that, if we fix HPACK, we could go back to a single stream per request; proponents of both "keep framing out of the way" and "fewer streams is easier to manage" have weighed in there.  Please add your voice.

As to buffering, it's actually easy enough -- just don't read from data streams until you've seen the headers.  Sure, the sender will fill up their flow control window on that stream -- and then they'll stop and send you QUIC BLOCKED frames, until you start reading the body and generating WINDOW_UPDATE frames.  One of the biggest arguments for two streams in my mind is making sure that a request blocked in this way doesn't impede the flow of control frames on that stream -- for example, should we successfully get certificate auth frames added, a certificate request.  I've had two customers this week telling me it's a bug in our server code that their TLS reneg gets stuck in TCP buffers behind a giant request body, because the server won't read an unauthenticated body and the client won't hold off sending the body until authentication succeeds.  I'd like to see blocks like that become less possible in QUIC, at least.

Yes, making SETTINGS immutable reduces the flexibility.  The opinion at the interim was that we could live without it, as we know of exactly one implementation of one extension that does mid-stream setting changes, and I own it.  😊  If there's a compelling use case for changing settings mid-stream we weren't aware of, that's new information that could justify the complexity.  (But note that with 0-RTT connection setup, it would be nearly as cheap to open a new connection with the new setting and transition your traffic to it.)

Negotiation still works, since each side would simply advertise that they support an extension or not, and consider the extension to be active once you've seen the other side's SETTINGS frame.  See https://tools.ietf.org/html/draft-ietf-quic-http-01#section-5.2.5.3 for the complexity this allowed us to remove.  An extension could add to the list easily -- if you support extension X, you should also remember the value for setting X_VAL.  Clients that don't use the extension don't care.  An extension that needed some more complex negotiation would have to use an extension-specific frame, it's true, but we haven't so far seen an extension do that.

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Stefan Eissing
Sent: Wednesday, March 15, 2017 6:22 AM
To: Martin Thomson <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Core drafts -02 out

[...]

> 4.  Stream Mapping and Usage

+1 to directly using quic stream ids instead of virtual h2 stream 
+identifier

Taking two steps back:

Something like that.

-Stefan

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Mar 2017 08:48:21 +0100

> https://tools.ietf.org/html/draft-ietf-quic-http-02

Here's a nit: the drafts currently cite their sibling drafts in a way 
that doesn't conform to the applicable rules. For instance, in 
<https://tools.ietf.org/html/draft-ietf-quic-http-02#section-10.1> we have:

>    [QUIC-TRANSPORT]
>               Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
>               Multiplexed and Secure Transport".

That lacks the draft name, the publication date, and the mandatory "work 
in progress" clause.

Essentially, a newbie reader would have to do a web search for the 
document title.

Best regards, Julian

---

From: Martin Thomson <[email-masked]>
Date: Tue, 21 Mar 2017 19:27:58 +1100

Hi Julian, yes I am aware of this, but I wanted to avoid manual fix ups for
these. If you could open an issue, I will work on automating the process of
generating references.

On 21 Mar. 2017 18:48, "Julian Reschke" <[email-masked]> wrote:

> Here's a nit: the drafts currently cite their sibling drafts in a way that
> doesn't conform to the applicable rules. For instance, in <
> https://tools.ietf.org/html/draft-ietf-quic-http-02#section-10.1> we have:
>
>    [QUIC-TRANSPORT]

---

From: "Charles 'Buck' Krasic" <[email-masked]>
Date: Tue, 21 Mar 2017 15:04:19 -0700

Hi Folks.

I've put together a first attempt at my QPACK proposal:

draft-krasic-quic-hpack-00.html
draft-krasic-quic-hpack-00.txt
<https://krasic.github.io/draft-krasic-quic-hpack/draft-krasic-quic-hpack-00.txt>

Apologies in advance, this is my first attempt at an IETF draft.

On Wed, Mar 15, 2017 at 10:37 AM, Mike Bishop <[email-masked]>
wrote:

> Thanks for the feedback!
>
> Yes, you've run straight into the big quandary with HPACK.  I don't think
> anyone expects that we will ship this way; I have a proposal in
> https://tools.ietf.org/html/draft-bishop-quic-http-and-qpack for an
> HPACK-replacement that would solve many of these.  Buck Krasic has a
> proposal which he has sketched in e-mail but not submitted as a draft.  The
> main point of the sequence number was to get us off the "everything on
> stream 3" model and let us sort out the problems of HPACK later.  #228
> tracks fixing HPACK, in whatever form that takes.
>
> We could solve it in the same way that we did PRIORITY, by adding an
> "affected stream number" field and moving the HEADERS/PUSH_PROMISE frames
> to Stream 3 as well.  However, that is just as blocking as the current
> approach, so not really an improvement.  Worse, the reason we can tolerate
> large header frames is because they occur on their own streams and don't
> block arrival of data from other streams.  If all headers occur on a single
> stream, that's not true -- you *are* blocking muxing of other streams again.
>
> I like the comparison to concurrent edits.  We've discussed having rolling
> deltas in various mechanisms; the problem is that it requires reaching into
> the transport for ACK state to figure out when you can discard old deltas
> for good on the receiver side.  Buck's proposal is similar, essentially
> requiring the receiver to echo back the point up to which it has received
> all frames, and the sender shouldn't reference state that the receiver
> hasn't fully assimilated yet.  It feels like adding application-level ACKs
> to me, which I'd like to avoid.
>
> HPACK is also one of the reasons for the two-stream-per-request approach.
> There are two reasons for this -- one is that HPACK frames (in the current
> design) can't be lost when a stream is RST, so the draft forbids resetting
> control streams.  Since we still need to be able to RST requests, we add
> the semantic that killing the data stream implies the same thing about the
> control stream.  It's messy, and hopefully we can remove that once HPACK is
> fixed.  The second, as you guessed, is not dealing with DATA frames.  #245
> notes that, if we fix HPACK, we could go back to a single stream per
> request; proponents of both "keep framing out of the way" and "fewer
> streams is easier to manage" have weighed in there.  Please add your voice.
>
> As to buffering, it's actually easy enough -- just don't read from data
> streams until you've seen the headers.  Sure, the sender will fill up their
> flow control window on that stream -- and then they'll stop and send you
> QUIC BLOCKED frames, until you start reading the body and generating
> WINDOW_UPDATE frames.  One of the biggest arguments for two streams in my
> mind is making sure that a request blocked in this way doesn't impede the
> flow of control frames on that stream -- for example, should we
> successfully get certificate auth frames added, a certificate request.
> I've had two customers this week telling me it's a bug in our server code
> that their TLS reneg gets stuck in TCP buffers behind a giant request body,
> because the server won't read an unauthenticated body and the client won't
> hold off sending the body until authentication succeeds.  I'd like to see
> blocks like that become less possible in QUIC, at least.
>
> Yes, making SETTINGS immutable reduces the flexibility.  The opinion at
> the interim was that we could live without it, as we know of exactly one
> implementation of one extension that does mid-stream setting changes, and I
> own it.  😊  If there's a compelling use case for changing settings
> mid-stream we weren't aware of, that's new information that could justify
> the complexity.  (But note that with 0-RTT connection setup, it would be
> nearly as cheap to open a new connection with the new setting and
> transition your traffic to it.)
>
> Negotiation still works, since each side would simply advertise that they
> support an extension or not, and consider the extension to be active once
> you've seen the other side's SETTINGS frame.  See
> https://tools.ietf.org/html/draft-ietf-quic-http-01#section-5.2.5.3 for
> the complexity this allowed us to remove.  An extension could add to the
> list easily -- if you support extension X, you should also remember the
> value for setting X_VAL.  Clients that don't use the extension don't care.
> An extension that needed some more complex negotiation would have to use an
> extension-specific frame, it's true, but we haven't so far seen an
> extension do that.
>
> To: Martin Thomson <[email-masked]>; IETF QUIC WG <[email-masked]
> >
> > Am 14.03.2017 um 00:57 schrieb Martin Thomson <[email-masked]
> >:
> >
> I have some comments to the proposed HTTP mapping approach. Where the wg
> has already discussed and exhausted alternatives, please excuse my
> ignorance and ignore my comments. I had not the time to follow all
> discussions ongoing on this topic. Feel free to cherry-pick what seems
> helpful.
>
>
> However, using 2 quic streams for a single request/response does not sit
> well with me. I assume that stems from the wish to get rid of DATA frames.
> Which sounds nice, but is it worth it? By doubling the # of streams for a
> client, how much overhead does that introduce (I speak of a server holding
> >10k quic "connections")?
>
> Also, the server needs to buffer data on quic streams 7, 11, 15, etc.
> because HEADERs might arrive some time in the future on streams 5, 9, 13,
> etc. or not. There is no way to route this data somewhere, because the meta
> information is still missing.
>
> - it is a highly specific solution to a common problem in http/quic:
> synchronicity in connection level state changes. SETTINGS (see below) has
> the same problem, as does have PRIORITY in HEADERS. It seems that
> performance wise, all HEADERS could as well be sent on stream 3.
>
> This is a severe restriction on extensions mechanisms that want to affect
> a connection. Because if the http/quic does not solve this problem, how are
> they expected to do it? They either announce themselves on the first
> SETTINGS or remain silent forever, it seems. How would an extension
> handshake work then? Own stream 3 handshake frames?
>
> > 5.2.3.3.  Usage in 0-RTT
>
> I think the main difficulty comes from the lack of a "hq connection state"
> concept and how changes to that state can be managed. Evidence:
> - The 0-RTT mentions certain SETTINGS that need to be remembered by a
> client. How would an extension add to this? Will every extension have to
> come up with its own solution?
> If hq defines stream 3 as the place where connection state changes happen
> *AND* synchronizes OPEN/CLOSE/RST of other streams on it, client and server
> can have shareable concept of the connection state.
>
>
> Both client and server start with connection state zero (CS-0) and the
> predefined HPACK dictionary (HP-0). After SETTINGS exchange, client is in
> CS-1 and server is in CS-2 for its side. Let's call the  CS-1 HPACK state
> HP-1.
>
> Client sends new HEADERS on 5 and keeps the HPACK delta around (HP-1.5).
> The HEADERS carries the connection state number it is based on (CS-1).
> Client sends new HEADERS on stream 9, also based on CS-1. Client keeps that
> delta around (HP-1.9).
>
> Client then decides to announce a new connection state (CS-3) by sending
> how it applied the deltas HP-1.5 and HP-1.9 to come up with HP-3. The
> description allows the server to update its HP-1 to HP-3 as well.
>
> Response HEADERS are also based on a server connection state, explicitly,
> so the client knows which HP-X to use when decoding them. At some time, the
> server sends an announcment of CS-4 with HPACK data on stream 3 back to the
> client.
>
> For a 0-RTT, client and server could exchange the connection state id in
> the initial SETTINGS, to make sure they have at least the same name
> remembered as the other side. Client: "I was in CS-19 and you were in
> CS-8." Server: "Yep."
>
> New SETTINGS value: MAX_CONN_STATE number of maximum connection state the
> client/server is willing to keep, exchanged initially. Announcing a new
> connection state allows the other side to drop the lowest one, if
> MAX_CONN_STATE are used.
>
> To get optimal HPACK size compression, every HEADERs would also announce a
> new connections state. To have less potential HOLB, connection states do
> not change during request bursts.
>
> Something like that.
>
> -Stefan
>
>
>

-- 
Charles 'Buck' Krasic | Software Engineer | [email-masked] | +1 (408)
412-1141

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 21 Mar 2017 22:14:49 +0000

Hi Buck,

Thanks for your proposal, I'll have a proper read on the plane over to Chicago!

The HTML version link was broken for me, but this one works - https://krasic.github.io/draft-krasic-quic-hpack/draft-krasic-quic-hpack-00.html.

Regards
Lucas
________________________________
From: QUIC [[email-masked]] on behalf of Charles 'Buck' Krasic [[email-masked]]
Sent: 21 March 2017 22:04
To: Mike Bishop
Cc: Stefan Eissing; IETF QUIC WG; Martin Thomson
Subject: Re: Core drafts -02 out

Hi Folks.

draft-krasic-quic-hpack-00.html<http://draft-krasic-quic-hpack-00.html>
draft-krasic-quic-hpack-00.txt<https://krasic.github.io/draft-krasic-quic-hpack/draft-krasic-quic-hpack-00.txt>

On Wed, Mar 15, 2017 at 10:37 AM, Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for the feedback!

-----Original Message-----
From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Stefan Eissing
Sent: Wednesday, March 15, 2017 6:22 AM
To: Martin Thomson <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Core drafts -02 out

> Am 14.03.2017 um 00:57 schrieb Martin Thomson <[email-masked]<mailto:[email-masked]>>:
>
[...]

Taking two steps back:

Something like that.

-Stefan

--
Charles 'Buck' Krasic | Software Engineer | [email-masked]<mailto:[email-masked]> | +1 (408) 412-1141

---

From: Mike Bishop <[email-masked]>
Date: Thu, 23 Mar 2017 17:52:03 +0000

All I see in the links are http:// and the draft name, no host.  I’m guessing you meant https://github.com/krasic/draft-krasic-quic-hpack/blob/master/draft-krasic-quic-hpack-00.html?  Once the tracker reopens (sometime Sunday, I think) you should probably upload those at datatracker.ietf.org to make it an official submission.  You’ll need to make the draft name in the file match the file name before it will successfully upload.

Also, for ease of discussion, you might pick a different name than QPACK, since there’s already a draft with that name.  (Or we both change names, and call the eventually-adopted one QPACK; that’d be fine, too.)

As to actual feedback:  I see what you’re getting at here because you’ve explained it – the decoder acknowledges the sequence number it’s up to, then the encoder tracks that and uses it on the encode side.  I think I’d have a really hard time parsing the document without that.  I think the strongest advantage of this proposal is that an encoder for this scheme could be used over HTTP/2, enabling shared code; you should bring that out more.

The way you’ve described the encoder requires tight integration between the packetization logic and the header compressor in two ways:

  *   It needs to know the “packet epoch” (encode epoch of the first frame in the current packet) while compressing, but until you know the size of the compressed output, you can’t know whether it will fit in the current packet.  That in turn means you’ll need to roll back any state and re-encode with a different packet epoch if it turns out not to fit.
  *   The “commit epoch” is determined, not by data at the compressor level, but by reaching into the transport and checking which packets containing the STREAM frames containing the HPACK data in question have been acknowledged.  Also note that ACK doesn’t signify delivery to the application layer (HTTP), just presence in the appropriate queue.

In 2.2.1.1, the encoder is informed whether “QPACK is enabled,” but since that can be toggled on/off on a per-header-frame basis (and I hope you really meant per-header-block), it seems like the decoder is just following the encoder’s lead on this.  Who’s actually making that decision?  You say it gets turned off if a drop results, but the encoder is the one who knows/determines that.

This has the same problem as the current HPACK state that RST on a control stream loses critical data and there’s no way to recover.  I’m personally hoping we can make our way out of that requirement.

From: Charles 'Buck' Krasic [mailto:[email-masked]]
Sent: Tuesday, March 21, 2017 6:04 PM
To: Mike Bishop <[email-masked]>
Cc: Stefan Eissing <[email-masked]>; Martin Thomson <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Core drafts -02 out

Hi Folks.

-----Original Message-----
From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Stefan Eissing
Sent: Wednesday, March 15, 2017 6:22 AM
To: Martin Thomson <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Core drafts -02 out

[...]

Taking two steps back:

Something like that.

-Stefan

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 23 Mar 2017 14:30:24 -0400

On Thu, Mar 23, 2017 at 1:52 PM, Mike Bishop <[email-masked]>
wrote:

> All I see in the links are http:// and the draft name, no host.  I’m
> guessing you meant https://github.com/krasic/draft-krasic-quic-hpack/blob/
> master/draft-krasic-quic-hpack-00.html?  Once the tracker reopens
> (sometime Sunday, I think) you should probably upload those at
> datatracker.ietf.org to make it an official submission.  You’ll need to
> make the draft name in the file match the file name before it will
> successfully upload.
>
>
>
> Also, for ease of discussion, you might pick a different name than QPACK,
> since there’s already a draft with that name.  (Or we both change names,
> and call the eventually-adopted one QPACK; that’d be fine, too.)
>

Just on this one point: I like QPACK for the final one. Maybe it's fine to
call it krasic-qpack and bishop-qpack for now?

- jana

>
>
> As to actual feedback:  I see what you’re getting at here because you’ve
> explained it – the decoder acknowledges the sequence number it’s up to,
> then the encoder tracks that and uses it on the encode side.  I think I’d
> have a really hard time parsing the document without that.  I think the
> strongest advantage of this proposal is that an encoder for this scheme
> could be used over HTTP/2, enabling shared code; you should bring that out
> more.
>
>
>
> The way you’ve described the encoder requires tight integration between
> the packetization logic and the header compressor in two ways:
>
>    - It needs to know the “packet epoch” (encode epoch of the first frame
>    in the current packet) while compressing, but until you know the size of
>    the compressed output, you can’t know whether it will fit in the current
>    packet.  That in turn means you’ll need to roll back any state and
>    re-encode with a different packet epoch if it turns out not to fit.
>    - The “commit epoch” is determined, not by data at the compressor
>    level, but by reaching into the transport and checking which packets
>    containing the STREAM frames containing the HPACK data in question have
>    been acknowledged.  Also note that ACK doesn’t signify delivery to the
>    application layer (HTTP), just presence in the appropriate queue.
>
>
>
> In 2.2.1.1, the encoder is informed whether “QPACK is enabled,” but since
> that can be toggled on/off on a per-header-frame basis (and I hope you
> really meant per-header-block), it seems like the decoder is just following
> the encoder’s lead on this.  Who’s actually making that decision?  You say
> it gets turned off if a drop results, but the encoder is the one who
> knows/determines that.
>
>
>
> This has the same problem as the current HPACK state that RST on a control
> stream loses critical data and there’s no way to recover.  I’m personally
> hoping we can make our way out of that requirement.
>
>
>
> *From:* Charles 'Buck' Krasic [mailto:[email-masked]]
> *Sent:* Tuesday, March 21, 2017 6:04 PM
> *To:* Mike Bishop <[email-masked]>
> *Cc:* Stefan Eissing <[email-masked]>; Martin Thomson <
> [email-masked]>; IETF QUIC WG <[email-masked]>
>
> *Subject:* Re: Core drafts -02 out
>
>
>
> Hi Folks.
>
>
>
> On Wed, Mar 15, 2017 at 10:37 AM, Mike Bishop <
> [email-masked]> wrote:
>
> Thanks for the feedback!
>

---

From: "Charles 'Buck' Krasic" <[email-masked]>
Date: Thu, 23 Mar 2017 11:57:55 -0700

On Thu, Mar 23, 2017 at 10:52 AM, Mike Bishop <[email-masked]>
wrote:

> guessing you meant https://github.com/krasic/draf
> t-krasic-quic-hpack/blob/master/draft-krasic-quic-hpack-00.html?  Once
> the tracker reopens (sometime Sunday, I think) you should probably upload
> those at datatracker.ietf.org to make it an official submission.  You’ll
> need to make the draft name in the file match the file name before it will
> successfully upload.
>
>
>
Thanks for the prompt feedback!

Apologies, as I mentioned it is my first attempt at an IETF draft, and it
my first time using the tools as described in SETUP.md
<https://github.com/martinthomson/i-d-template/blob/master/doc/SETUP.md>.

Until I the tracker reopens and I submit properly, I would suggest
https://krasic.github.io/draft-krasic-quic-hpack/dra
ft-krasic-quic-qpack-latest.html

> and call the eventually-adopted one QPACK; that’d be fine, too.)

I'm fine with whatever.  If  draft-krasic vs draft-bishop isn't enough to
differentiate,  how about QPACK-ALT, or maybe QPACK-LITE?

> could be used over HTTP/2, enabling shared code; you should bring that out

I'll try to rework the text to emphise those points more and clarify.

Nits:  the decoder per-se doesn't ack, but the encoder tracks using
existing transport level ack mechanisms (more on this below).   Also, I
think of this scheme as something a shared implementation could offer as an
option, but I hadn't imagined the option would be ever be enabled in
HTTP/2.  It might be interesting to consider though.

> I don't think roll back is esssential.

The space available in the current packet could be known when invoking the
hpack encoder, so it would stop emitting representations if the packet
reaches the space limit, and in that case return a header block that for a
prefix of the provided header field list, and in later packets it would
generate a continuation header block(s) for remaining header fields.

>    -
My understanding is that APIs are out of scope of most of the docs.
However, in our implementation, the stream write interfaces uniformly
provide a means of to confirm acknowledgement,  so it is possible to do it
without reaching down.

I agree that ACK doesn't signify delivery to what is above HTTP, but one of
advantages of not having legacy APIs (POSIX sockets) between QUIC and HTTP
is that the implemenation can certainly ensure that ACK signifies delivery
to HTTP.

> it gets turned off if a drop results, but the encoder is the one who
Yes, all decisions are made by the encoder, and as in HPACK the decoder
strictly follows the encoder's lead.

The "informed" language was simply alluding to the encoder side of the
HPACK API being extended with QPACK related params.

And yes, I meant "header block".  Will fix that.

> hoping we can make our way out of that requirement.
My position is that REFUSED should be the only kind of RST needed or
allowed for control streams, and dealing with that is tractible.

-- 
Charles 'Buck' Krasic | Software Engineer | [email-masked] | +1 (408)
412-1141 <(408)%20412-1141>

---

From: Mike Bishop <[email-masked]>
Date: Thu, 23 Mar 2017 23:12:52 +0000

With regard to header block versus frame, though, you explicitly said in your reply before that the encoder can switch modes between frames in a single block.  So I think you mean what you wrote; it’s just that switching whether things can be interpreted out of order mid-block feels weird.

And POSIX or not, I don't think you can unilaterally define a semantic about delivery to the remote app layer.  Though you could come close – if the call completed when the written data and all preceding data on the stream had been ACK’d, you know at least that the data is available to the remote app layer – whether it reads promptly is its own problem.

Sent from my Windows 10 phone

From: Charles 'Buck' Krasic<mailto:[email-masked]>
Sent: Thursday, March 23, 2017 2:58 PM
To: Mike Bishop<mailto:[email-masked]>
Cc: Stefan Eissing<mailto:[email-masked]>; Martin Thomson<mailto:[email-masked]>; IETF QUIC WG<mailto:[email-masked]>
Subject: Re: Core drafts -02 out

On Thu, Mar 23, 2017 at 10:52 AM, Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
All I see in the links are http:// and the draft name, no host.  I’m guessing you meant https://github.com/krasic/draft-krasic-quic-hpack/blob/master/draft-krasic-quic-hpack-00.html?  Once the tracker reopens (sometime Sunday, I think) you should probably upload those at datatracker.ietf.org<http://datatracker.ietf.org> to make it an official submission.  You’ll need to make the draft name in the file match the file name before it will successfully upload.

Apologies, as I mentioned it is my first attempt at an IETF draft, and it my first time using the tools as described in SETUP.md<https://github.com/martinthomson/i-d-template/blob/master/doc/SETUP.md>.

Until I the tracker reopens and I submit properly, I would suggest https://krasic.github.io/draft-krasic-quic-hpack/draft-krasic-quic-qpack-latest.html

I'm fine with whatever.  If  draft-krasic vs draft-bishop isn't enough to differentiate,  how about QPACK-ALT, or maybe QPACK-LITE?

Nits:  the decoder per-se doesn't ack, but the encoder tracks using existing transport level ack mechanisms (more on this below).   Also, I think of this scheme as something a shared implementation could offer as an option, but I hadn't imagined the option would be ever be enabled in HTTP/2.  It might be interesting to consider though.

The space available in the current packet could be known when invoking the hpack encoder, so it would stop emitting representations if the packet reaches the space limit, and in that case return a header block that for a prefix of the provided header field list, and in later packets it would generate a continuation header block(s) for remaining header fields.

  *

My understanding is that APIs are out of scope of most of the docs.   However, in our implementation, the stream write interfaces uniformly provide a means of to confirm acknowledgement,  so it is possible to do it without reaching down.

I agree that ACK doesn't signify delivery to what is above HTTP, but one of advantages of not having legacy APIs (POSIX sockets) between QUIC and HTTP is that the implemenation can certainly ensure that ACK signifies delivery to HTTP.
In 2.2.1.1, the encoder is informed whether “QPACK is enabled,” but since that can be toggled on/off on a per-header-frame basis (and I hope you really meant per-header-block), it seems like the decoder is just following the encoder’s lead on this.  Who’s actually making that decision?  You say it gets turned off if a drop results, but the encoder is the one who knows/determines that.
Yes, all decisions are made by the encoder, and as in HPACK the decoder strictly follows the encoder's lead.

The "informed" language was simply alluding to the encoder side of the HPACK API being extended with QPACK related params.

This has the same problem as the current HPACK state that RST on a control stream loses critical data and there’s no way to recover.  I’m personally hoping we can make our way out of that requirement.
My position is that REFUSED should be the only kind of RST needed or allowed for control streams, and dealing with that is tractible.

From: Charles 'Buck' Krasic [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, March 21, 2017 6:04 PM
To: Mike Bishop <[email-masked]<mailto:[email-masked]>>
Cc: Stefan Eissing <[email-masked]<mailto:[email-masked]>>; Martin Thomson <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>

Hi Folks.

-----Original Message-----
From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Stefan Eissing
Sent: Wednesday, March 15, 2017 6:22 AM
To: Martin Thomson <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Core drafts -02 out

[...]

Taking two steps back:

Something like that.

-Stefan

--
Charles 'Buck' Krasic | Software Engineer | [email-masked]<mailto:[email-masked]> | +1 (408) 412-1141<tel:(408)%20412-1141>

--
Charles 'Buck' Krasic | Software Engineer | [email-masked]<mailto:[email-masked]> | +1 (408) 412-1141<tel:(408)%20412-1141>

---

From: "Charles 'Buck' Krasic" <[email-masked]>
Date: Thu, 23 Mar 2017 17:18:10 -0700

On Thu, Mar 23, 2017 at 4:12 PM, Mike Bishop <[email-masked]>
wrote:

> With regard to header block versus frame, though, you explicitly said in
> your reply before that the encoder can switch modes between frames in a
> single block.  So I think you mean what you wrote; it’s just that switching
> whether things can be interpreted out of order mid-block feels weird.
>

You're right.  After re-reading Header Compression and Decompression
<https://tools.ietf.org/html/rfc7540#section-4.3> just now, I see that
 I've been messing up the terminology, I was confusing header blocks with
header block fragments.  I will take a pass to correct that, and I think I
need to call out that what I'm proposing diverges from the HTTP/2
description---my thinking is that a header block should be serialized
packet/fragment-wise, as opposed to rfc5740 which says serialize the whole
header block and then divide it into header block fragments.

I agree that switching mid-block would feel weird, but it wouldn't be
common at all.  It should only be necessary if the table becomes full.  I
think experience will bear out that avoiding that

>
> And POSIX or not, I don't think you can unilaterally define a semantic
> about delivery to the remote app layer.  Though you could come close – if
> the call completed when the written data and all preceding data on the
> stream had been ACK’d, you know at least that the data is available to the
> remote app layer – whether it reads promptly is its own problem.
>

I think you can go further than that and reason correctly about ordering in
a coherent implementation.

If HA has been fully acked before HB is encoded, then I think an
implementation can quite reasonably guarantee that HA will be decoded
before HB.   I am confident this is true of ours.

If it helps clarify at all, I'd remind that our implementation is
single-threaded user level code.   I can imagine how it might be more
difficult (but not impossible) to reason about in an implementation that is
multi-threaded and/or divided across a kernel-user boundary.

> *From: *Charles 'Buck' Krasic <[email-masked]>
> *Sent: *Thursday, March 23, 2017 2:58 PM
> *To: *Mike Bishop <[email-masked]>
> *Cc: *Stefan Eissing <[email-masked]>; Martin Thomson
> <[email-masked]>; IETF QUIC WG <[email-masked]>
>
> *Subject: *Re: Core drafts -02 out
>
>
>
>
> On Thu, Mar 23, 2017 at 10:52 AM, Mike Bishop <
>>    - It needs to know the “packet epoch” (encode epoch of the first
>>    frame in the current packet) while compressing, but until you know the size
>>    of the compressed output, you can’t know whether it will fit in the current
> I agree that ACK doesn't signify delivery to what is above HTTP, but one
> of advantages of not having legacy APIs (POSIX sockets) between QUIC and
> HTTP is that the implemenation can certainly ensure that ACK signifies
> delivery to HTTP.
>
>> I like the comparison to concurrent edits.  We've discussed having
>> rolling deltas in various mechanisms; the problem is that it requires
>> reaching into the transport for ACK state to figure out when you can
>> discard old deltas for good on the receiver side.  Buck's proposal is
>> similar, essentially requiring the receiver to echo back the point up to
>> which it has received all frames, and the sender shouldn't reference state
>> that the receiver hasn't fully assimilated yet.  It feels like adding
>> application-level ACKs to me, which I'd like to avoid.
>>
>> HPACK is also one of the reasons for the two-stream-per-request
>> approach.  There are two reasons for this -- one is that HPACK frames (in
>> the current design) can't be lost when a stream is RST, so the draft
>> forbids resetting control streams.  Since we still need to be able to RST
>> requests, we add the semantic that killing the data stream implies the same
>> thing about the control stream.  It's messy, and hopefully we can remove
>> that once HPACK is fixed.  The second, as you guessed, is not dealing with
>> DATA frames.  #245 notes that, if we fix HPACK, we could go back to a
>> single stream per request; proponents of both "keep framing out of the way"
>> and "fewer streams is easier to manage" have weighed in there.  Please add
>> your voice.
>>
>> I think the main difficulty comes from the lack of a "hq connection
>> state" concept and how changes to that state can be managed. Evidence:
>> To get optimal HPACK size compression, every HEADERs would also announce
>> a new connections state. To have less potential HOLB, connection states do