Subject: [QUIC] Network Path Requirements for QUIC

From: 🔓Dan Wing <[email-masked]>
Date: Tue, 14 Jun 2016 12:06:04 -0600

Joe and I just published "Network Path Requirements for QUIC", https://tools.ietf.org/html/draft-wing-quic-network-req, which offers some requirements we believe will help universal QUIC deployment.

-d

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 14 Jun 2016 18:26:42 +0000

>Joe and I just published "Network Path Requirements for QUIC", https://tools.ietf.org/html/draft-wing-quic-network-req, which offers some requirements we believe will help universal QUIC deployment.

I believe that a small addition to the charter would be useful to ensure that we make some explicit decisions about the work implied by this draft, adding the clause "network path interactions" as one of the things that needs to be described in the transport work:

OLD:

'''
The first of these is the core transport work, which will describe the wire format, along with the mechanisms for connection establishment, stream multiplexing, data reliability, loss detection and recovery, congestion control, version negotiation, and options negotiation.
'''

NEW:

'''
The first of these is the core transport work, which will describe the wire format, along with the mechanisms for connection establishment, stream multiplexing, data reliability, loss detection and recovery, congestion control, version negotiation, options negotiation, and network path interactions.
'''

(Pull Request: https://github.com/IETF-QUIC/Charter/pull/8)

We can argue over what those interactions should be (including "none"), but I would like us to make explicit decisions, rather than leaving the middlebox vendors to ossify and break QUIC in the exciting ways that are likely if they proceed without any guidance. 

As well, I would like to request some time on the BoF agenda to discuss this draft, at least at a high level. 

-- 
Joe Hildebrand

---

From: Christian Huitema <[email-masked]>
Date: Tue, 14 Jun 2016 18:54:35 +0000

On Tuesday, June 14, 2016 11:27 AM, Joe Hildebrand wrote:
> ...
> We can argue over what those interactions should be (including "none"), but I
> would like us to make explicit decisions, rather than leaving the middlebox
> vendors to ossify and break QUIC in the exciting ways that are likely if they
> proceed without any guidance.

Since when did guidance stop middle box designers from breaking protocols? The proper defense against middle boxes is to carefully chose what is outside the encryption envelope, thus limiting the amount of misguided cleverness that can be applied by intermediaries. 

-- Christian Huitema

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 14 Jun 2016 19:23:01 +0000

>Since when did guidance stop middle box designers from breaking protocols? The proper defense against middle boxes is to carefully chose what is outside the encryption envelope, thus limiting the am…

Please read the draft.  I hope we're not suggesting particular design approaches.  If we are, we'll change the draft.  What I hope we're raising is a set of potential requirements to start the discussion of exactly what you're proposing: what goes inside and outside the crypto veil.  Anything that we expose will ossify immediately.  Anything that we don't expose may still ossify, but it even less predictable ways.

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 14 Jun 2016 12:48:42 -0700

On Tue, Jun 14, 2016 at 12:23 PM, Joe Hildebrand (jhildebr) <j
<[email-masked]>

>
> Please read the draft.  I hope we're not suggesting particular design
> approaches.

That comes across as a little odd, honestly, because there are design
approaches already in QUIC for much of what you're looking for (e.g. the
visible connection-ID).  That makes it appear that you're suggesting
changes, when you don't aver you don't intend to..

I kind of suspect that this would work as  a "Requirements for UDP-based
protocols desiring to provide TCP-like visibility to the path", with a
later checklist by QUIC to show how they're met (or if they're not met, and
why).

> If we are, we'll change the draft.  What I hope we're raising is a set of
> potential requirements to start the discussion of exactly what you're
> proposing: what goes inside and outside the crypto veil.

So, there is more than one type of crypto that may be at play here:
encryption to make confidential and encryption to make tampering evident.

> Anything that we expose will ossify immediately.  Anything that we don't
> expose may still ossify, but it even less predictable ways.
>
>
I think you're using ossify in an odd way here.  I normally use it when
something flexible has become rigid.  Exposing connection-id, as QUIC does,
doesn't ossify connection-id.  Middlebox vendors who fire-and-forget may
understand that version and never update, but that's not triggered by
exposure so much as update cycle issue (some of which are so bad that
merely exposing connection-id and QUIC won't trigger an update at all;
those will never see the traffic as QUIC).

In the long term, I want to see PLUS in this space and hope that it gives
us the flexibility we need to go forward.  Absent that, I appreciate your
taking the time to express what middleboxes want to see on the apth.  But I
do want us to be careful not to either try to slam the two efforts together
nor to make UDP-based protocols mimic TCP-based ones exactly.  They will
vary, for large values of "they".

Take with a grain of salt, as I had little sleep and a strange morning,

Ted

> --
> Joe Hildebrand
>
>
>
> _______________________________________________
> QUIC mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/quic
>

---

From: Ted Hardie <[email-masked]>
Date: Tue, 14 Jun 2016 12:50:20 -0700

The typos below are strong evidence on the need for the mentioned salt.  I
meant, of course,  "That makes it appear that you're suggesting changes,
when you aver you don't intend to."  (I must be sleepy, as no one awake
uses "aver" in a sentence).

apologies,

Ted

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 14 Jun 2016 20:08:23 +0000

On 6/14/16, 1:48 PM, "Ted Hardie" <[email-masked]> wrote:

>On Tue, Jun 14, 2016 at 12:23 PM, Joe Hildebrand (jhildebr) 
><j <mailto:[email-masked]>
>
>Please read the draft.  I hope we're not suggesting particular design approaches. 

(note: I apologize for assuming that someone didn't read the draft.  That was unkind and unnecessary.)

>That comes across as a little odd, honestly, because there are design approaches already in QUIC for much of what you're looking for (e.g. the visible connection-ID).  That makes it appear that you're suggesting changes, when you don't aver you don't intend
> to..  

It appeared to me as if the connection ID might change over the course of a connection, or be elided in packets later in the flow.  If that isn't the case, QUIC already meets this requirement.

>I kind of suspect that this would work as  a "Requirements for UDP-based protocols desiring to provide TCP-like visibility to the path", with a later checklist by QUIC to show how they're met (or if they're not met, and why). 

That's a valid approach, and I'll likely be presenting this in the PLUS BoF to that effect

>So, there is more than one type of crypto that may be at play here:  encryption to make confidential and encryption to make tampering evident. 

Agree.  I can't imagine much other than the address/port 4-tuple being outside the tamper-evident seal (because of NAT and PAT).  There may be some cases for scratch space, but I suspect we would need a really important use case before we got consensus on such a thing. 

>I think you're using ossify in an odd way here.  I normally use it when something flexible has become rigid.  Exposing connection-id, as QUIC does, doesn't ossify connection-id.  Middlebox vendors who fire-and-forget may understand that version and never
> update, but that's not triggered by exposure so much as update cycle issue (some of which are so bad that merely exposing connection-id and QUIC won't trigger an update at all; those will never see the traffic as QUIC).

In particular, I meant the version numbers here.  In absence of some more thought, I would expect middlebox vendors to use a particular version number to try to classify traffic as "this is QUIC", which would be a horrible outcome for everyone.

>In the long term, I want to see PLUS in this space and hope that it gives us the flexibility we need to go forward.  Absent that, I appreciate your taking the time to express what middleboxes want to see on the apth.  But I do want us to be careful not
> to either try to slam the two efforts together nor to make UDP-based protocols mimic TCP-based ones exactly.  They will vary, for large values of "they".

I think a potential PLUS working group would be a useful way to explore both the requirements and a nicely-layered protocol.  Absent a charter-level commitment that a potential QUIC working group would use the output of that work (which I'm *not* arguing for), having a small addition to the charter to ensure that we discuss what kinds of path breakage QUIC is explicitly signing up for seems like a relatively small ask.

-- 
Joe Hildebrand

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 14 Jun 2016 23:34:57 -0700

Hi Dan, Joe,

Thanks for putting this draft together and for sharing it. I want to add
just a few thoughts to the conversation.

QUIC currently has most of the QUIC header fields encrypted and
authenticated (invisible to the network), and some header fields
authenticated but not encrypted (visible to the network but tamper
resistant). The fields that are exposed have well-defined use cases ---
Connection ID, for example, is visible and can be used by load-balancers to
direct traffic belonging to a QUIC connection to the same server. I expect
that these fields will be debated and discussed in the wg, and I think
these considerations will be quite helpful when that conversation is
underway. It'll be good to identify precisely which fields of QUIC that
aren't currently exposed should be exposed (or vice versa), to have a
debate in more concrete terms.

Specifically, if you have suggestions for which fields in QUIC ought to be
made visible or invisible, or about which mechanisms ought to be added, I
think that'd be helpful.

On the proposed addition to the charter text, I am unclear on the intent.
Is it to document possible breakages, or is the intent to prescribe
middlebox behavior? I'm not sure how useful the former can be, and I think
the latter is a pretty significant effort (and central to PLUS, I believe.)
I don't think the proposed addition to the charter text is appropriate.

FWIW, I agree that work in PLUS should inform work in QUIC. I'd like to
suggest that, for purely pragmatic reasons, we wait for PLUS to do its work
and for QUIC to do its work before incorporating PLUS's lessons into QUIC.
Mixing these two at this early stage, when scoping as tightly as reasonably
possible is critical, does not seem advisable. This may not be what you're
suggesting at all, so we may already be on the same page.

- jana

On Tue, Jun 14, 2016 at 1:08 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> It appeared to me as if the connection ID might change over the course of
> a connection, or be elided in packets later in the flow.  If that isn't the
> case, QUIC already meets this requirement.
>
> Agree.  I can't imagine much other than the address/port 4-tuple being
> outside the tamper-evident seal (because of NAT and PAT).  There may be
> some cases for scratch space, but I suspect we would need a really
> important use case before we got consensus on such a thing.
>
> understand that version and never
> > update, but that's not triggered by exposure so much as update cycle
> issue (some of which are so bad that merely exposing connection-id and QUIC
> won't trigger an update at all; those will never see the traffic as QUIC).
>
> In particular, I meant the version numbers here.  In absence of some more
> thought, I would expect middlebox vendors to use a particular version
> number to try to classify traffic as "this is QUIC", which would be a
> horrible outcome for everyone.
>
> do want us to be careful not
> > to either try to slam the two efforts together nor to make UDP-based
> protocols mimic TCP-based ones exactly.  They will vary, for large values
> of "they".
>
> I think a potential PLUS working group would be a useful way to explore
> both the requirements and a nicely-layered protocol.  Absent a
> charter-level commitment that a potential QUIC working group would use the
> output of that work (which I'm *not* arguing for), having a small addition
> to the charter to ensure that we discuss what kinds of path breakage QUIC
> is explicitly signing up for seems like a relatively small ask.
>
> --
> Joe Hildebrand
>
>

---

From: 🔓Dan Wing <[email-masked]>
Date: Wed, 15 Jun 2016 15:33:02 -0600

On 15-Jun-2016 12:34 am, Jana Iyengar <[email-masked]> wrote: 
> Hi Dan, Joe,
> 
> Thanks for putting this draft together and for sharing it. I want to add just a few thoughts to the conversation.
> 
> QUIC currently has most of the QUIC header fields encrypted and authenticated (invisible to the network), and some header fields authenticated but not encrypted (visible to the network but tamper resistant). The fields that are exposed have well-defined use cases --- Connection ID, for example, is visible and can be used by load-balancers to direct traffic belonging to a QUIC connection to the same server. I expect that these fields will be debated and discussed in the wg, and I think these considerations will be quite helpful when that conversation is underway. It'll be good to identify precisely which fields of QUIC that aren't currently exposed should be exposed (or vice versa), to have a debate in more concrete terms.  

Yes, hence Joe's charter pull request.

> Specifically, if you have suggestions for which fields in QUIC ought to be made visible or invisible, or about which mechanisms ought to be added, I think that'd be helpful.

draft-wing-quic-network-req tries to up-level its recommendations by saying 'we need consent' rather than suggesting how to achieve it.  Consent could be achieved in probably 3 or 4 different ways, and we need a technique that works with multicast QUIC and with persistent QUIC connections.  It's up to WG to agree consent is necessary, then to agree how to accomplish consent.  We have identified other things that impact the network and need discussion in the working group (some small, some larger):  Should the network clear its state immediately after QUIC public reset, set timer, wait for a little while.  Is the network's sole identification the QUIC version in the client-initiated connection, and can we avoid the network treating that specially (which will be good and bad, I am remembering network treatment of  IKE's UDP 500 and 4500).  If QUIC endpoint sends or receives many bogus QUIC packets, how can network help stop or rate limit those, to defend the links and defend other hosts on that network.  If path drops state due to a timeout (or crash, or software update, or whatever), how should the endpoints learn and how should they react.  Those previous things are discussed in the I-D.  In addition, QUIC complicates network diagnostics and measurements, as well, which will be added in a later version of the I-D.

> On the proposed addition to the charter text, I am unclear on the intent. Is it to document possible breakages, or is the intent to prescribe middlebox behavior?

The intent is to change QUIC, as necessary, and to change network devices, as necessary, so that QUIC can be successfully deployed on more networks.

> I'm not sure how useful the former can be, and I think the latter is a pretty significant effort (and central to PLUS, I believe.) I don't think the proposed addition to the charter text is appropriate. 
> 
> FWIW, I agree that work in PLUS should inform work in QUIC. I'd like to suggest that, for purely pragmatic reasons, we wait for PLUS to do its work and for QUIC to do its work before incorporating PLUS's lessons into QUIC. Mixing these two at this early stage, when scoping as tightly as reasonably possible is critical, does not seem advisable. This may not be what you're suggesting at all, so we may already be on the same page.

Correct -- it's not what I am suggesting at all.  The document and charter pull request are for QUIC, not PLUS.

-d

> - jana
> 
> 
> On Tue, Jun 14, 2016 at 1:08 PM, Joe Hildebrand (jhildebr) <[email-masked] <mailto:[email-masked]>> wrote:
> 
> On 6/14/16, 1:48 PM, "Ted Hardie" <[email-masked] <mailto:[email-masked]>> wrote:
> 
> ><j <mailto:[email-masked] <mailto:[email-masked]>>
> >
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/quic <https://www.ietf.org/mailman/listinfo/quic>
>

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 15 Jun 2016 18:41:09 -0700

Hi Dan,

> draft-wing-quic-network-req tries to up-level its recommendations by
> saying 'we need consent' rather than suggesting how to achieve it.  Consent
> could be achieved in probably 3 or 4 different ways, and we need a
> technique that works with multicast QUIC and with persistent QUIC
> connections.  It's up to WG to agree consent is
>
necessary, then to agree how to accomplish consent.  We have identified
> other things that impact the network and need discussion in the working
> group (some small, some larger):  Should the network clear its state
> immediately after QUIC public reset, set timer, wait for a little while.
> Is the network's sole identification the QUIC version in the
> client-initiated connection, and can we avoid the network treating that
> specially (which will be good and bad, I am remembering network treatment
> of  IKE's UDP 500 and 4500).  If QUIC endpoint sends or receives many bogus
> QUIC packets, how can network help stop or rate limit those, to defend the
> links and defend other hosts on that network.  If path drops state due to
> a timeout (or crash, or software update, or whatever), how should the
> endpoints learn and how should they react.  Those previous things are
> discussed in the I-D.  In addition, QUIC complicates network diagnostics
> and measurements, as well, which will be added in a later version of the
> I-D.
>

I understand what the draft describes, and as I said, I think this is
useful when the wg is discussing what is visible in the QUIC header and
what isn't. The wg can decide whether to care about a particular middlebox
function or not.

That said, I think this may be one of our core disagreements: absent
agreement on what middlebox functions are essential, it's hard to argue how
QUIC should solve for them. We saw a very similar conversation play out in
MPTCP. There were many questions about how legacy middleboxes would treat
MPTCP options and modifications to TCP header bits. The problem was that
there were too many boogeymen middleboxes out there, and it was usually
anybody's guess how important any one particular behavior was. There was
small-scale measurement work done that helped direct the conversation, but
this is tricky territory -- tricky largely because of the lack of
substantial data about particular middlebox behaviors and their prevalence
and importance.

In terms of how these apply to QUIC, we have data from the current
deployment of QUIC, and that can hopefully be brought to bear on the
decisions in the working group. Beyond that, I think it's all thin ice. We
can certainly try and get consensus on specific middlebox behaviors as
important, but that conversation is more general than just QUIC, and since
your draft is precisely such a list, I'd argue that it probably belongs in
PLUS, TSVAREA/TSVWG, or INTAREA.

> middlebox behavior?
> The intent is to change QUIC, as necessary, and to change network devices,
> as necessary, so that QUIC can be successfully deployed on more networks.
>

That QUIC needs to work around middleboxes is already stated in the charter
as "Enabling deployment over unmodified Internet paths". If legacy
middleboxes exist on path, this goal has them covered.

On changing network devices, this may be our other core disagreement. The
charter is proposing to work on QUIC, and not to modify network devices,
again because of what I said above. I think that a generalized form of this
draft is more relevant in a broader forum (as noted earlier), and once the
requirements are made clear, surely the QUIC wg can work on figuring out
how to solve for them in QUIC.

- jana

---

From: Brian Trammell <[email-masked]>
Date: Thu, 16 Jun 2016 11:12:56 +0200

hi Jana, all,

(adding [email-masked])

On one point, below:

> On 16 Jun 2016, at 03:41, Jana Iyengar <[email-masked]> wrote:
> 
> Hi Dan,
> 
>> draft-wing-quic-network-req tries to up-level its recommendations by saying 'we need consent' rather than suggesting how to achieve it.  Consent could be achieved in probably 3 or 4 different ways, and we need a technique that works with multicast QUIC and with persistent QUIC connections.  It's up to WG to agree consent is
>> necessary, then to agree how to accomplish consent.  We have identified other things that impact the network and need discussion in the working group (some small, some larger):  Should the network clear its state immediately after QUIC public reset, set timer, wait for a little while.  Is the network's sole identification the QUIC version in the client-initiated connection, and can we avoid the network treating that specially (which will be good and bad, I am remembering network treatment of  IKE's UDP 500 and 4500).  If QUIC endpoint sends or receives many bogus QUIC packets, how can network help stop or rate limit those, to defend the links and defend other hosts on that network.  If path drops state due to a timeout (or crash, or software update, or whatever), how should the endpoints learn and how should they react.  Those previous things are discussed in the I-D.  In addition, QUIC complicates network diagnostics and measurements, as well, which will be added in a later version of the I-D.
> 
> I understand what the draft describes, and as I said, I think this is useful when the wg is discussing what is visible in the QUIC header and what isn't. The wg can decide whether to care about a particular middlebox function or not.
> 
> That said, I think this may be one of our core disagreements: absent agreement on what middlebox functions are essential, it's hard to argue how QUIC should solve for them. We saw a very similar conversation play out in MPTCP. There were many questions about how legacy middleboxes would treat MPTCP options and modifications to TCP header bits. The problem was that there were too many boogeymen middleboxes out there, and it was usually anybody's guess how important any one particular behavior was. There was small-scale measurement work done that helped direct the conversation, but this is tricky territory -- tricky largely because of the lack of substantial data about particular middlebox behaviors and their prevalence and importance.
> 
> In terms of how these apply to QUIC, we have data from the current deployment of QUIC, and that can hopefully be brought to bear on the decisions in the working group. Beyond that, I think it's all thin ice. We can certainly try and get consensus on specific middlebox behaviors as important, but that conversation is more general than just QUIC, and since your draft is precisely such a list, I'd argue that it probably belongs in PLUS, TSVAREA/TSVWG, or INTAREA.

PLUS BoF proponent hat on: I think there is a point here that might make sense to add to the PLUS draft charter that would naturally feed into QUIC:

- Define a view of "essential middlebox functions", and a description of the information from endpoints required to drive them.

As you say, part of the problem is there is no agreement on either of these questions. A focused discussion here could hopefully come to some consensus. QUIC isn't really the place for this IMO, since the answer is bigger than one instance of a new protocol over UDP. I'm not sure a general INTAREA/TSVAREA activity would have the right focus, and I'm not sure it's really in scope for TSVWG.

draft-wing-quic-network-req looks to me (slightly rearranged and with the QUIC-specific language stripped out) as a candidate document for this work item. It's alsouseful input to the engineering work on a universal shim protocol we intend to do in PLUS. (And as a bonus, if both QUIC and PLUS are working from a common, defined view of what sort of functions are essential, future integration between them won't be complicated by differing architectural views).

Cheers,

Brian

---

From: 🔓Dan Wing <[email-masked]>
Date: Thu, 16 Jun 2016 06:58:25 -0600

> I understand what the draft describes, and as I said, I think this is useful when the wg is discussing what is visible in the QUIC header and what isn't. The wg can decide whether to care about a p…

The document does not discuss just middleboxes.

-d

> That QUIC needs to work around middleboxes is already stated in the charter as "Enabling deployment over unmodified Internet paths". If legacy middleboxes exist on path, this goal has them covered. 
> 
> On changing network devices, this may be our other core disagreement. The charter is proposing to work on QUIC, and not to modify network devices, again because of what I said above. I think that a generalized form of this draft is more relevant in a broader forum (as noted earlier), and once the requirements are made clear, surely the QUIC wg can work on figuring out how to solve for them in QUIC.
> 
> - jana
> 
>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Thu, 16 Jun 2016 14:03:05 +0000

>I understand what the draft describes, and as I said, I think this is useful when the wg is discussing what is visible in the QUIC header and what isn't. The wg can decide whether to care about a pa…

Great.  Let's get a little bit of charter text added that says that, then.  It may be that the text I proposed didn't capture that nuance adequately.

>That said, I think this may be one of our core disagreements: absent agreement on what middlebox functions are essential, it's hard to argue how QUIC should solve for them. We saw a very similar conversation play out in MPTCP. There were many questions
> about how legacy middleboxes would treat MPTCP options and modifications to TCP header bits. The problem was that there were too many boogeymen middleboxes out there, and it was usually anybody's guess how important any one particular behavior was. There was
> small-scale measurement work done that helped direct the conversation, but this is tricky territory -- tricky largely because of the lack of substantial data about particular middlebox behaviors and their prevalence and importance.

We're not talking about boogeyman middleboxes here.  There is no measurement needed, and no testing.

We're talking about firewalls that currently block UDP, such as the one I'm going through right now.  I would like to add explicit QUIC support to that box so that we can get the benefits of QUIC deployment.  I would like any working group we form to do new transport protocols to have in scope how that new transport deploys, and firewall traversal is a key real-world deployment mechanism.

I would be happy with different charter language that allows a potential working group to consider adding language that tells firewalls and load balancers what to do with the bits that are exposed, in order to maximize user experience, and allow firewalls and load balancers to do the best they can to protect the applications, hosts, and networks on whose behalf they are deployed.  I would like a potential working group to be able to discuss what happens when signals from the network path such as ICMP are received, as well as when they should be sent.  That might mean some changes to the protocol, but hopefully most of it is just writing down some of the assumptions that people with more QUIC intuition have about what firewalls and load balancers are going to do with the information that is already exposed to the path.

I agree that any more interesting path interactions, to the extent that they might be interesting, are more in the realm of PLUS.

>In terms of how these apply to QUIC, we have data from the current deployment of QUIC, and that can hopefully be brought to bear on the decisions in the working group. Beyond that, I think it's all thin ice. We can certainly try and get consensus on specific
> middlebox behaviors as important, but that conversation is more general than just QUIC, and since your draft is precisely such a list, I'd argue that it probably belongs in PLUS, TSVAREA/TSVWG, or INTAREA.

I'm fine with working on a generic version of the draft in PLUS or TSVWG.  I am not fine with any discussion of how that draft applies to QUIC being ruled out of charter by a potential QUIC working group.  I believe we need some charter text to protect that conversation.

>On the proposed addition to the charter text, I am unclear on the intent. Is it to document possible breakages, or is the intent to prescribe middlebox behavior?

The intent is to describe how QUIC deploys on the network, including considering what, if any, changes are needed to make that deployment more efficient.

>That QUIC needs to work around middleboxes is already stated in the charter as "Enabling deployment over unmodified Internet paths". If legacy middleboxes exist on path, this goal has them covered. 

I do not believe this is adequate.  The intent is not to describe legacy devices.  The intent is to be able to build new devices that are QUIC-enabled, and to advantage QUIC traffic against large-scale UDP biases that some network operators have in place. 

>On changing network devices, this may be our other core disagreement. The charter is proposing to work on QUIC, and not to modify network devices, again because of what I said above. I think that a generalized form of this draft is more relevant in a broader
> forum (as noted earlier), and once the requirements are made clear, surely the QUIC wg can work on figuring out how to solve for them in QUIC.

QUIC will not be able to be deployed on some networks, such as the one I'm on right now, unless we describe the changes that are necessary for the network devices that are currently blocking it.

-- 
Joe Hildebrand

---

From: Christian Huitema <[email-masked]>
Date: Thu, 16 Jun 2016 17:14:56 +0000

On Thursday, June 16, 2016 7:03 AM, Joe Hildebrand wrote:
> ...
> I do not believe this is adequate.  The intent is not to describe legacy devices.
> The intent is to be able to build new devices that are QUIC-enabled, and to
> advantage QUIC traffic against large-scale UDP biases that some network
> operators have in place.

Joe, I think that we need state the requirements precisely. I heard a wide range of potential requirements in the MARNEW workshop, and then on the SPUD and PLUS mailing list, and then in your draft. I am concerned that general goals like "make QUIC friendly to firewalls" can derive into an endless list of feature requirements. If we write anything at all in the charter, it has to cut to the core of the issue.

I heard two common requirements, "marking the beginning and end of a flow," because that helps firewalls and load balancers manage resource, and "assessing consent," because that helps differentiating between regular traffic and attack traffic. Is there more?

> QUIC will not be able to be deployed on some networks, such as the 
> one I'm on right now, unless we describe the changes that are necessary
> for the network devices that are currently blocking it.

There are such networks. But there also are networks today that block outgoing TCP connections, because of some policy considerations. We don't change TCP for that. We simply accept that these networks are making a tradeoff. They care so much about some of their policies that they are willing to forego some of the value of the Internet. That's probably fine. Over time, the cost/advantage balance of the tradeoff will change, and the policies will probably evolve.

-- Christian Huitema

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Thu, 16 Jun 2016 17:40:06 +0000

>Joe, I think that we need state the requirements precisely. I heard a wide range of potential requirements in the MARNEW workshop, and then on the SPUD and PLUS mailing list, and then in your draft.…

We can enumerate these if you wish, but if we come up with another one that the working group wants to tackle, I’d prefer not to have to recharter.  A little bit of language saying “and similar issues” would be enough for me.

>I heard two common requirements, "marking the beginning and end of a flow," because that helps firewalls and load balancers manage resource, and "assessing consent," because that helps differentiati…

If you want the processing to be efficient (and not ossify something unintended), being able to separate QUIC traffic from non-QUIC UDP traffic easily would be useful.  Otherwise, path devices are likely to grab whatever version number they see as the discriminant.

If “marking the beginning of a flow” also includes being able to tell easily which side intended to initiate the conversation, then that’s fine.

The protocol needs to ensure that a path element can associate further traffic on a QUIC connection with the beginning of the flow.

There probably needs to be recommendations for time-outs, particularly around the end of the flow.

The protocol doc needs to describe what happens with associated ICMP.

The protocol needs to say what happens when a network or host receives a QUIC packet it wasn’t expecting.

We need to think about return routability.

The protocol needs to keep in mind that state will be generated in the network, so that as we add multi-path extensions (for example), we don’t just start sending traffic on a new path without first starting a new flow.

It is possible that we might want to describe what a load balancer should do when it drops QUIC state due to a timeout (or when it is *about* to drop state).  We might want it to send a hint to one or both of the endpoints, to avoid having to diagnose mid-stream black holes.  Some load TCP load balancers send RSTs in both directions in this case, for example.

We should analyze the known attacks on TCP (at least) to ensure that QUIC is not immediately vulnerable to the sorts of things we’ve already fixed over the last 40 years. 

There are probably a couple of others, and I’m open to modifying the list as people see fit.  Alternately, if we can find a way to describe these kind of discussions in a way that does not bring in the explicit path communication bits that we explored in SPUD, I think the charter would be easier to understand. 

>There are such networks. But there also are networks today that block outgoing TCP connections, because of some policy considerations. We don't change TCP for that. We simply accept that these netwo…

Are you saying that people will want QUIC so much that they will open their firewalls up to all UDP traffic in both directions?  Or that the terrible timing-based heuristics we have for generic UDP will be enough for a large enterprise?  Or that the current protocol description, without discussion and analysis, should be presumed to be perfect in this regard?

For once, we have a chance to do some thoughtful engineering before a protocol is fully specified to consider how it will be deployed, baking decades of learning into the design.  I think the approach that the TLS working group has been doing on TLS 1.3 to consider attack vectors, backward-compatibility, etc. is a model for what is needed from a protocol of QUIC’s potential impact.

-- 
Joe Hildebrand

---

From: Brian Trammell <[email-masked]>
Date: Thu, 16 Jun 2016 20:45:11 +0200

hi Joe, Christian, all,

(hatless, inline reply...)

> On 16 Jun 2016, at 19:40, Joe Hildebrand (jhildebr) <[email-masked]> wrote:
> 

It seems to me that a lot of what we're talking about here goes to aligning the operation of QUIC with the expectations of people who actually run networks. We've got lots of running code out there that assumes that "connection-oriented Internet transport protocol" and "TCP" are synonymous. There's the code on firewalls, but that's relatively easy to patch compared to the "code" in the heads of network and system administrators. The closer the control points and network-level information available are equivalent to those available in TCP, the less friction with this existing practice. There are obvious tradeoffs in visibility and exploitability that must be evaulated, but this is engineering the WG should do. In any event, reusing people's intuition about TCP to the extent it fits with QUIC seems useful to me.

Given that, would the following change to the charter (adding one sentence to the paragraph for focus area one) be acceptable?

OLD:

The first of these is the core transport work, which will describe the wire format, along with the mechanisms for connection establishment, stream multiplexing, data reliability, loss detection and recovery, congestion control, version negotiation, and options negotiation. Work on congestion control will describe use of an existing congestion controller as a default scheme for QUIC. QUIC is expected to support rapid iterability and experimentation, and this work will describe a versioning process that enables distributed experimentation with QUIC.

NEW:

The first of these is the core transport work, which will describe the wire format, along with the mechanisms for connection establishment, stream multiplexing, data reliability, loss detection and recovery, congestion control, version negotiation, and options negotiation. This work will consider deployability, manageability and diagnosability of the wire format, including how the wire format will be processed by devices on path. Work on congestion control will describe use of an existing congestion controller as a default scheme for QUIC. QUIC is expected to support rapid iterability and experimentation, and this work will describe a versioning process that enables distributed experimentation with QUIC.

Indeed, I would hope that manageability and diagnosability -- which includes enabling firewalls to keep state, and making it possible to differentiate attack traffic -- are requirements for any IETF transport protocol going forward.

The discussion below, I think, needs to happen in the WG. General guidance here can come from PLUS (and indeed, I see a lot of points raised in the SPUD-derived requirements document draft-trammell-spud-req in Joe's list below), but the detailed evaluation of these aspects of how the protocol will interact with devices on path in the specific case of QUIC are a matter for QUIC. I believe the suggestion above allows that work to happen.

Cheers,

Brian

---

From: <[email-masked]>
Date: Fri, 17 Jun 2016 06:52:15 +0000

Dear Brian, all, 

The NEW proposal is an enhancement but IMHO the text should be explicit that the wire format can be changed accordingly. The decision whether a change is needed and which changes are needed, is up to the WG.

I suggest this modification to the Brian's text:  

OLD:

   This work will consider deployability, manageability and
   diagnosability of the wire format, including how the wire format will
   be processed by devices on path.

NEW:

   This work will consider issues raised by deployability and
   diagnosability, as well as network management implications, including
   how the wire format will be processed by devices on path.  The
   specification of the wire format will be updated to accommodate these
   issues.

Thank you.

Cheers,
Med

> -----Message d'origine-----
> De : QUIC [mailto:[email-masked]] De la part de Brian Trammell
> Envoyé : jeudi 16 juin 2016 20:45
> À : Joe Hildebrand (jhildebr)
> Cc : Christian Huitema; [email-masked]
> Objet : Re: [QUIC] Network Path Requirements for QUIC
> 
> hi Joe, Christian, all,
> 
> >> Joe, I think that we need state the requirements precisely. I heard a
> wide range of potential requirements in the MARNEW workshop, and then on
> the SPUD and PLUS mailing list, and then in your draft. I am concerned
> that general goals like "make QUIC friendly to firewalls" can derive into
> an endless list of feature requirements. If we write anything at all in
> the charter, it has to cut to the core of the issue.
> >
> > We can enumerate these if you wish, but if we come up with another one
> that the working group wants to tackle, I’d prefer not to have to
> recharter.  A little bit of language saying “and similar issues” would be
> enough for me.
> 
> It seems to me that a lot of what we're talking about here goes to
> aligning the operation of QUIC with the expectations of people who
> actually run networks. We've got lots of running code out there that
> assumes that "connection-oriented Internet transport protocol" and "TCP"
> are synonymous. There's the code on firewalls, but that's relatively easy
> to patch compared to the "code" in the heads of network and system
> administrators. The closer the control points and network-level
> information available are equivalent to those available in TCP, the less
> friction with this existing practice. There are obvious tradeoffs in
> visibility and exploitability that must be evaulated, but this is
> engineering the WG should do. In any event, reusing people's intuition
> about TCP to the extent it fits with QUIC seems useful to me.
> 
> Given that, would the following change to the charter (adding one sentence
> to the paragraph for focus area one) be acceptable?
> 
> OLD:
> 
> The first of these is the core transport work, which will describe the
> wire format, along with the mechanisms for connection establishment,
> stream multiplexing, data reliability, loss detection and recovery,
> congestion control, version negotiation, and options negotiation. Work on
> congestion control will describe use of an existing congestion controller
> as a default scheme for QUIC. QUIC is expected to support rapid
> iterability and experimentation, and this work will describe a versioning
> process that enables distributed experimentation with QUIC.
> 
> NEW:
> 
> The first of these is the core transport work, which will describe the
> wire format, along with the mechanisms for connection establishment,
> stream multiplexing, data reliability, loss detection and recovery,
> congestion control, version negotiation, and options negotiation. This
> work will consider deployability, manageability and diagnosability of the
> wire format, including how the wire format will be processed by devices on
> path. Work on congestion control will describe use of an existing
> congestion controller as a default scheme for QUIC. QUIC is expected to
> support rapid iterability and experimentation, and this work will describe
> a versioning process that enables distributed experimentation with QUIC.
> 
> Indeed, I would hope that manageability and diagnosability -- which
> includes enabling firewalls to keep state, and making it possible to
> differentiate attack traffic -- are requirements for any IETF transport
> protocol going forward.
> 
> The discussion below, I think, needs to happen in the WG. General guidance
> here can come from PLUS (and indeed, I see a lot of points raised in the
> SPUD-derived requirements document draft-trammell-spud-req in Joe's list
> below), but the detailed evaluation of these aspects of how the protocol
> will interact with devices on path in the specific case of QUIC are a
> matter for QUIC. I believe the suggestion above allows that work to
> happen.
> 
> 
> Cheers,
> 
> Brian
> 
> 
> >> I heard two common requirements, "marking the beginning and end of a
> flow," because that helps firewalls and load balancers manage resource,
> and "assessing consent," because that helps differentiating between
> regular traffic and attack traffic. Is there more?
> >
> > If you want the processing to be efficient (and not ossify something
> unintended), being able to separate QUIC traffic from non-QUIC UDP traffic
> easily would be useful.  Otherwise, path devices are likely to grab
> whatever version number they see as the discriminant.
> >
> > If “marking the beginning of a flow” also includes being able to tell
> easily which side intended to initiate the conversation, then that’s fine.
> >
> > The protocol needs to ensure that a path element can associate further
> traffic on a QUIC connection with the beginning of the flow.
> >
> > The protocol needs to keep in mind that state will be generated in the
> network, so that as we add multi-path extensions (for example), we don’t
> just start sending traffic on a new path without first starting a new
> flow.
> >
> > It is possible that we might want to describe what a load balancer
> should do when it drops QUIC state due to a timeout (or when it is *about*
> to drop state).  We might want it to send a hint to one or both of the
> endpoints, to avoid having to diagnose mid-stream black holes.  Some load
> TCP load balancers send RSTs in both directions in this case, for example.
> >
> > We should analyze the known attacks on TCP (at least) to ensure that
> QUIC is not immediately vulnerable to the sorts of things we’ve already
> fixed over the last 40 years.
> >
> > There are probably a couple of others, and I’m open to modifying the
> list as people see fit.  Alternately, if we can find a way to describe
> these kind of discussions in a way that does not bring in the explicit
> path communication bits that we explored in SPUD, I think the charter
> would be easier to understand.
> >
> >> There are such networks. But there also are networks today that block
> outgoing TCP connections, because of some policy considerations. We don't
> change TCP for that. We simply accept that these networks are making a
> tradeoff. They care so much about some of their policies that they are
> willing to forego some of the value of the Internet. That's probably fine.
> Over time, the cost/advantage balance of the tradeoff will change, and the
> policies will probably evolve.
> >
> > Are you saying that people will want QUIC so much that they will open
> their firewalls up to all UDP traffic in both directions?  Or that the
> terrible timing-based heuristics we have for generic UDP will be enough
> for a large enterprise?  Or that the current protocol description, without
> discussion and analysis, should be presumed to be perfect in this regard?
> >
> > For once, we have a chance to do some thoughtful engineering before a
> protocol is fully specified to consider how it will be deployed, baking
> decades of learning into the design.  I think the approach that the TLS
> working group has been doing on TLS 1.3 to consider attack vectors,
> backward-compatibility, etc. is a model for what is needed from a protocol
> of QUIC’s potential impact.
> >
> > --
> > Joe Hildebrand
> >

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Fri, 17 Jun 2016 19:55:05 +0000

Your suggested charter text works for me.  I think it is much more clear than what I originally suggested.

-- 
Joe Hildebrand

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 17 Jun 2016 18:26:19 -0700

On Thu, Jun 16, 2016 at 2:12 AM, Brian Trammell <[email-masked]> wrote:

> links and defend other hosts on that network.  If path drops state due to a
> timeout (or crash, or software update, or whatever), how should the
> PLUS BoF proponent hat on: I think there is a point here that might make
> sense to add to the PLUS draft charter that would naturally feed into QUIC:
>
> - Define a view of "essential middlebox functions", and a description of
> the information from endpoints required to drive them.
>
> As you say, part of the problem is there is no agreement on either of
> these questions. A focused discussion here could hopefully come to some
> consensus. QUIC isn't really the place for this IMO, since the answer is
> bigger than one instance of a new protocol over UDP. I'm not sure a general
> INTAREA/TSVAREA activity would have the right focus, and I'm not sure it's
> really in scope for TSVWG.
>
> draft-wing-quic-network-req looks to me (slightly rearranged and with the
> QUIC-specific language stripped out) as a candidate document for this work
> item. It's alsouseful input to the engineering work on a universal shim
> protocol we intend to do in PLUS. (And as a bonus, if both QUIC and PLUS
> are working from a common, defined view of what sort of functions are
> essential, future integration between them won't be complicated by
> differing architectural views).
>

+1.

- jana

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 17 Jun 2016 19:25:46 -0700

Joe,

I'd like to first say that I'm glad we're having this conversation now,
since I expected this conversation to happen sooner or later. We clearly
have differences, but I understand where you're coming from and I want
to thank you for engaging reasonably (not that I would have expected
anything lesser from you.)

Responses inline.

> Great.  Let's get a little bit of charter text added that says that,
> then.  It may be that the text I proposed didn't capture that nuance
> adequately.

I think the wording was unclear, and I think I'm probably fine with Brian's
proposed text (I'll respond to the proposed text separately.) That said,
the charter text is only a placeholder for a conversation we want to have
in the wg. My disagreement is with the conversation that you're proposing.

> MPTCP. There were many questions
> > about how legacy middleboxes would treat MPTCP options and modifications
> to TCP header bits. The problem was that there were too many boogeymen
> middleboxes out there, and it was usually anybody's guess how important any
> one particular behavior was. There was
> > small-scale measurement work done that helped direct the conversation,
> but this is tricky territory -- tricky largely because of the lack of

That's not a convincing argument -- with no measurement or data, there's
only boogeymen middleboxes...

> We're talking about firewalls that currently block UDP, such as the one
> I'm going through right now.  I would like to add explicit QUIC support to
> that box so that

... and that's your boogeyman. Fortunately, there's data on this -- we
shared this in the BarBoF last year. That's about 7% of users. QUIC works
well for 92% of the users trying to use it (this is at massive scales of
usage), as shown on this slide
<https://docs.google.com/presentation/d/15e1bLKYeN56GL1oTJSF9OZiUsI-rcxisLo9dEyDkWQs/edit#slide=id.gc34025b9a_0_0>.
This data has been independently validated by non-Google WebRTC
deployments. So, this question can at least be scoped to 7% of users. I'll
also point out that just being about to identify QUIC is not going to be
enough for the entire 7%, so it's really < 7%.

we can get the benefits of QUIC deployment.  I would like any working group
> we form to do new transport protocols to have in scope how that new
> transport deploys, and firewall traversal is a key real-world deployment
> mechanism.
>
> I would be happy with different charter language that allows a potential
> working group to consider adding language that tells firewalls and load
> balancers what to do with the bits that are exposed, in order to maximize
> user experience, and allow firewalls and load balancers to do the best they
> can to protect the applications, hosts, and networks on whose behalf they
> are deployed.  I would like a potential working group to be able to discuss
> what happens when signals from the network path such as ICMP are received,
> as well as when they should be sent.  That might mean some changes to the
> protocol, but hopefully most of it is just writing down some of the
> assumptions that people with more QUIC intuition have about what firewalls
> and load balancers are going to do with the information that is already
> exposed to the path.
>

I don't agree with this line of reasoning. The QUIC wg should specify what
the endpoints do on receiving ICMP messages from the network. But you are
also asking for the QUIC wg to specify normative behavior of firewalls,
when there is no equivalent consensus on what they ought to do for TCP.
We're talking about solving for a list of problems that has no consensus.
More importantly, my opinion is that this wg may not have the right set of
folks with whom you want to drive consensus on this list of firewall
behaviors. "People with more QUIC intuition" certainly will have some idea
of what behaviors to build for (and I expect that to happen), but these
people will have quite limited idea what firewalls need to do on path, and
what information firewalls need. IMO, you need network operations folks to
be having that conversation, and QUIC is inconveniently not being proposed
in OPSAREA (I'm not sure which wg would be appropriate, which is why I
proposed INTAREA. Perhaps OPSAREA is a better fit.)

I agree that any more interesting path interactions, to the extent that
> they might be interesting, are more in the realm of PLUS.

Why are these less interesting path interactions? Exposing bits to allow
firewalls to do "what they do" is not going to be trivial, since "what they
do" is not agreed upon elsewhere. I would love to believe you, but you must
understand that your word (or yours and Dan's) just isn't enough here.

> can certainly try and get consensus on specific
> > middlebox behaviors as important, but that conversation is more general
> than just QUIC, and since your draft is precisely such a list, I'd argue
> that it probably belongs in PLUS, TSVAREA/TSVWG, or INTAREA.
>
> I'm fine with working on a generic version of the draft in PLUS or TSVWG.
> I am not fine with any discussion of how that draft applies to QUIC being
> ruled out of charter by a potential QUIC working group.  I believe we need
> some charter text to protect that conversation.

Fair enough; though I'll continue to ask for data when the conversation
comes up.

> The intent is to describe how QUIC deploys on the network, including
> considering what, if any, changes are needed to make that deployment more
> efficient.
>
> >That QUIC needs to work around middleboxes is already stated in the
> charter as "Enabling deployment over unmodified Internet paths". If legacy
> I do not believe this is adequate.  The intent is not to describe legacy
> devices.  The intent is to be able to build new devices that are
> QUIC-enabled, and to advantage QUIC traffic against large-scale UDP biases
> that some network operators have in place.

I definitely agree with the spirit, but I worry about this not being scoped
tightly enough to prevent ratholes of various shapes and sizes.

> draft is more relevant in a broader
> > forum (as noted earlier), and once the requirements are made clear,
> surely the QUIC wg can work on figuring out how to solve for them in QUIC.
>
> QUIC will not be able to be deployed on some networks, such as the one I'm
> on right now, unless we describe the changes that are necessary for the
> network devices that are currently blocking it.
>

Understood, and as much as I would love to see the universal deployment of
QUIC, let's be clear that it's not a problem for 92% of users at the moment.

- jana

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 17 Jun 2016 20:11:05 -0700

> wire format, along with the mechanisms for connection establishment, stream
> multiplexing, data reliability, loss detection and recovery, congestion
> control, version negotiation, and options negotiation. Work on congestion
> control will describe use of an existing congestion controller as a default
> scheme for QUIC. QUIC is expected to support rapid iterability and
> experimentation, and this work will describe a versioning process that
> enables distributed experimentation with QUIC.
>
> NEW:
>
> wire format, along with the mechanisms for connection establishment, stream
> multiplexing, data reliability, loss detection and recovery, congestion
> control, version negotiation, and options negotiation. This work will
> consider deployability, manageability and diagnosability of the wire
> format, including how the wire format will be processed by devices on path.
> Work on congestion control will describe use of an existing congestion
> controller as a default scheme for QUIC. QUIC is expected to support rapid

> differentiate attack traffic -- are requirements for any IETF transport

I'm fine with adding this text. That said, I don't think this requirement
that you mention for any IETF transport is clear. What state are we talking
about? I don't know of a clear definition of "attack traffic", though I
very much agree that network operators have to deal with this problem for
UDP. Without concrete requirements, it's hard to define a solution in the
transport.

- jana

> matter for QUIC. I believe the suggestion above allows that work to happen.
>
>
> Cheers,
>
> Brian
>
>
> flow," because that helps firewalls and load balancers manage resource, and
> "assessing consent," because that helps differentiating between regular
> traffic and attack traffic. Is there more?
> >
> just start sending traffic on a new path without first starting a new flow.
> >
> these kind of discussions in a way that does not bring in the explicit path
> communication bits that we explored in SPUD, I think the charter would be
> easier to understand.
> >
> terrible timing-based heuristics we have for generic UDP will be enough for
> a large enterprise?  Or that the current protocol description, without

---

From: Brian Trammell <[email-masked]>
Date: Sun, 19 Jun 2016 14:16:00 +0200

hi Jana,

(hatless, inline)

> On 18 Jun 2016, at 05:11, Jana Iyengar <[email-masked]> wrote:
> 
> 
> I'm fine with adding this text. That said, I don't think this requirement that you mention for any IETF transport is clear. What state are we talking about?

The *bare minimum* here is perhaps best called "connection establishable": (1) what current practice (ab)uses SYN/ACK to mean (the server decided a given connection request was OK enough to continue) and (2) some indication that the entity that sent the first packet on an n-tuple can actually receive traffic on the source address it gave, roughly equivalent to the third ACK leg of the TCP 3WHS.

We should have a longer discussion about how much more state might be useful in the WG; that's what I hope to accomplish with my charter text proposal. But without at least this much being visible to on-path devices, QUIC will be largely incompatible with TCP-centric network management practice, which I'm afraid will come with negative consequences for deployment.

> I don't know of a clear definition of "attack traffic",

Heh. If I knew of one of *those*, that was always network visible and didn't rely on the evil bit, I'd be in a different business. :) But I think there are some basic signals a transport protocol can radiate toward devices on the path that makes it possible for such devices to reduce such traffic, as above.

Cheers,

Brian

> though I very much agree that network operators have to deal with this problem for UDP. Without concrete requirements, it's hard to define a solution in the transport.
> 
> - jana
>

---

From: "Matt Miller (mamille2)" <[email-masked]>
Date: Mon, 20 Jun 2016 16:04:26 +0000

To me it's clear this discussion needs to continue to happen, and needs to be in scope for the proposed WG.  To that end, I think Brian's suggested text does that and ought to be adopted.

--
- m&m

Matt Miller
Cisco Systems, Inc.

> On Jun 19, 2016, at 06:16, Brian Trammell <[email-masked]> wrote:
> 
> hi Jana,
> 
> (hatless, inline)
>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 21 Jun 2016 01:35:39 +0000

On 6/17/16, 10:25 PM, "Jana Iyengar" <[email-masked]> wrote:

>Joe,
>
>I'd like to first say that I'm glad we're having this conversation now, since I expected this conversation to happen sooner or later. We clearly have differences, but I understand where you're coming from and I want to thank you for
> engaging reasonably (not that I would have expected anything lesser from you.) 

Of course let me know if you think I’m being unreasonable at any point; that’s usually where I learn things.

>I think the wording was unclear, and I think I'm probably fine with Brian's proposed text (I'll respond to the proposed text separately.) That said, the charter text is only a placeholder for a conversation we want to have in the wg. My disagreement is with the conversation that you're proposing.

I’m also ok with Brian’s text, and would likely be ok with other changes to his suggested approach.

>That's not a convincing argument -- with no measurement or data, there's only boogeymen middleboxes...

I’m looking into getting a pre-alpha build of a firewall implementing QUIC code available for at least some identified people to test against.  Ideally, we would continue to update that code as a potential working group progressed.

>>We're talking about firewalls that currently block UDP, such as the one I'm going through right now.  I would like to add explicit QUIC support to that box so that
>
>... and that's your boogeyman. 

That’s just a little harsh.  If you want to give me some instrumentation to run against this firewall and those of other people running similar configurations, I’d be happy to help you gather data about what that looks like to the application.  If there is a particular set of data that you would believe that would allow you to care about this user base, let’s discuss what that would look like.

> Fortunately, there's data on this -- we shared this in the BarBoF last year. That's about 7% of users. QUIC works well for 92% of the users trying to use it (this is at massive scales of usage), as shown on this
> slide <https://docs.google.com/presentation/d/15e1bLKYeN56GL1oTJSF9OZiUsI-rcxisLo9dEyDkWQs/edit#slide=id.gc34025b9a_0_0>. This data has been independently validated by non-Google WebRTC deployments. So, this question can at least be scoped to 7% of users. I'll also point out that just being about to identify QUIC is not going to be enough for the entire 7%, so it's
> really < 7%.

I believe that my network, and those of people who believe in using path elements such as firewalls to protect their network, may be under-represented by your metric – or may be a large portion of the 7%.  We would have to dig into your experiment design for me to have a more informed opinion.

I also think the benefits of QUIC should be available for use cases that aren’t a close match for what I think you’ve measured so far.

>I don't agree with this line of reasoning. The QUIC wg should specify what the endpoints do on receiving ICMP messages from the network. But you are also asking for the QUIC wg to specify normative behavior of firewalls, when there is no equivalent consensus
> on what they ought to do for TCP.  

As Brian said in his message, we might be able to agree on a couple of things, particularly having to do with direction of connection, correlation of subsequent packets in the flow, and termination.  But I don’t think we need to decide that list right now.

But, for example, it might be nice for QUIC connections to be able to auto-repair state on a firewall whenever the client sends traffic (which isn’t possible for TCP firewalls that look for a SYN at the start of connection).  It may even be possible with what’s in the current protocol.  It might even be obvious how such a firewall should be implemented to the small group of people that currently has QUIC development experience.  I think you should be allowed to share that experience with us if you’re willing to.

>We're talking about solving for a list of problems that has no consensus. 

We don’t have consensus to form a working group yet, either.  We could whittle down the list to something much more concrete if that would help get consensus on forming a working group.

>More importantly, my opinion is that this wg may not have the right set of folks with whom you want to drive consensus on this list of firewall behaviors.

I’m working to get those people involved.  We haven’t designed a new transport that any of them have cared about recently, so it may take a while to get them interested.  Many of them won’t want to be told that their experience isn’t relevant unless they can run an experiment at the same size as a browser manufacturer.

> "People with more QUIC intuition" certainly will have some idea of what behaviors to build for (and I expect that to happen), but these people will have quite limited idea what firewalls need to do on path, and what information firewalls need. IMO, you need
> network operations folks to be having that conversation, and QUIC is inconveniently not being proposed in OPSAREA (I'm not sure which wg would be appropriate, which is why I proposed INTAREA. Perhaps OPSAREA is a better fit.)

When those people show up at a potential QUIC working group, will they be told that their use cases aren’t interesting, and that their concerns are out of scope anyway?

If what you’re saying is that we can charter work somewhere else in the IETF to deal with some of these questions, I’m open to that.  I’d like us to know what work needs to be chartered in what area, and have charter language for QUIC that says how a potential QUIC working group would interface with that other work.

>Why are these less interesting path interactions? Exposing bits to allow firewalls to do "what they do" is not going to be trivial, since "what they do" is not agreed upon elsewhere. 

Sorry, “more interesting” was an unfortunate choice of words.  What I mean by that was the explicit path-to-application and application-to-path signaling that we explored in the SPUD prototype.  Unless someone else argues we ought to add that to QUIC, it seems clearly out of scope until it has been explored much more fully by a potential PLUS working group or elsewhere.

The “less interesting” bits (to me, comparatively) are the ones we’ve been talking about in this thread.  While I don’t expect this work to be trivial, I don’t expect it to require any fundamental research. 

> I would love to believe you, but you must understand that your word (or
> yours and Dan's) just isn't enough here.

I don’t know how to respond to this. Please understand that Dan and I have reviewed working QUIC firewall code and talked to a lot of people who do both the implementation and deployment work for Cisco’s firewall products.  Very few of those people are willing to participate at the IETF; even if they were, I’m not sure how we could prove to everyone that they should be listened to.

>Fair enough; though I'll continue to ask for data when the conversation comes up.

If that’s going to be the bar that we have to clear for every single conversation we have, I would like us to pre-negotiate what sort of data you will accept, and how you can come to trust that it was gathered in a reasonable way.

>I definitely agree with the spirit, but I worry about this not being scoped tightly enough to prevent ratholes of various shapes and sizes.

Great.  Let’s talk about how to scope it tightly enough.

>Understood, and as much as I would love to see the universal deployment of QUIC, let's be clear that it's not a problem for 92% of users at the moment.

I agree we’re unlikely to get to 100%.  However, I would like the users who are disadvantaged to have the option of one day getting QUIC support.  One of the questions that would be interesting to explore, for example, would be to find out if this 7% is also the set of folks that upgrade their firewalls frequently.

Just writing off more than 200 million users because we weren’t willing to take the time or effort to describe how they could one day use QUIC doesn’t seem like we are doing adequate engineering.

-- 
Joe Hildebrand

---

From: "Tirumaleswar Reddy (tireddy)" <[email-masked]>
Date: Tue, 21 Jun 2016 02:15:04 +0000

> -----Original Message-----
> From: QUIC [mailto:[email-masked]] On Behalf Of Joe Hildebrand
> (jhildebr)
> Sent: Tuesday, June 21, 2016 7:06 AM
> To: Jana Iyengar
> Cc: [email-masked]
> Subject: Re: [QUIC] Network Path Requirements for QUIC
> 
> >since I expected this conversation to happen sooner or later. We
> >clearly have differences, but I understand where you're coming from and
> >I want to thank you for  engaging reasonably (not that I would have
> >expected anything lesser from you.)
> 
> proposed text (I'll respond to the proposed text separately.) That said, the
> charter text is only a placeholder for a conversation we want to have in the
> wg. My disagreement is with the conversation that you're proposing.
> 
> I’m looking into getting a pre-alpha build of a firewall implementing QUIC
> code available for at least some identified people to test against.  Ideally, we
> would continue to update that code as a potential working group progressed.
> 
> >>We're talking about firewalls that currently block UDP, such as the
> >>one I'm going through right now.  I would like to add explicit QUIC
> >>support to that box so that
> >
> That’s just a little harsh.  If you want to give me some instrumentation to run
> against this firewall and those of other people running similar configurations,
> I’d be happy to help you gather data about what that looks like to the
> application.  If there is a particular set of data that you would believe that
> would allow you to care about this user base, let’s discuss what that would
> look like.
> 
> > Fortunately, there's data on this -- we shared this in the BarBoF last
> > year. That's about 7% of users. QUIC works well for 92% of the users
> > trying to use it (this is at massive scales of usage), as shown on this slide
> <https://docs.google.com/presentation/d/15e1bLKYeN56GL1oTJSF9OZiUsI-
> rcxisLo9dEyDkWQs/edit#slide=id.gc34025b9a_0_0>. This data has been
> independently validated by non-Google WebRTC deployments. So, this
> question can at least be scoped to 7% of users. I'll also point out that just
> being about to identify QUIC is not going to be enough for the entire 7%, so
> it's really < 7%.
> 
> I believe that my network, and those of people who believe in using path
> elements such as firewalls to protect their network, may be under-
> represented by your metric – or may be a large portion of the 7%.  We would
> have to dig into your experiment design for me to have a more informed
> opinion.
> 
> I also think the benefits of QUIC should be available for use cases that aren’t
> a close match for what I think you’ve measured so far.
> 
> >I don't agree with this line of reasoning. The QUIC wg should specify
> >what the endpoints do on receiving ICMP messages from the network. But
> you are also asking for the QUIC wg to specify normative behavior of
> firewalls, when there is no equivalent consensus  on what they ought to do
> for TCP.
> 
> As Brian said in his message, we might be able to agree on a couple of things,
> particularly having to do with direction of connection, correlation of
> subsequent packets in the flow, and termination.  But I don’t think we need
> to decide that list right now.
> 
> But, for example, it might be nice for QUIC connections to be able to auto-
> repair state on a firewall whenever the client sends traffic (which isn’t
> possible for TCP firewalls that look for a SYN at the start of connection).  It
> may even be possible with what’s in the current protocol.  It might even be
> obvious how such a firewall should be implemented to the small group of
> people that currently has QUIC development experience.  I think you should
> be allowed to share that experience with us if you’re willing to.
> 
> We don’t have consensus to form a working group yet, either.  We could
> whittle down the list to something much more concrete if that would help get
> consensus on forming a working group.
> 
> behaviors.
> I’m working to get those people involved.  We haven’t designed a new
> transport that any of them have cared about recently, so it may take a while
> to get them interested.  Many of them won’t want to be told that their
> experience isn’t relevant unless they can run an experiment at the same size
> as a browser manufacturer.
> 
> > "People with more QUIC intuition" certainly will have some idea of
> > what behaviors to build for (and I expect that to happen), but these
> > people will have quite limited idea what firewalls need to do on path,
> > and what information firewalls need. IMO, you need network operations
> > folks to be having that conversation, and QUIC is inconveniently not
> > being proposed in OPSAREA (I'm not sure which wg would be appropriate,
> > which is why I proposed INTAREA. Perhaps OPSAREA is a better fit.)
> 
> When those people show up at a potential QUIC working group, will they be
> told that their use cases aren’t interesting, and that their concerns are out of
> scope anyway?
> 
> If what you’re saying is that we can charter work somewhere else in the IETF
> to deal with some of these questions, I’m open to that.  I’d like us to know
> what work needs to be chartered in what area, and have charter language
> for QUIC that says how a potential QUIC working group would interface with
> that other work.
> 
> >>I agree that any more interesting path interactions, to the extent that they
> might be interesting, are more in the realm of PLUS.
> >
> firewalls to do "what they do" is not going to be trivial, since "what they do"
> is not agreed upon elsewhere.
> 
> Sorry, “more interesting” was an unfortunate choice of words.  What I mean
> by that was the explicit path-to-application and application-to-path signaling
> that we explored in the SPUD prototype.  Unless someone else argues we
> ought to add that to QUIC, it seems clearly out of scope until it has been
> explored much more fully by a potential PLUS working group or elsewhere.
> 
> The “less interesting” bits (to me, comparatively) are the ones we’ve been
> talking about in this thread.  While I don’t expect this work to be trivial, I
> don’t expect it to require any fundamental research.
> 
> > I would love to believe you, but you must understand that your word
> > (or yours and Dan's) just isn't enough here.
> 
> I don’t know how to respond to this. Please understand that Dan and I have
> reviewed working QUIC firewall code and talked to a lot of people who do
> both the implementation and deployment work for Cisco’s firewall products.
> Very few of those people are willing to participate at the IETF; even if they
> were, I’m not sure how we could prove to everyone that they should be
> listened to.
> 
> If that’s going to be the bar that we have to clear for every single
> conversation we have, I would like us to pre-negotiate what sort of data you
> will accept, and how you can come to trust that it was gathered in a
> reasonable way.
> 
> >Understood, and as much as I would love to see the universal deployment
> of QUIC, let's be clear that it's not a problem for 92% of users at the moment.
> 
> I agree we’re unlikely to get to 100%.  However, I would like the users who
> are disadvantaged to have the option of one day getting QUIC support.  One
> of the questions that would be interesting to explore, for example, would be
> to find out if this 7% is also the set of folks that upgrade their firewalls
> frequently.
> 
> Just writing off more than 200 million users because we weren’t willing to
> take the time or effort to describe how they could one day use QUIC doesn’t
> seem like we are doing adequate engineering.

WebRTC media streams also had similar problems, were blocked by firewalls. WebRTC added support for consent checks (RFC 7675), Firewalls updated to recognize STUN, and open/close pin-holes(https://tools.ietf.org/html/draft-jennings-behave-rtcweb-firewall-04#section-4.5), TRAM WG introduced TURN extensions (TURN third party authorization RFC 7635) to permit media and data channels over UDP. 

Firewall traversal should be considered in the design of QUIC.

-Tiru

---

From: Christian Huitema <[email-masked]>
Date: Tue, 21 Jun 2016 18:03:41 +0000

On Monday, June 20, 2016 7:15 PM, Tirumaleswar Reddy wrote:
> ...
> 
> WebRTC media streams also had similar problems, were blocked by firewalls.
> WebRTC added support for consent checks (RFC 7675), Firewalls updated to
> recognize STUN, and open/close pin-holes(https://tools.ietf.org/html/draft-
> jennings-behave-rtcweb-firewall-04#section-4.5), TRAM WG introduced TURN
> extensions (TURN third party authorization RFC 7635) to permit media and data
> channels over UDP.
> 

Considered, certainly. But lot of the complexity with WebRTC, ICE and STUN derives from "peer to peer" scenarios, in which both ends are behind firewalls. These peer-to-peer scenarios are not part of the first wave of applications targeted by QUIC, so we can debate whether they belong in the charter.

The first wave of QUIC application is solidly "browser to data center," and we observe that for these scenarios QUIC is traversing "stateful firewalls" just fine. The normal flow of data and acknowledgments is largely sufficient to maintain the firewall state. You mention "consent," but that's very much implicitly granted by the flow of reverse traffic.

The only potential issue there is the case where the reverse traffic is an angry "go away" message. It is obviously not an indication that more content is desired, but since it is encrypted, the firewall cannot tell. On the other hand, we have a specification for a clear text "reset," which the client could send if the other party fails to go away. And the firewall can certainly notice that.

So, in practice, I don't believe that we have much of a problem.

-- Christian Huitema