Subject: Is the invariants draft really standards track?

From: Martin Duke <[email-masked]>
Date: Tue, 26 May 2020 10:55:17 -0700

The only RFC 2119 keywords in this document are related to Version
Negotiation packets, and I suspect these requirements might best be in the
quic-transport draft. (At the very least, the quic-transport draft should
refer to these normative VN requirements hidden in quic-invariants).

If there are really no normative requirements on future versions of QUIC
with respect to long and short headers, etc, than this really is an
informational draft to help middleboxes understand the current IETF
consensus about future versions of QUIC.

I would much prefer that either:
- This draft becomes fully normative on future versions of QUIC ("Future
versions of QUIC MUST NOT mess with this format..."), or
- We pull the VN normative text out of -invariants and make this purely
Informational.

What do others think?

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 27 May 2020 09:18:06 +1000

On Wed, May 27, 2020, at 03:55, Martin Duke wrote:
> The only RFC 2119 keywords in this document 

Just because we don't use MUST (much), it doesn't make the language any less normative.

> What do others think?

I think that this has to be standards track.  It's making a commitment.  If we need more MUSTs, then feel free to propose it.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 27 May 2020 00:08:37 +0000

What Martin said, I say too!
-=R

On 5/26/20, 4:18 PM, "QUIC on behalf of Martin Thomson" <[email-masked] on behalf of [email-masked]> wrote:

---

From: Ian Swett <[email-masked]>
Date: Tue, 26 May 2020 20:10:06 -0400

I also agree with Martin.

On Tue, May 26, 2020 at 8:08 PM Roberto Peon <fenix=[email-masked]>
wrote:

> On 5/26/20, 4:18 PM, "QUIC on behalf of Martin Thomson" <
> [email-masked] on behalf of [email-masked]> wrote:
>
>     I think that this has to be standards track.  It's making a
> commitment.  If we need more MUSTs, then feel free to propose it.
>
>
>

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 26 May 2020 23:23:43 -0700

Which Martin are you both agreeing with?

On Tue, May 26, 2020 at 5:10 PM Ian Swett <ianswett=
[email-masked]> wrote:

> On Tue, May 26, 2020 at 8:08 PM Roberto Peon <fenix=
> [email-masked]> wrote:
>

---

From: Lars Eggert <[email-masked]>
Date: Wed, 27 May 2020 09:50:29 +0300

Hi,

On 2020-5-26, at 20:55, Martin Duke <[email-masked]> wrote:
> - This draft becomes fully normative on future versions of QUIC ("Future versions of QUIC MUST NOT mess with this format..."), or

I agree that it MUST (heh) be normative. If that requires us to stick a 2119 keyword somewhere in there, your suggestion would work.

Lars

---

From: Ian Swett <[email-masked]>
Date: Wed, 27 May 2020 10:34:04 -0400

I was agreeing with MT, but I'm happy to see some more MUSTs added if
people feel that'd be helpful.

On Wed, May 27, 2020 at 2:50 AM Lars Eggert <[email-masked]> wrote:

> I agree that it MUST (heh) be normative. If that requires us to stick a
> 2119 keyword somewhere in there, your suggestion would work.
>
> Lars
>

---

From: Martin Duke <[email-masked]>
Date: Wed, 27 May 2020 08:12:37 -0700

Thanks guys, I'll file an issue and PR, probably next week.

On Wed, May 27, 2020 at 7:34 AM Ian Swett <[email-masked]> wrote:

>> > - This draft becomes fully normative on future versions of QUIC
>> ("Future versions of QUIC MUST NOT mess with this format..."), or
>>

---

From: Kyle Rose <[email-masked]>
Date: Wed, 27 May 2020 11:28:24 -0400

> people feel that'd be helpful.

Coincidentally, we were just talking about this internally at Akamai
yesterday. IMO, an invariants document isn't really helpful if it isn't
normative, and for it to be normative it (or a related practices doc for
operators) really needs to spell out clear boundaries for operators with
MUSTs.

The example that came up yesterday was around operators filtering QUIC in
the event of a DDoS: one recommendation based on some conversations going
back at least to Prague 2019 was to hash packets on 4-tuple and filter
those below a hash value chosen for a desired ingress limit instead of
doing what most operators do with UDP today, which is to cap UDP throughput
and just drop packets randomly or tail drop.

This recommendation certainly imposes some constraints on future protocol
development that motivate new invariants: for instance, it would preclude
sharding a connection across multiple source ports (not that there is
necessarily a reason to do this; it's just an example). But more
importantly, it goes beyond invariants: it's one among many practices
compatible with the current set of invariants, some reasonable and some
terrible.

Operators are going to do things to QUIC traffic, so it would be good to
offer them recommendations that are compatible with broad deployability.

Kyle

---

From: Christian Huitema <[email-masked]>
Date: Wed, 27 May 2020 08:34:12 -0700

On 5/27/2020 8:28 AM, Kyle Rose wrote:
> On Wed, May 27, 2020 at 10:34 AM Ian Swett
> <ianswett=[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     I was agreeing with MT, but I'm happy to see some more MUSTs added
>     if people feel that'd be helpful.
>
>
> yesterday. IMO, an invariants document isn't really helpful if it
> isn't normative, and for it to be normative it (or a related practices
> doc for operators) really needs to spell out clear boundaries for
> operators with MUSTs..
>
> The example that came up yesterday was around operators filtering QUIC
> in the event of a DDoS: one recommendation based on some conversations
> going back at least to Prague 2019 was to hash packets on 4-tuple and
> filter those below a hash value chosen for a desired ingress limit
> instead of doing what most operators do with UDP today, which is to
> cap UDP throughput and just drop packets randomly or tail drop.

Interesting. Did they consider using the CID, or a fraction of it? This
looks entirely like the scenario for which we developed stateless retry.

>
> This recommendation certainly imposes some constraints on future
> protocol development that motivate new invariants: for instance, it
> would preclude sharding a connection across multiple source ports (not
> that there is necessarily a reason to do this; it's just an example).
> But more importantly, it goes beyond invariants: it's one among many
> practices compatible with the current set of invariants, some
> reasonable and some terrible.

This would break the "preferred address" redirection. Preferred address
migration may or may not be spelled out in the invariants.

>
> Operators are going to do things to QUIC traffic, so it would be good
> to offer them recommendations that are compatible with broad
> deployability.

Yes, we do need the invariants for that.

-- Christian Huitema

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 27 May 2020 09:03:38 -0700

I think this is basically one MUST in the invariants document about
conformance. ... and yes, I'm happy to have it too.

On Wed, May 27, 2020 at 8:28 AM Kyle Rose <[email-masked]> wrote:

> MUSTs..

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 27 May 2020 16:06:57 +0000

I’m working on a manageability draft PR for this (how to rate limit UDP to reduce disruption to QUIC if you have to rate limit UDP).  ETA end of the week (if I do not get pulled into something again).

The relevant observation is that DDoS with UDP that is indistinguishable from QUIC will happen.  UDP is already the most prevalent DDoS vector, since it is easy for a compromised non-admin app to send a flood of huge UDP packets (with TCP you get throttled by the congestion controller).  So there WILL be DDoS protection devices out there to try to mitigate the problem, possibly by observing both directions of the flow and deciding whether a packet belongs to a valid flow or not.

Since such middle boxes will be created, the more explicit and normative Invariants are about what one can expect, the less such middle boxes may decide for themselves.  For example (I did not think long about it), if some elements of path validation could land into Invariants (roughly, “no more than X packets/bytes can be sent on a new path w/o a return packet”), a DDoS middle box may use this fact and active connection migration might still have a chance during an attack (NAT rebinding could be linked by DDoS boxes to an old connection via unchanged CID).

  *   Igor

From: Christian Huitema <[email-masked]>
Sent: Wednesday, May 27, 2020 11:34 AM

On 5/27/2020 8:28 AM, Kyle Rose wrote:
On Wed, May 27, 2020 at 10:34 AM Ian Swett <ianswett=[email-masked]<mailto:[email-masked]>> wrote:
I was agreeing with MT, but I'm happy to see some more MUSTs added if people feel that'd be helpful.

Coincidentally, we were just talking about this internally at Akamai yesterday. IMO, an invariants document isn't really helpful if it isn't normative, and for it to be normative it (or a related practices doc for operators) really needs to spell out clear boundaries for operators with MUSTs..

The example that came up yesterday was around operators filtering QUIC in the event of a DDoS: one recommendation based on some conversations going back at least to Prague 2019 was to hash packets on 4-tuple and filter those below a hash value chosen for a desired ingress limit instead of doing what most operators do with UDP today, which is to cap UDP throughput and just drop packets randomly or tail drop.

Interesting. Did they consider using the CID, or a fraction of it? This looks entirely like the scenario for which we developed stateless retry.

This recommendation certainly imposes some constraints on future protocol development that motivate new invariants: for instance, it would preclude sharding a connection across multiple source ports (not that there is necessarily a reason to do this; it's just an example). But more importantly, it goes beyond invariants: it's one among many practices compatible with the current set of invariants, some reasonable and some terrible.

This would break the "preferred address" redirection. Preferred address migration may or may not be spelled out in the invariants.

Operators are going to do things to QUIC traffic, so it would be good to offer them recommendations that are compatible with broad deployability.

-- Christian Huitema

---

From: Jared Mauch <[email-masked]>
Date: Wed, 27 May 2020 12:25:39 -0400

As a former (public) backbone operator we had to deploy these limits due to the volume of the attacks and the disruption it posed to our services. 

There is an ongoing thread about how operators can undo this as people are more pitched against certain vectors. 

QUIC will work until it hits these limiters which may be tied to specific port numbers and such. 

I recommend QUIC folks avoid ephemeral ports that align with prior vectors such as NTP, slammer, memcached amongst others. 

UDP/80 and 443 are also used in attacks so expect operators without a clue to limit them similar to how many didn't realize that DNS was a UDP+TCP protocol for queries as well. 

Sent from my iCar

> On May 27, 2020, at 12:07 PM, Lubashev, Igor <[email-masked]> wrote:
> 
> 
> Igor
> On Wed, May 27, 2020 at 10:34 AM Ian Swett <ianswett=[email-masked]> wrote:

---

From: Paul Vixie <[email-masked]>
Date: Wed, 27 May 2020 20:44:32 +0000

On Wednesday, 27 May 2020 16:06:57 UTC Lubashev, Igor wrote:
> I’m working on a manageability draft PR for this (how to rate limit UDP to
> reduce disruption to QUIC if you have to rate limit UDP).  ETA end of the
> week (if I do not get pulled into something again).
> 
> The relevant observation is that DDoS with UDP that is indistinguishable
> from QUIC will happen.  UDP is already the most prevalent DDoS vector,
> since it is easy for a compromised non-admin app to send a flood of huge
> UDP packets (with TCP you get throttled by the congestion controller).  So
> there WILL be DDoS protection devices out there to try to mitigate the
> problem, possibly by observing both directions of the flow and deciding
> whether a packet belongs to a valid flow or not.

+1.
 
> Since such middle boxes will be created, the more explicit and normative
> Invariants are about what one can expect, the less such middle boxes may
> decide for themselves.  For example (I did not think long about it), if
> some elements of path validation could land into Invariants (roughly, “no
> more than X packets/bytes can be sent on a new path w/o a return packet”),
> a DDoS middle box may use this fact and active connection migration might
> still have a chance during an attack (NAT rebinding could be linked by DDoS
> boxes to an old connection via unchanged CID).

this thinking ties in closely with my own manageability concern, which is 
policy control by managed private (endpoint) networks. right now the quic 
manageability draft is in a state where QUIC traffic is not identifiable, 
which is a goal for some but a problem for others. i'd like to be able to 
permit outbound QUIC through my firewall for most of the ways i do with TCP 
today, but i can't currently identify QUIC flows. i think that's the same 
problem as for your predicted "middlebox" situation.

-- 
Paul