Subject: QUIC as a transport substrate (was:re-chartering: include DNS-over-QUIC?)

From: Spencer Dawkins at IETF <[email-masked]>
Date: Sun, 12 Apr 2020 17:57:04 -0500

Hi, Erik,

Changing subjects because the QUIC chairs revectored the thread to DPRIVE.

On Sun, Apr 12, 2020, 17:09 Erik Kline <[email-masked]> wrote:

> Tangent: is there already, or are there plans for, a document like BCP 56
> but for QUIC?  I'm thinking generally about whether there should be
> guidance for folks wanting to port Foo protocol to Foo-over-QUIC
>

Nice question.

After sitting through MASQUE, WEBTRANS, and maybe even RIPT, I'm not sure
we have consensus on how to do that - basically, the question is whether we
think https://datatracker.ietf.org/doc/draft-ietf-quic-transport/ is the
general-purpose upper interface we want, or not. That's not to say no one
has thoughts about that, only that I don't think everyone has the same
thoughts.

We've even had threads on putting a session layer over QUIC. Or not.

So my own answer would be "I hope so, but we might not be ready to do that
now".

Other people may also have thoughts :-)

Best,

Spencer

---

From: Erik Kline <[email-masked]>
Date: Sun, 12 Apr 2020 23:02:34 -0700

> Other people may also have thoughts :-)

I agree it might be a bit early, but I hope something like this might added
to the TODO pile.

All good things in time.  Thanks,
-ek

This came to mind because I had a silly idea.  As I watched Christian's
presentation in dprive I thought (a) this DoQ doc really benefits from
Christian knowing what he's doing with QUIC, and (b) I wondered if I could
run NTP over QUIC.  Like I said, this is probably silly and there might be
nothing to gain from the attempt, but it got me thinking about guidance for
anyone else with both (a) such an idea and (b) the time to write/implement
a proposal.

---

From: Watson Ladd <[email-masked]>
Date: Sun, 12 Apr 2020 23:18:25 -0700

On Sun, Apr 12, 2020 at 11:03 PM Erik Kline <[email-masked]> wrote:
<chop>
>
> This came to mind because I had a silly idea.  As I watched Christian's presentation in dprive I thought (a) this DoQ doc really benefits from Christian knowing what he's doing with QUIC, and (b) I wondered if I could run NTP over QUIC.  Like I said, this is probably silly and there might be nothing to gain from the attempt, but it got me thinking about guidance for anyone else with both (a) such an idea and (b) the time to write/implement a proposal.

NTP has a very unique property: one packet is sent every 1024 or 512
seconds or so by the client. A single server can handle a high packet
rate easily because of the statelessness. Adding state would start
getting problematic at the high packet rates.

You also don't want automatic retransmission in NTP, and the flow
control is essentially nonexistent: it's driven by the synchronization
loop.

Getting through middleboxes isn't nothing! But that's more a new port,
and ideally you want middleboxes to correct for packet queuing times.

Sincerely,
Watson Ladd

---

From: Tal Mizrahi <[email-masked]>
Date: Mon, 13 Apr 2020 12:22:53 +0300

Hi,

> > This came to mind because I had a silly idea.  As I watched Christian's presentation in dprive I thought (a) this DoQ doc really benefits from Christian knowing what he's doing with QUIC, and (b)…

NTP-over-QUIC is an interesting idea.

The main benefits I can think of of running NTP over QUIC are enjoying
the inherent security of QUIC, and avoiding middlebox tampering.
I wonder if these benefits are worth the effort.

On the other side running NTP over QUIC may consume significantly
higher resources on NTP servers than are necessary today. NTP servers
try to minimize the per-client state (or keep it zero if possible),
while QUIC may require per-client state in order to allow 0-RTT. Also
the precision of NTP may be lower than existing implementations if
implemented in user space.

> control is essentially nonexistent: it's driven by the synchronization

One may consider using QUIC in datagram mode, and this would prevent
retransmissions. ACKs would still be sent in this case.

Cheers,
Tal.

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Mon, 13 Apr 2020 02:30:30 -0700

I think these observations on resource usage are correct, but changing
system time could be a significant attack vector. For NTP to make sense
over QUIC, the datagram extension would be needed to avoid much of the
state and retransmission overhead. A simplified implementation could even
forego a lot of the QUIC machinery such as streams and only rely on the
handshake and datagram. Of course you could also use DTLS but that is yet
another stack.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 13 April 2020 at 11.23.06, Tal Mizrahi ([email-masked]) wrote:

Hi,

Cheers,
Tal.

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 13 Apr 2020 12:58:26 +0100

Hey Erik,

You make some good observations, see response below.

On Mon, 13 Apr 2020, 07:02 Erik Kline, <[email-masked]> wrote:

>
> I agree it might be a bit early, but I hope something like this might
> added to the TODO pile.
>

I think the QUIC Applicability [1] document is probably closest aligned to
the points you raised:

"This document discusses the applicability of the QUIC transport protocol,
focusing on caveats impacting application protocol development and
deployment over QUIC. Its intended audience is designers of application
protocol mappings to QUIC, and implementors of these application protocols."

This document sometimes gets less attention so I'd encourage everyone to
provide feedback and input while the challenges are fresh in people's minds.

Cheers
Lucas

[1] - https://tools.ietf.org/html/draft-ietf-quic-applicability-06

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 13 Apr 2020 07:19:31 -0500

Hi, Erik,

On Mon, Apr 13, 2020, 01:02 Erik Kline <[email-masked]> wrote:

>>> Tangent: is there already, or are there plans for, a document like BCP
>>> 56 but for QUIC?  I'm thinking generally about whether there should be
>> So my own answer would be "I hope so, but we might not be ready to do
>> that now".
>>

This is also something that might be QUICDISPATCHed, if that's the way
people want to roll, when it's time to move forward.

> anyone else with both (a) such an idea and (b) the time to write/implement

You silly ideas are better than many of my serious ideas.

One theme that rolled around when we were chartering QUIC in 2016, was to
charter for HTTP, to allow Google to use IETF QUIC instead of their own
QUIC protocol, but to charter for a "completely different" application next
- DNS and RTP/media were the most frequently mentioned. At that point, the
theory went, we'd know that QUIC was a reasonable general-purpose transport
protocol.

That was several IESGs ago, but it seemed reasonable to us then.

Whenever we convince ourselves that that's what QUIC is, a BCP56-style
guide for QUIC might make perfect sense.

Best,

Spencer

---

From: Erik Kline <[email-masked]>
Date: Mon, 13 Apr 2020 10:27:56 -0700

> [1] - https://tools.ietf.org/html/draft-ietf-quic-applicability-06

Aha!  This does look like great doc for what I had in mind.

Thanks all!

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Tue, 14 Apr 2020 11:54:47 -0700

You don’t need the datagram extension, rather, you need a promise to not
retransmit or a promise to communicate the time in any retransmission.

I don’t understand this promise. In QUIC v1 you have streams, pings,
bookkeeping frames, and extensions. You cannot promise not to retransmit
something in as stream except by terminating the stream. Pings are not
suitable for conveying time information unless perhaps its mere arrival
acts a clock tick. You can of course create a new stream for each new time
event but it still doesn’t prevent retransmission unless you cancel, and if
you cancel, delivery of the payload is not guaranteed.

You could define a promise through a new extension with the desired
semantics but that is close enough to datagrams that it doesn’t make much
sense - but you might want a datagram that does not have ACKs - this
affects flow control, but since transmission is infrequent, that could work.

Something else:

It is hard to prevent an attacker from dropping or delay packets even on an
encrypted QUIC connection and a retransmission would not be a useful
mitigation, so I am not sure how you can make NTP truly secure. A voting
system from multiple sources could make such an attack more difficult.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 14 April 2020 at 19.40.00, Roberto Peon ([email-masked]) wrote:

You don’t need the datagram extension, rather, you need a promise to not
retransmit or a promise to communicate the time in any retransmission.
While datagram provides, implicitly today, a promise to not retransmit,
datagrams themselves aren’t the requirement.
(Also, in the face of multipath, is it true that we won’t have duplicates
even for datagrams. I’m not sure this is clear right now?)

If the first packet has higher measurable (latency) overhead in the
processing stack, then you probably don’t want to use that one for NTP if
you’re high time accuracy.
In other words, you may find that you get better accuracy if you make a
connection, and then do the NTP “stuff”. Similarly, you may find you get
better accuracy if you **always** do a handshake on every packet (though
that is more expensive)…

While QUIC is a wonderful hammer, I do wonder if it provides any
significant benefit in this case.

-=R

*From: *QUIC <[email-masked]> on behalf of Mikkel Fahnøe Jørgensen <
[email-masked]>
*Date: *Monday, April 13, 2020 at 2:31 AM
*To: *Erik Kline <[email-masked]>, Tal Mizrahi <[email-masked]>,
Watson Ladd <[email-masked]>
*Cc: *"[email-masked]" <[email-masked]>, IETF QUIC WG <[email-masked]>, Spencer
Dawkins at IETF <[email-masked]>, Lucas Pardue <
[email-masked]>
*Subject: *Re: NTP is unique (was Re: QUIC as a transport substrate
(was:re-chartering: include DNS-over-QUIC?))

Kind Regards,

Mikkel Fahnøe Jørgensen

Hi,

Cheers,
Tal.

---

From: Kyle Rose <[email-masked]>
Date: Tue, 14 Apr 2020 15:01:11 -0400

On Tue, Apr 14, 2020 at 2:54 PM Mikkel Fahnøe Jørgensen <[email-masked]>
wrote:

>
> It is hard to prevent an attacker from dropping or delay packets even on
> an encrypted QUIC connection and a retransmission would not be a useful

Depends on your threat model. In this case, if the RTT is too high, you can
just discard the response. An attacker can't skew your clock by very much.
NTP is a good demonstration of the principle that proper security is not a
black box.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 14 Apr 2020 19:01:58 +0000

I agree that Qv1 doesn’t have an ability to express this, since Qv1 doesn’t allow for partial reliability except in extensions.
Regardless, if the requirement is to have no retransmission, then that is the requirement, and should be (at least in a spec) written that way.

I’ll disagree on whether that is similar to datagrams. Datagrams don’t have the same “address space” as does a stream, and there are certainly many things out there that would prefer to have a stream-with-holes (i.e. an address space that every party to the interaction agrees upon) than require the application to do reassembly of everything (and caching) on its own. This is really a separate conversation we can move somewhere else, though, assuming it is work having at all!
-=R

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Tuesday, April 14, 2020 at 11:54 AM
To: Erik Kline <[email-masked]>, Watson Ladd <[email-masked]>, Roberto Peon <[email-masked]>, Tal Mizrahi <[email-masked]>
Cc: IETF QUIC WG <[email-masked]>, "[email-masked]" <[email-masked]>, Lucas Pardue <[email-masked]>, Spencer Dawkins at IETF <[email-masked]>
Subject: Re: NTP is unique (was Re: QUIC as a transport substrate (was:re-chartering: include DNS-over-QUIC?))

You don’t need the datagram extension, rather, you need a promise to not retransmit or a promise to communicate the time in any retransmission.
I don’t understand this promise. In QUIC v1 you have streams, pings, bookkeeping frames, and extensions. You cannot promise not to retransmit something in as stream except by terminating the stream. Pings are not suitable for conveying time information unless perhaps its mere arrival acts a clock tick. You can of course create a new stream for each new time event but it still doesn’t prevent retransmission unless you cancel, and if you cancel, delivery of the payload is not guaranteed.

You could define a promise through a new extension with the desired semantics but that is close enough to datagrams that it doesn’t make much sense - but you might want a datagram that does not have ACKs - this affects flow control, but since transmission is infrequent, that could work.

Something else:

It is hard to prevent an attacker from dropping or delay packets even on an encrypted QUIC connection and a retransmission would not be a useful mitigation, so I am not sure how you can make NTP truly secure. A voting system from multiple sources could make such an attack more difficult.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 14 April 2020 at 19.40.00, Roberto Peon ([email-masked]<mailto:[email-masked]>) wrote:
You don’t need the datagram extension, rather, you need a promise to not retransmit or a promise to communicate the time in any retransmission.
While datagram provides, implicitly today, a promise to not retransmit, datagrams themselves aren’t the requirement.
(Also, in the face of multipath, is it true that we won’t have duplicates even for datagrams. I’m not sure this is clear right now?)

If the first packet has higher measurable (latency) overhead in the processing stack, then you probably don’t want to use that one for NTP if you’re high time accuracy.
In other words, you may find that you get better accuracy if you make a connection, and then do the NTP “stuff”. Similarly, you may find you get better accuracy if you *always* do a handshake on every packet (though that is more expensive)…

While QUIC is a wonderful hammer, I do wonder if it provides any significant benefit in this case.

-=R

From: QUIC <[email-masked]<mailto:[email-masked]>> on behalf of Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>
Date: Monday, April 13, 2020 at 2:31 AM
To: Erik Kline <[email-masked]<mailto:[email-masked]>>, Tal Mizrahi <[email-masked]<mailto:[email-masked]>>, Watson Ladd <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, IETF QUIC WG <[email-masked]<mailto:[email-masked]>>, Spencer Dawkins at IETF <[email-masked]<mailto:[email-masked]>>, Lucas Pardue <[email-masked]<mailto:[email-masked]>>
Subject: Re: NTP is unique (was Re: QUIC as a transport substrate (was:re-chartering: include DNS-over-QUIC?))

I think these observations on resource usage are correct, but changing system time could be a significant attack vector. For NTP to make sense over QUIC, the datagram extension would be needed to avoid much of the state and retransmission overhead. A simplified implementation could even forego a lot of the QUIC machinery such as streams and only rely on the handshake and datagram. Of course you could also use DTLS but that is yet another stack.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 13 April 2020 at 11.23.06, Tal Mizrahi ([email-masked]<mailto:[email-masked]>) wrote:
Hi,

Cheers,
Tal.

On Mon, Apr 13, 2020 at 9:18 AM Watson Ladd <[email-masked]<mailto:[email-masked]>> wrote:
>
> On Sun, Apr 12, 2020 at 11:03 PM Erik Kline <[email-masked]<mailto:[email-masked]>> wrote:
> <chop>
> >

---

From: Christian Huitema <[email-masked]>
Date: Tue, 14 Apr 2020 17:16:00 -0700

We also have timestamp frames. That may be related to NTP.

-- Christian Huitema 

> On Apr 14, 2020, at 12:03 PM, Roberto Peon <[email-masked]> wrote:
> 
> 
> From: QUIC <[email-masked]> on behalf of Mikkel Fahnøe Jørgensen <[email-masked]>
> To: Erik Kline <[email-masked]>, Tal Mizrahi <[email-masked]>, Watson Ladd <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>, IETF QUIC WG <[email-masked]>, Spencer Dawkins at IETF <[email-masked]>, Lucas Pardue <[email-masked]>

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 15 Apr 2020 01:23:32 -0700

Umm - where are these timestamp frames?

As to partially reliable streams. I am a big advocagte of these, but I do
not think that they are essential to time data because the payload - time -
is naturally providing a sequential marker. Thus you can avoid waiting for
QUIC v2 and you can avoid the additional complexity. The receiver can just
ignore receipt of out of order messages in, for example, datagram frames.
Datagrams do and do not have identifiers - the intention was not to remove
them, but to let the application provide them as early data in the frame.
If the connection only deals with NTP and you only have one unreliable
stream, you don’t even need an identifier because all datagrams implicitly
belong to the same stream.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 15 April 2020 at 02.16.10, Christian Huitema ([email-masked]) wrote:

-- Christian Huitema

I agree that Qv1 doesn’t have an ability to express this, since Qv1 doesn’t
allow for partial reliability except in extensions.
Regardless, if the requirement is to have no retransmission, then that is
the requirement, and should be (at least in a spec) written that way.

I’ll disagree on whether that is similar to datagrams. Datagrams don’t have
the same “address space” as does a stream, and there are certainly many
things out there that would prefer to have a stream-with-holes (i.e. an
address space that every party to the interaction agrees upon) than require
the application to do reassembly of everything (and caching) on its own.
This is really a separate conversation we can move somewhere else, though,
assuming it is work having at all!
-=R

*From: *Mikkel Fahnøe Jørgensen <[email-masked]>
*Date: *Tuesday, April 14, 2020 at 11:54 AM
*To: *Erik Kline <[email-masked]>, Watson Ladd <[email-masked]>,
Roberto Peon <[email-masked]>, Tal Mizrahi <[email-masked]>
*Cc: *IETF QUIC WG <[email-masked]>, "[email-masked]" <[email-masked]>, Lucas
Pardue <[email-masked]>, Spencer Dawkins at IETF <
[email-masked]>
*Subject: *Re: NTP is unique (was Re: QUIC as a transport substrate
(was:re-chartering: include DNS-over-QUIC?))

Something else:

Kind Regards,

Mikkel Fahnøe Jørgensen

-=R

Kind Regards,

Mikkel Fahnøe Jørgensen

Hi,

Cheers,
Tal.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 15 Apr 2020 08:28:20 +0000

I've seen a couple of applications where we had to do ntp or ntp-like things, but wanted to ensure that we were talking to the same server whom we were transacting requests.

You need something to ensure the routing continues to work in that particular case, especially if bouncing through a reverse proxy which may not otherwise understand your application.

-=R

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Mikkel Fahnøe Jørgensen <[email-masked]>
Sent: Wednesday, April 15, 2020 1:23:32 AM
To: Christian Huitema <[email-masked]>; Roberto Peon <[email-masked]>
Cc: IETF QUIC WG <[email-masked]>; [email-masked] <[email-masked]>; Watson Ladd <[email-masked]>; Spencer Dawkins at IETF <[email-masked]>; Lucas Pardue <[email-masked]>; Tal Mizrahi <[email-masked]>; Erik Kline <[email-masked]>
Subject: Re: NTP is unique (was Re: QUIC as a transport substrate (was:re-chartering: include DNS-over-QUIC?))

As to partially reliable streams. I am a big advocagte of these, but I do not think that they are essential to time data because the payload - time - is naturally providing a sequential marker. Thus you can avoid waiting for QUIC v2 and you can avoid the additional complexity. The receiver can just ignore receipt of out of order messages in, for example, datagram frames. Datagrams do and do not have identifiers - the intention was not to remove them, but to let the application provide them as early data in the frame. If the connection only deals with NTP and you only have one unreliable stream, you don’t even need an identifier because all datagrams implicitly belong to the same stream.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 15 April 2020 at 02.16.10, Christian Huitema ([email-masked]<mailto:[email-masked]>) wrote:

-- Christian Huitema

On Apr 14, 2020, at 12:03 PM, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:

From: Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 14, 2020 at 11:54 AM
To: Erik Kline <[email-masked]<mailto:[email-masked]>>, Watson Ladd <[email-masked]<mailto:[email-masked]>>, Roberto Peon <[email-masked]<mailto:[email-masked]>>, Tal Mizrahi <[email-masked]<mailto:[email-masked]>>
Cc: IETF QUIC WG <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Lucas Pardue <[email-masked]<mailto:[email-masked]>>, Spencer Dawkins at IETF <[email-masked]<mailto:[email-masked]>>
Subject: Re: NTP is unique (was Re: QUIC as a transport substrate (was:re-chartering: include DNS-over-QUIC?))

Something else:

Kind Regards,

Mikkel Fahnøe Jørgensen

-=R

Kind Regards,

Mikkel Fahnøe Jørgensen

Hi,

Cheers,
Tal.

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 15 Apr 2020 01:46:22 -0700

I do agree on all that but can you please clarify what you mean to address
by that comment?

Is it merely a vote for NTP over QUIC, or are you saying that a connection
carrying NTP only is insufficient because you need to link the time data to
other application protocol specific content?

In the latter case this is a concern I share - how do you mix multiple
application protocols on the same connection? QUIC consensus appears to be
that you do don’t do that except where the two, or more, protocols are
designed explicitly to coexist. Thus you could have an HTTP/3 extension for
time.

It would be possible to design an NTP extension for QUIC that is very
similar to datagrams but explicitly designed for NTP under the assumption
that NTP like data is frequently needed by other application protocols. In
that case is simply a new extension frame. I am not opposed to that - but
using datagrams is simpler for common case of just NTP over a connection.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 15 April 2020 at 10.28.23, Roberto Peon ([email-masked]) wrote:

I've seen a couple of applications where we had to do ntp or ntp-like
things, but wanted to ensure that we were talking to the same server whom
we were transacting requests.

You need something to ensure the routing continues to work in that
particular case, especially if bouncing through a reverse proxy which may
not otherwise understand your application.

-=R

Get Outlook for Android <https://aka.ms/ghei36>

------------------------------
*From:* Mikkel Fahnøe Jørgensen <[email-masked]>
*Sent:* Wednesday, April 15, 2020 1:23:32 AM
*To:* Christian Huitema <[email-masked]>; Roberto Peon <[email-masked]>
*Cc:* IETF QUIC WG <[email-masked]>; [email-masked] <[email-masked]>; Watson
Ladd <[email-masked]>; Spencer Dawkins at IETF <
[email-masked]>; Lucas Pardue <[email-masked]>;
Tal Mizrahi <[email-masked]>; Erik Kline <[email-masked]>
*Subject:* Re: NTP is unique (was Re: QUIC as a transport substrate
(was:re-chartering: include DNS-over-QUIC?))

Kind Regards,
Mikkel Fahnøe Jørgensen

-- Christian Huitema

Something else:

Kind Regards,

Mikkel Fahnøe Jørgensen

-=R

Kind Regards,

Mikkel Fahnøe Jørgensen

Hi,

Cheers,
Tal.