Subject: Draft time stamps for Quic

From: Christian Huitema <[email-masked]>
Date: Mon, 24 Feb 2020 22:05:44 -0800

After I submitted draft-huitema-quic-1wd-00, we started a discussion on
the possibility to compose extensions. The obvious way to do that is to
define a "time stamp" frame instead of modifying the ACK frame. As long
as both are in the same packet, we get the same measurements. The added
benefit is that time stamps could be composed with other frames too --
such as for example path responses, to measure one way delays when
evaluating multiple migration options.

-- Christian Huitema

-------- Forwarded Message --------
Subject: 	New Version Notification for draft-huitema-quic-ts-00.txt
Date: 	Mon, 24 Feb 2020 21:58:41 -0800
From: 	[email-masked]
To: 	Christian Huitema <[email-masked]>

A new version of I-D, draft-huitema-quic-ts-00.txt
has been successfully submitted by Christian Huitema and posted to the
IETF repository.

Name: draft-huitema-quic-ts
Revision: 00
Title: Quic Timestamps For Measuring One-Way Delays
Document date: 2020-02-24
Group: Individual Submission
Pages: 6
URL: https://www.ietf.org/internet-drafts/draft-huitema-quic-ts-00.txt
Status: https://datatracker.ietf.org/doc/draft-huitema-quic-ts/
Htmlized: https://tools.ietf.org/html/draft-huitema-quic-ts-00
Htmlized: https://datatracker.ietf.org/doc/html/draft-huitema-quic-ts

Abstract:
The TimeStamp frame can be added to Quic packets when one way delay
measurements is useful. The timestamp is set to the number of
microseconds from the beginning of the connection to the time at
which the packet is sent. The draft defines the "enable_time_stamp"
transport parameter for negotiating the use of this extension frame,
and a new frame types for the time_stamped frame.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Dmitri Tikhonov <[email-masked]>
Date: Tue, 25 Feb 2020 08:30:09 -0500

Great, this is a much lower barrier to entry than modifying the ACK
frame!

How is "the beginning of the connection" defined?  Is it as follows:

    - for the client, the time when the first Initial packet
      is sent;
    - for the server, the time when the first Initial packet
      is received.

  - Dmitri.

---

From: Christian Huitema <[email-masked]>
Date: Tue, 25 Feb 2020 20:47:37 -0800

> Great, this is a much lower barrier to entry than modifying the ACK

I am still a bit saddened by the increased overhead, but if that's the
price of interop...

>     - for the server, the time when the first Initial packet

Yes. Will add that text in the next revision.

-- Christian Huitema

---

From: Watson Ladd <[email-masked]>
Date: Tue, 25 Feb 2020 21:19:06 -0800

So I was under the impression that one cannot determine whether
asymmetric delay or clock skew is the problem between two machines.
There are as many variables as unknowns. At best you can determine if
the one way delay is changing on a leg. Having spent a good part of
the last month on a project that would be much simpler if that wasn't
case I'd be happy to know I was wrong.

Sincerely,
Watson

---

From: Tal Mizrahi <[email-masked]>
Date: Wed, 26 Feb 2020 08:57:59 +0200

Hi Christian,

Thanks for putting this draft together.

Two aspects that may be worth a bit more consideration:
- One-way-delay measurement requires the endpoints to be synchronized.
Since the epoch that was chosen in this draft is the beginning of the
connection, endpoints need to be synchronized about the beginning time
of the connection. The draft suggests to synchronize the beginning of
the connection by measuring the "phase_shift" value. Using
"latest_rtt/2" for the phase_shift calculation seems to assume that
the delay is symmetric. I agree with Watson's comment about symmetry -
if the path is asymmetric all the measurements will be biased.
- "We understand that clocks may drift over time, and that simply
estimating a phase shift at the beginning of a connection may be too
simplistic" - this is an important issue. It is not uncommon for PC
clocks to drift by 1 millisecond (or more) per minute. Accurate
measurement will probably require a more accurate synchronization
mechanism.

Cheers,
Tal.

---

From: Christian Huitema <[email-masked]>
Date: Wed, 26 Feb 2020 10:30:40 -0800

Tal, you and Watson are making good points. The draft is probably
missing a section discussing the issues of time synchronization. Here is
something I could write:

There is a two known issues with deducing one way delays from RTT
measurements: clock drift and undefined phase difference.

The phase difference problem is easy to understand. We start from a list
of measurements associating the send time of packet number x (s[x]), the
receive time of the acknowledgement of packet (a[x]), and the time stamp
indicating when packet x was received by the peer (p[x]). The peer's
time stamp are expressed in the peer's clock. 

Suppose that we model the peer's clock as local time plus phase
difference f, and that we model the rtt as the sum of two one way
delays, up (u[x]) and down (d[x]). We get:

    u[x] = p[x] + f - s[x]

    d[x] = a[x] - p[x] - f

Just looking at the equation shows that the value of f cannot be
determined from the a series of measurement (s[x], a[x], p[x]). You can
just add constraints that all u[x] and d[x] are positive numbers, which
gives a range of plausible values for f:   max(s[x] - p[x]) < f <
min(a[x]-p[x]). In case you wonder, you get similar formulations in a
multipath scenario. The plausible range may narrow to the min rtt of the
shortest path, but no further.

The phase difference uncertainty is not a big issue in practice, because
control algorithms are much more interested in the variations of the
delays than by their absolute values. Suppose we want to compare one way
delays at measurement (x) and (y). we get:

    u[x] = p[x] + f - s[x]

    u[y] = p[y] + f - s[y]

    u[x] - u[y] = p[x] - p[y] - s[x] + s[y]

The phase difference does not affect the measurement of deltas.

The clock drift is another matter. All the equations above assume that
the local clock and the remote clock have the same frequency. This is an
approximation. Clocks drift over time. Instead of just considering a
stable phase difference, one should consider the sum of a phase
difference and a time-varying drift component. Estimating drift is a
complex problem. This was studied in detail in the development of the
network time protocol (NTP) (RFC 5905). In theory, implementations of
Quic could copy the algorithms of NTP to build a model of the clocks
used by the local node and the peer. That would be very complex.

Hopefully, implementations of Quic no not need to implement something as
complex as NTP. Most time based algorithms are only interested in
variations of delays over a short horizon. Clock drift happens at a slow
pace, maybe 1 millisecond per minute. Algorithms like BBR or LEDBAT
already have to cope with the potential drift of the minimum RTT due to
changing network conditions. They do that by periodically restarting the
measurement of the minimum RTT after some delay, typically less than a
minute. A simple implementation of one way delay measurements could
follow the same approach, for example resetting the phase difference
very 30 seconds or so.