Subject: [Ntp] comments on draft-mlichvar-ntp-ntpv5-03

From: Dan Drown <[email-masked]>
Date: Tue, 23 Nov 2021 13:15:01 -0600

After reading this draft, I'm glad to see these things are included:

1. the unknown leap seconds flag seems useful for embedded systems  
that don't have easy access to that information
2. the explicit interleaved mode with server cookies
3. client timestamps are switched to a client cookie
4. the removal of all modes except client/server

I have a few comments, and I think I'll break them up into multiple  
emails to make discussion easier.

This email will be on the "3. Data Types" section.

For the time16 definition "the maximum value is 32767/32768" This  
confused me on first read because I parsed it as either/or at first  
which made no sense. I had to think about it until I realized this was  
talking about a fraction. I'm proposing a change to: "the maximum  
value is the fraction 32767/32768 (almost 1 second)"

Should the log2 format be explained in more detail in this section?  
I've seen a lot of people confused about this data type, so I'm hoping  
a definition would help. Maybe something like "log2 - an 8 bit signed  
integer representing the rounded log2 value of seconds. For example, a  
log2 value of -2 would be 2 to the power of -2 or 0.25 seconds"

Should the change from NTPv4's time32 format be mentioned here?   
Something like:
"A 32-bit fixed-point type containing values in seconds. It has 4  
unsigned integer bits and 28 fractional bits. The maximum value is 16  
seconds and the resolution is about 3.7 nanoseconds. Note: this is  
different than the 32 bit time format in NTPv4 (RFC5905)"

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 24 Nov 2021 08:32:09 +0100

>>> Dan Drown <[email-masked]> schrieb am 23.11.2021 um 20:15 in Nachricht
<[email-masked]>:
> a definition would help. Maybe something like "log2 ‑ an 8 bit signed  
> log2 value of ‑2 would be 2 to the power of ‑2 or 0.25 seconds"
> 
> "A 32‑bit fixed‑point type containing values in seconds. It has 4  

I wonder: Up to NTPv3 floating-point math was avoided, but starting in NTPv4
floating-point math is used.
Do we want to use floating-point math in NTPv5, too? What about those embedded
systems?
If so, are those many special fixed-point formats justified (outside of
packets)?

Regards,
Ulrich

> 
> _______________________________________________
> ntp mailing list
> [email-masked] 
> https://www.ietf.org/mailman/listinfo/ntp

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 24 Nov 2021 09:24:55 +0100

On Tue, Nov 23, 2021 at 01:15:01PM -0600, Dan Drown wrote:
> I have a few comments, and I think I'll break them up into multiple emails
> to make discussion easier.
> 

All three suggestions make sense to me. Should make it clearer.
Thanks!

> For the time16 definition "the maximum value is 32767/32768" This confused
> me on first read because I parsed it as either/or at first which made no
> sense. I had to think about it until I realized this was talking about a
> fraction. I'm proposing a change to: "the maximum value is the fraction
> 32767/32768 (almost 1 second)"
> 
> Should the log2 format be explained in more detail in this section? I've
> seen a lot of people confused about this data type, so I'm hoping a
> definition would help. Maybe something like "log2 - an 8 bit signed integer
> representing the rounded log2 value of seconds. For example, a log2 value of
> -2 would be 2 to the power of -2 or 0.25 seconds"
> 
> Should the change from NTPv4's time32 format be mentioned here?  Something
> like:
> "A 32-bit fixed-point type containing values in seconds. It has 4 unsigned
> integer bits and 28 fractional bits. The maximum value is 16 seconds and the
> resolution is about 3.7 nanoseconds. Note: this is different than the 32 bit
> time format in NTPv4 (RFC5905)"

-- 
Miroslav Lichvar

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 24 Nov 2021 09:53:32 +0100

> floating-point math is used.

That is up to the implementation. The specification doesn't require
floating-point math, but it is easier to implement if you can use it.

> If so, are those many special fixed-point formats justified (outside of

For internal calculations, you could keep everything in the 64-bit
format. Even in the NTPv5 packet we could do that, if we don't mind
wasting 16 octets per packet. The specification would certainly be
simpler.

I implemented recently some of the NTPv5 ideas in chrony in an
experimental extension field. I noticed there is a minor compatibility
issue with the proposed time32 format and NTPv4. The maximum value is
not really 16, but slightly less than that. That means it cannot reach
the MAXDISP value from RFC5905. To avoid having to work with two
different MAXDISP values, we could specify an infinite value for the
maximum 32-bit value of the type.

-- 
Miroslav Lichvar

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 24 Nov 2021 12:37:00 +0100

>>> Miroslav Lichvar <[email-masked]> schrieb am 24.11.2021 um 09:53 in
Nachricht <YZ39jGBrF+zeiYm3@localhost>:
>> I wonder: Up to NTPv3 floating‑point math was avoided, but starting in
NTPv4
>> floating‑point math is used.
> 
> floating‑point math, but it is easier to implement if you can use it.
> 
>> If so, are those many special fixed‑point formats justified (outside of
>> packets)?
> 
> For internal calculations, you could keep everything in the 64‑bit
> maximum 32‑bit value of the type.

I had always wondered whether MAXDISP wasn't just some arbitrary larger
number; Couldn't it be 1 second as well?
Is there any specific reasoning behind?

> ‑‑

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 24 Nov 2021 13:09:06 +0100

> Is there any specific reasoning behind?

MAXDISP is the threshold when the server is considered unsynchronized
after it stops updating its clock. It should be larger than any
normally expected dispersion to not block synchronization.

With the default PHI of 15 ppm, it takes 16/15e-6 seconds (about
12.3 days) for the dispersion to accumulate 16 seconds. In the worst
case it should cover up to 15 strata updating clocks at a maximum
polling interval.

I think 1 second would be too small.

-- 
Miroslav Lichvar

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 24 Nov 2021 13:34:06 +0100

>>> Miroslav Lichvar <[email-masked]> schrieb am 24.11.2021 um 13:09 in
Nachricht <YZ4rYlSynuH2RWpu@localhost>:
> With the default PHI of 15 ppm, it takes 16/15e‑6 seconds (about

But doesn't that value also say "the reference clock can be off by as much as
MAXDISP"?
Considering that values above MAXDIST (1s) are ignored anyway, I wonder.
Probably the real reason is that 16 needs five left shifts until it is less
than MAXDIST (samples until server is accepted).

It's all a lot of heuristics, possibly still rooting from the time when it was
important to avoid integer overflows in the math.

With MAXDISP being just one second, the first sample from a server will
probably make it a valid one.

Regards,
Ulrich

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 24 Nov 2021 14:51:12 +0100

> But doesn't that value also say "the reference clock can be off by as much as

In the sense that if a stratum 1 server had a dispersion of 1 second,
it wouldn't be accepted by its clients as synchronized?

> probably make it a valid one.

That is a good point. The clock filter does have an assumption about
the MAXDISP/MAXDIST ratio as it uses MAXDISP for dummy sumples when
empty. I think it would be better if it simply counted the samples
instead of processing dummy values.

-- 
Miroslav Lichvar

---

From: Dan Drown <[email-masked]>
Date: Wed, 24 Nov 2021 22:17:52 -0600

> floating-point math, but it is easier to implement if you can use it.

Floating point hardware is pretty common in embedded devices with an  
internet connection.  For example esp8266 and esp32 have single  
precision FPU hardware.  Cortex-M4/M7s have optional FPU hardware as  
well.

> wasting 16 octets per packet. The specification would certainly be

I don't see the value in 64 bits of root delay/dispersion.  Maybe the  
timescale offset field would benefit from more precision, but at the  
moment I prefer to keep it at 16 bits.

> maximum 32-bit value of the type.

Defining 0xFFFFFFFF (15.999999996) to mean MAXDISP makes sense to me.

---

From: Dan Drown <[email-masked]>
Date: Wed, 24 Nov 2021 22:38:10 -0600

This email will be on the "4. Message Format" section.

Right now the flags/era/timescale fields come before root dispersion  
and root distance. This puts them at a different offset in the packet  
compared to their location in NTPv4. Was there a reason not to keep  
these at the same offset? It's not an implementation problem, just a  
cosmetic difference.

For timescale offset, there is an overlap with the unknown value  
(0x8000/-1.0 seconds) during a negative leap second smear. Is this  
worth mentioning in the rfc?

Transmit Timestamp currently has: "In responses it is the time when a  
response to the client was transmitted. The specific response depends  
on the selected mode (basic or interleaved). The timestamp corresponds  
to the beginning of the transmission"

I'm suggesting this rewording: "In responses it is the beginning of  
the transmission of a response to the client. Which response it refers  
to depends on the selected mode (basic or interleaved). See  
Measurement Modes section below for detail."

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 24 Nov 2021 21:33:07 -0800

This is what happens when the choice is made to ignore the design of NTP 
packet version "evolution".

The original design is that the packet format to the length of a version 
N packet is ALWAYS the same.  This is why ntpd will respond to a 
"higher" version number packet with the requested newer version but with 
a packet length of the responding ntpd's version.

If a v4 NTP packet is 48 bytes, the design and expectation has been that 
if v5 wants any different data then the packet is made longer and the 
new data lives AFTER the v4 data.

Assume, for purposes of this thread, that an NTPv5 packet format is now 
64 bytes.

Should a v5 client send a request to a v4 server, the v5 client sends a 
64-byte v5 request, and assuming the v4 server responds, it will do so 
with a 48-byte v5 response that is effectively a v4 response packet. 
The client will see this, note that the packet is short, and be in a 
position to understand that the target server is a v4 server, not a v5 
server.

Should a v4 client send a request to a v5 server, the v5 server should 
respond with a 48-byte v4 response.

H

On 11/24/2021 8:38 PM, Dan Drown wrote:
> Right now the flags/era/timescale fields come before root dispersion and 
> root distance. This puts them at a different offset in the packet 
> (0x8000/-1.0 seconds) during a negative leap second smear. Is this worth 
> mentioning in the rfc?
> 
> response to the client was transmitted. The specific response depends on 
> the selected mode (basic or interleaved). The timestamp corresponds to 
> the beginning of the transmission"
> 
> I'm suggesting this rewording: "In responses it is the beginning of the 
> transmission of a response to the client. Which response it refers to 
> depends on the selected mode (basic or interleaved). See Measurement 
> Modes section below for detail."
> 
> 

-- 
Harlan Stenn <[email-masked]>
http://networktimefoundation.org - be a member!

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 25 Nov 2021 10:36:00 +0100

On Wed, Nov 24, 2021 at 10:38:10PM -0600, Dan Drown wrote:
> root distance. This puts them at a different offset in the packet compared
> to their location in NTPv4. Was there a reason not to keep these at the same
> offset? It's not an implementation problem, just a cosmetic difference.

The reason was to not break the ordering of the fields by their
length. As the format of root delay/dispersion is different, I thought
moving them might have a small advantage that a combined NTPv4+NTPv5
implementation would be less likely to incorrectly interpret their
values as there would have to be two separate paths. But I wouldn't mind
the order to be changed if there is a good reason.

> (0x8000/-1.0 seconds) during a negative leap second smear. Is this worth

Hm, good point. I think it depends on when the leap smear starts.
There is no +1.0 value. If we define 0x8000 in time16 to mean unknown,
at least it would be symmetric. +1.0/-1.0 would be rounded to
0x7fff/0x8001.

> response to the client was transmitted. The specific response depends on the
> selected mode (basic or interleaved). The timestamp corresponds to the
> beginning of the transmission"
> 
> depends on the selected mode (basic or interleaved). See Measurement Modes
> section below for detail."

Sounds good to me.

Thanks,

-- 
Miroslav Lichvar

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 25 Nov 2021 11:23:52 +0100

On Wed, Nov 24, 2021 at 09:33:07PM -0800, Harlan Stenn wrote:
> The original design is that the packet format to the length of a version N
> packet is ALWAYS the same.  This is why ntpd will respond to a "higher"
> version number packet with the requested newer version but with a packet
> length of the responding ntpd's version.

If NTP was intended to evolve in such a way, why wasn't the expected
server behavior specified in the NTP RFCs? RFC 5905 makes it quite clear
that packets using higher version number should be ignored.

>From 9.2:
  Format checks require correct field length and alignment, acceptable
  version number (1-4), and correct extension field syntax, if present.

>From A.5.1:
	if (r->version > VERSION /* or format error */)
		return;                 /* format error */

> If a v4 NTP packet is 48 bytes, the design and expectation has been that if
> v5 wants any different data then the packet is made longer and the new data
> lives AFTER the v4 data.

That might make sense if NTP packets had a constant length. But with
extension fields and MACs I don't see how could it work. The NTPv4
server wouldn't know how long is the NTPv5 header and where an
extension field starts.

> Assume, for purposes of this thread, that an NTPv5 packet format is now 64
> bytes.
> 
> 64-byte v5 request, and assuming the v4 server responds, it will do so with
> a 48-byte v5 response that is effectively a v4 response packet. The client
> will see this, note that the packet is short, and be in a position to
> understand that the target server is a v4 server, not a v5 server.

The vast majority of existing servers doesn't respond with a 48-octet
packet to a 64-octet v5 request.

> respond with a 48-byte v4 response.

If the server supported NTPv4, that would be expected. If it supported
NTPv5 only, it shouldn't respond.

-- 
Miroslav Lichvar

---

From: Steven Sommars <[email-masked]>
Date: Thu, 25 Nov 2021 11:24:25 -0600

Root dispersion should have the same resolution as the time stamp.
With today's 16-bit field the smallest root dispersions are 0,   15.3,
30.5 ... microseconds.
What should an NTP server having 1 microsecond uncertainty return?

---

From: Dan Drown <[email-masked]>
Date: Thu, 25 Nov 2021 20:43:00 -0600

> What should an NTP server having 1 microsecond uncertainty return?

I agree that NTPv4 root dispersion does not have enough resolution.

With this NTPv5 spec, root dispersion is 32 bits with 28 bits assigned  
to the subsection fraction. This gives 1/(2^28) or 3.7 nanosecond  
resolution. I don't see the point in taking root dispersion from 32  
bits to 64 bits for a smaller resolution than that.

---

From: Dan Drown <[email-masked]>
Date: Thu, 25 Nov 2021 21:16:13 -0600

Quoting Harlan Stenn <[email-masked]>:
> This is what happens when the choice is made to ignore the design of  
> NTP packet version "evolution".
>
> The original design is that the packet format to the length of a  
> version N packet is ALWAYS the same.  This is why ntpd will respond  
> to a "higher" version number packet with the requested newer version  
> but with a packet length of the responding ntpd's version.
>
> If a v4 NTP packet is 48 bytes, the design and expectation has been  
> that if v5 wants any different data then the packet is made longer  
> and the new data lives AFTER the v4 data.
>
> Assume, for purposes of this thread, that an NTPv5 packet format is  
> now 64 bytes.
>
> Should a v5 client send a request to a v4 server, the v5 client  
> sends a 64-byte v5 request, and assuming the v4 server responds, it  
> will do so with a 48-byte v5 response that is effectively a v4  
> response packet. The client will see this, note that the packet is  
> short, and be in a position to understand that the target server is  
> a v4 server, not a v5 server.
>
> Should a v4 client send a request to a v5 server, the v5 server  
> should respond with a 48-byte v4 response.

The v4 client with a v3/v4/v5 server seems to be a pretty easy case.  
And I agree that the server should respond with the same version that  
it received.

The v4/v5 client with an unknown version server does have challenges,  
and is covered in section 9, "NTPv5 Negotiation in NTPv4"

Sending a v5 packet to a v4-only server is likely going to be dropped.  
It might be nice to document how the most popular NTP server software  
handles this somewhere.

---

From: Dan Drown <[email-masked]>
Date: Thu, 25 Nov 2021 21:18:38 -0600

> at least it would be symmetric. +1.0/-1.0 would be rounded to

Ok, that rounding makes sense.

Oh, and another thing I've seen people mess up is which sign means  
which clock is faster.  The time formats specify:

* "it is the TAI-UTC offset"
* "it is the UT1-UTC offset"
* "it is the current offset between the leap smeared time and UTC"

Maybe these need to be reworded to be more explicit and follow the  
same phrasing? Something like

* "it is TAI minus UTC"
* "it is UT1 minus UTC"
* "it is the leap smeared time minus UTC"

---

From: Dan Drown <[email-masked]>
Date: Thu, 25 Nov 2021 21:47:48 -0600

This email will be on the "5. Extension fields" section.

for the reference id extension (5.3):
* I'm assuming if the client wants to receive it in the response, it  
has to send an empty one in the request.
* I'm not an expert at bloom filters, but with 16 stratums, I think we  
want one at least 160 bits and to set 7 bits for each clock (1% false  
positive rate).
* Because we don't actually need the random ID values themselves,  
instead of generating a random ID and hashing it, we can just generate  
7 random numbers between 0..159 and set that bit.
* Detecting a loop would involve checking to see if our 7 bits are set  
in the data from our upstream source

For the offset correction extension (5.5), are there any  
implementations of this? I've heard that it is harder to parse  
optional headers in asics, should this extension be required to be at  
a fixed location? something like: "if this extension is present, it  
MUST be the first extension field after the NTPv5 header"

---

From: "Ulrich Windl" <[email-masked]>
Date: Fri, 26 Nov 2021 08:04:11 +0100

Sorry, forgot to replay to all:
---

Hi!

That might be an issue, even though our GPS server has a root dispersion in
the 200µs range (DCF-77 in the 2000µs range).
Related question: What is the smalles amount added to the root dispersion? If
it's more than the resolution, the resolution seems unimportant.
PHI being 15PPM means 15µs added per second.

So for the GPS sampling might look like this:
ntpq> rl
rootdisp=0.140
ntpq> rl
rootdisp=0.140
ntpq> rl
rootdisp=0.155
ntpq> rl
rootdisp=0.170
ntpq> rl
rootdisp=0.230
ntpq> rl
rootdisp=0.245
ntpq> rl
rootdisp=0.140

Regards,
Ulrich

>>> Steven Sommars <[email-masked]> schrieb am 25.11.2021 um 18:24
in
Nachricht

---

From: "Ulrich Windl" <[email-masked]>
Date: Fri, 26 Nov 2021 09:29:28 +0100

>>> Dan Drown <[email-masked]> schrieb am 26.11.2021 um 03:43 in Nachricht
<[email-masked]>:
>> With today's 16‑bit field the smallest root dispersions are 0,   15.3,
>> 30.5 ... microseconds.

I wonder: Should a "virtual infinity" value (probably for every special
numeric type) be defined?
I mean: we are limiting the range for root dispersion rather arbitrarily
(through the number of non-fractional bits), while in theory (through math) the
value could be exceeded.
I think some of the algorithms might be more clear when using a symbolic
infinity value, even if that's far from infinity.

MAXDISP (16) somehow is that, as the value cannot actually be reached.
But it seems to have some double-role: The infinity value and some
protocol-related value.

Example:
if r->rootdelay / 2 + r->rootdisp >= MAXDISP 

Regards,
Ulrich

---

From: Dan Drown <[email-masked]>
Date: Fri, 26 Nov 2021 21:48:20 -0600

Quoting myself:
> * I'm not an expert at bloom filters, but with 16 stratums, I think  
> we want one at least 160 bits and to set 7 bits for each clock (1%  
> false positive rate).
> instead of generating a random ID and hashing it, we can just  
> generate 7 random numbers between 0..159 and set that bit.
> * Detecting a loop would involve checking to see if our 7 bits are  
> set in the data from our upstream source

After a day of thinking about my proposal above, I have the following  
concerns:

* The added packet size, 20+4 bytes is very large for NTP

* I need to consider compatibility between NTPv4 refid and v5  
extension. Specifically, what translations happen when switching  
between versions

* The 1% false positive rate would be a problem when you have a set of  
servers each with 10,000+ clients

* Packet logging for monitoring and management would be challenging

* Operators would probably want some way to make this more  
deterministic, maybe a configured local ref id

I'm doubting if a bloom filter is the right choice for NTP loop detection.

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 29 Nov 2021 10:58:02 +0100

> * The added packet size, 20+4 bytes is very large for NTP

What would you consider the maximum acceptable size? Please note that
only clients serving time to other servers would need to use it.
Clients that don't serve time, which I think is the most common case,
don't need to care about loops.

As you pointed out, it should be even much larger if we want a lower
false positive rate. But it doesn't have to be transmitted all at
once. The client could get it in multiple parts over multiple requests
(e.g. 4 or 8). This would add a delay to the loop detection.

> * I need to consider compatibility between NTPv4 refid and v5 extension.
> Specifically, what translations happen when switching between versions

Most of the information would be lost if an NTPv4 server was in the
chain. This would be common in the beginning. For NTPv4->NTPv5
compatibility, we could specify a mapping of the 32-bit refids to the
bloom filter.

> servers each with 10,000+ clients

Right. It needs to be much lower.

> * Packet logging for monitoring and management would be challenging

You mean manually checking whether a server is in the filter?

> * Operators would probably want some way to make this more deterministic,
> maybe a configured local ref id

It could be saved to disk to not change across service restarts.

> I'm doubting if a bloom filter is the right choice for NTP loop detection.

Are there any other reasonable options?

Maybe a combined approach using 32-bit or 64-bit IDs for detecting
loops between two servers and a slowly updated bloom filter to detect
loops over a larger number of servers?

-- 
Miroslav Lichvar

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 29 Nov 2021 11:05:29 +0100

On Thu, Nov 25, 2021 at 09:18:38PM -0600, Dan Drown wrote:
> Oh, and another thing I've seen people mess up is which sign means which
> clock is faster.  The time formats specify:
> 
> Maybe these need to be reworded to be more explicit and follow the same
> phrasing? Something like
> 
> * "it is TAI minus UTC"
> * "it is UT1 minus UTC"

I'd like to keep the word "offset" there. Maybe the explanation can be
added in parenthesis like this?

      *  In the UTC (0) and TAI (1) timescales it is the TAI-UTC offset
         (TAI minus UTC) as a signed integer, or 0x8000 if unknown.

      *  In the UT1 timescale (2) it is the UT1-UTC offset (UT1 minus
         UTC) using the time16 type (0x8000 if unknown).

      *  In the leap-smeared UTC, it is the current offset between the
         leap smeared time and UTC (former minus latter) using the
         time16 type (0x8000 if unknown).

-- 
Miroslav Lichvar

---

From: Dan Drown <[email-masked]>
Date: Mon, 29 Nov 2021 21:30:09 -0600

>          time16 type (0x8000 if unknown).

I would be fine with that.  Does leap-smeared UTC need a (3) to  
identify the scale identifier?

---

From: Dan Drown <[email-masked]>
Date: Mon, 29 Nov 2021 23:04:01 -0600

> don't need to care about loops.

Taking an embedded NTP server with a 100M ethernet link for example,  
my concern is that adding 24 bytes reduces maximum packets per second  
from 109.6kpps to 90.6kpps (17%). The number of NTP servers on a 100M  
link doing near maximum packet rate is probably a short list, but it  
is a tradeoff to consider. I don't really have a set acceptable size  
and I am open to increasing the size by some amount for this  
functionality.

I agree that only NTP servers would need to care about loop  
prevention. So that would help the average packet size needed given a  
mix of clients and downstream servers.

> (e.g. 4 or 8). This would add a delay to the loop detection.

I'd be worried about keeping that data in sync between systems. Having  
a "generation" identifier would be one option to handle that. If we  
went that route, why not just have static IDs for each stratum instead  
of a bloom filter?

Something like:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             Type              |             Length=12         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| generation    | stratum       | bits47..32                    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| bits31..0                                                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

request would set everything to 0 and then give the stratum it wanted (1..16).

response would set a generation ID that changes every time the  
upstream clock changes. The generation ID changing would signal to the  
downstreams that they need to refetch all the refIDs. The downstreams  
could cache the data as long as the generation ID stayed the same.  
Each stratum would have a random 48 bit number identifying it.

For generation ID size, I picked 8 bits arbitrarily. That risks  
wrapping when the requestor is polling at 1024 seconds and the  
responder is changing references every 4 seconds. Stratum only needs  
to be 4 bits (0..15), so generation could be 12 bits instead, trading  
longer wrap times for slightly more implementation complexity.

If I properly understand the birthday problem in statistics,  
1-e^(-n*(n-1)/(2^(48+1)))>0.001 is the estimate for how many servers  
you need before you have a 0.1% chance of two of those servers picking  
the same 48 bit number (assuming fully random id selection). Which is  
750,488 servers, so that seems unique enough to me.

A 12 bytes extension field is a 10% reduction in maximum packets per  
second vs 24 bytes' 17%.

There would have to be some sort of reserved ID for the case where a  
responder didn't know the data of a given stratum yet (or NTPv4 was  
involved in the path). Maybe 0xffffffffffff.

I'm not sure if I've fully explained this, let me know if there are  
any questions.

> compatibility, we could specify a mapping of the 32-bit refids to the

It could be mapped directly onto bits31..0

> You mean manually checking whether a server is in the filter?

For diagnosing "why did things break" after the fact, it would be nice  
to have a log of this data somewhere

> It could be saved to disk to not change across service restarts.

I guess I'd rather see it in a config file than a generated state  
file. If no config for the ID is present, it would be automatically  
generated each time.