Subject: Re: [Ntp] Leap second draft

From: Tony Finch <[email-masked]>
Date: Mon, 1 Apr 2019 12:54:19 +0100

Warner Losh <[email-masked]> wrote:
>
> And it's less obscure than the crazy notion that leap seconds could happen
> any time other than June and December as a practical matter.

RFC 5905 allows for monthly leap seconds. There's a chance we will need
more than two leap seconds in a year before the end of this century.

Some of the problems with leap second signalling are because NTP clients
only get a two bit warning signal, which isn't enough especially if the
server is setting the bits using assumptions about how leap seconds are
currently administered (at most every 6 months), and the client is
interpreting it according to RFC 5905 and TF.460 (at most every month) and
the server incorrectly provides more than one month warning.

I think if the server explicitly tells the client when the leap second(s)
occur, then there's less scope for problems caused by unwarranted
chumminess with the IERS's bulletin C schedule.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Lyme Regis to Lands End including the Isles of Scilly: East becoming variable,
3 or 4, then becoming northwest 5 or 6 later. Slight or moderate, becoming
smooth to slight for a time east of the Lizard. Rain for a time then showers.
Moderate or good.

---

From: Warner Losh <[email-masked]>
Date: Mon, 1 Apr 2019 06:00:32 -0600

> chumminess with the IERS's bulletin C schedule.

That would help. GPS give such a signal and it's quite helpful to know past
and future leaps to act as a sanity check on the data. Though in GPS' case,
it is needed because of the 1024 week wrapping issue.

We no longer live in an era where an extra 32bit word would kill us, yet
that would be extremely helpful if widely implemented.

Warner

Tony.

---

From: Tony Finch <[email-masked]>
Date: Mon, 1 Apr 2019 13:23:07 +0100

Warner Losh <[email-masked]> wrote:
>
> A reference to the POSIX standard would be likely be helpful.
> http://pubs.opengroup.org/onlinepubs/9699919799/ has the definition, and
> the reference on all the pages is "The Open Group Base Specifications Issue
> 7, 2018 edition IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)"

A slightly more direct link is
http://pubs.opengroup.org/onlinepubs/9699919799.2018edition/basedefs/V1_chap04.html#tag_04_16

The frames on the POSIX web site are annoying :-(

> POSIX does not assign a value to the leap second.

I think it does, because I can't see any caveat about plugging
tm_sec == 60 into the expression for seconds since the epoch.

There's also explicit support for leap seconds in struct tm
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/time.h.html
although it is not explicitly stated that the expression for seconds since
the epoch uses the same definitions as the fields in struct tm...

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Channel Islands: Northeast to east 3 to 4, locally 5 in the north until
mid-afternoon, becoming variable 1 to 3 this evening, then west to northwest 3
to 4 after midnight, increasing 4 to 5 around dawn, locally 6 at times.
Slight, locally moderate in the north early afternoon, generally increasing
moderate during Tuesday morning. Occasional haze risk of fog patches near
french coasts overnight outbreaks of rain spreading from the west by dawn,
clearing by midday. Moderate to good, locally poor at times, perhaps locally
very poor overnight.

---

From: Tony Finch <[email-masked]>
Date: Mon, 1 Apr 2019 13:36:02 +0100

Kurt Roeckx <[email-masked]> wrote:
>
> I'm not really sure what you mean here exactly. But I think that
> for an insert, either the receive or transmit happening during
> the leap second can cause problems if this extension is not used.

You need to allow a second either side as well, to allow for different
ways of implementing the ambiguous timestamps.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Cape Wrath to Rattray Head including Orkney: South or southwest 5 to 7,
becoming variable 3 or 4 later. Slight in Moray Firth, otherwise moderate or
rough. Occasional rain or showers. Moderate or good.

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 1 Apr 2019 14:46:06 +0200

>From what I've seen in the past there are only very few leap second
problems due to NTP itself, neither due to the protocol, nor due to the
implementation, ntpd.

If a GPS receiver emits a wrong leap second announcement, e.g. at the
end of September when in fact the leap second occurs at the end of
December, then it wouldn't even help if the full (wrong) date was passed
to clients instead of just an announcement flag.

If an OS kernel just steps the time back by 1s to insert a leap second
then this will happen, too, if a full date is specified for the leap
second, with the same effects on applications that get confused due to
the step.

If an OS kernel has a bug and the kernel hogs up the CPU (happened with
some older Linux kernels) when the kernel itself tries to insert a leap
second then there's no big difference whether the kernel has received
the leap second announcement from an NTP implementation evaluating just
a flag, or a full date, or from some PTP slave software, or from any
other source.

If an OS doesn't support leap seconds at all (Windows before *very*
current versions) then you can only try to implement a workaround.
Similar with smearing.

Please note I absolutely find it useful to be able to specify the full
date for which a leap second has been scheduled, but keep in mind that
doing so doesn't prevent from the kind of problems mentioned above,
which are not even related to NTP.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 1 Apr 2019 14:52:07 +0200

> the epoch uses the same definitions as the fields in struct tm...

The problem is not that struct tm can't store a specific point in time
that is a leap second. The problem is that if you take a system time
stamp in time_t format and call gmtime() or localtime() you never get a
struct tm back where the value of the second is 60.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Warner Losh <[email-masked]>
Date: Mon, 1 Apr 2019 07:00:04 -0600

> tm_sec == 60 into the expression for seconds since the epoch.

I'd say it doesn't, because it doesn't say what to do with leap seconds.

It also says that mapping of time of day to epoch second is undefined.

> the epoch uses the same definitions as the fields in struct tm...

It doesn't say explicitly. One can make a reasonable argument that it's a
correct mapping. But other correct mappings are possible because time_t
does not have an explicit encoding for leap seconds.

Which is my point. POSIX does not define it explicitly, so strong
statements about what it does and why are not appropriate.

Warner

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 1 Apr 2019 15:02:44 +0200

On Fri, Mar 29, 2019 at 08:57:42PM +0100, Daniel Franke wrote:
> 1.2.  The Leap Indicator field
> 
>    NTP packets include a Leap Indicator field which warns receivers of
>    an upcoming insertion or deletion of a leap second.  [RFC5905]
>    specifies that this field warns "of an impending leap second to be
>    inserted or deleted in the last minute of the current month".

RFC 5905 actually says both "current month" and "current day". I think
the leap field was supposed to be compatible with NTPv3 (it's not
mentioned in the RFC as an incompatible change - unlike the refid
field) and both instances in the document should say "day". I've
submitted an errata for this couple years ago.

>    The Era Number subfield gives the era number of the Receive
>    Timestamp.  The Reference Timestamp always precedes the Receive
>    Timestamp, and the Transmit Timestamp always succeeds the Receive
>    Timestamp.

I don't think this is always the case. It's not required by RFC 5905.

Some reasons for a TX timestamp not being after RX:
- low-precision clock (randomization of low-order bits may, but
  doesn't have to enforce the order of generated timestamps)
- clock stepped between RX and TX (e.g. in symmetric mode)
- very simple server using a single timestamp for both TX and RX
- interleaved mode (in this case it would be unusual to see RX < TX)

>       F: When set, indicates that the Reference Timestamp was captured
>       during a leap second.
> 
>       R: When set, indicates that the Receive Timestamp was captured
>       during a leap second.
> 
>       X: When set, indicates that the Transmit Timestamp was captured
>       during a leap second

This sounds nice, but I'm curious if it can actually be implemented on
current systems. One issue is that the system clock may not be stepped
exactly at 00:00:00.000000000. It may happen few microseconds or even
milliseconds later.

Timestamps captured by ntp_gettime(), which gives also the clock
status code, could be better at least in theory, but it may be much
slower than clock_gettime() and there is no way to get this
information for timestamps captured by the kernel (e.g. with the
SO_TIMESTAMP socket option).

To me it would make more sense to specify that clients should ignore
any measurements that have a timestamp that is too close to the leap
second and servers should respond with "unsynchronized" leap bits
around leap second. Some implementations already do this.

-- 
Miroslav Lichvar

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 1 Apr 2019 15:02:36 +0200

> ways of implementing the ambiguous timestamps.

This why currently ntpd sends leap bits "unsynchronized" (0x3) back to
clients for a short interval across the leap second.

I very much like this idea since the chance that a client even sees this
is very low, and if it should really happen then it doesn't hurt the
client, even if the client is not so sophisticated to handle the new EFs.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Kurt Roeckx <[email-masked]>
Date: Mon, 1 Apr 2019 18:18:16 +0200

One question I have that I can't seem to find the answer for is
which second do you announce during the leap second in NTP.

There was a leap second at 3692217600. Do you send:
3692217599
3692217599 (leap second)
3692217600

Or:

3692217599
3692217600 (leap second)
3692217600

I can see arguments for both ways, and if it's not specified
somewhere yet, I think we need to specify it.

Kurt

---

From: Watson Ladd <[email-masked]>
Date: Mon, 1 Apr 2019 18:14:45 -0700

On Sat, Mar 30, 2019 at 8:02 PM Warner Losh <[email-masked]> wrote:
>
>
>
> On Sat, Mar 30, 2019, 6:36 PM Daniel Franke <[email-masked]> wrote:
>>
>> On Sat, Mar 30, 2019 at 7:53 PM Kurt Roeckx <[email-masked]> wrote:
>> > Anyway, I think we should avoid defining what things like NTP
>> > time, UTC and TAI is. They should all have external definitions
>> > we can just point to.
>>
>> Where the definitions are normative, sure. But as Steve's infodumps in
>> this thread rather vividly illustrate, readers need a comprehensible
>> summary; we can't just incorporate the whole body of relevant
>> standards by reference and leave it at that. The statements in the
>> background section aren't normative, and should be accurate but don't
>> have to be comprehensive. And I won't be worried if, for example, the
>> quoted definition of an SI second gets superseded a month after the
>> RFC is published, because the quote isn't there to be authoritative;
>> it's there to illustrate what such a definition can look like.
>
>
> The current definition doesn't help. A reference to the BIPM with the latest definition doc at the time of publication would suffice. The hard part is getting the right pointers.
>
> It doesn't matter what a SI second is. NTP counts them and exchanges that count. They could be anything. People have done GPS times (also SI seconds) and UT1 times (solar variable length seconds). They've even done Julian Time NTP.. Now obviously these different time scales won't interoperate, but they are fine so long as everybody exchanging time agrees on the rules for the time scale. That brings us to UTC. It's the odd man out. It ticks at a fixed rate, and occasionally has seconds inserted into it. UTC is related to TAI by a stepwise function that always increases or decreases by 1. This increase or decrease historically has always been December or June, but technically could be at the end of any month. UTC is synchronized to UT1 every so often based on a set of rules. The current rule is that | DUT1 | must remain < .9s so you move it when you think it might exceed .9 before the next chance to move it. Other rules are possible, but just like leaps at the end of any other month than December or June not likely.

I've tried citing the BIPM definitions, but they are specified in
years, months, days, hours, minutes, seconds, and fraction of second,
while NTP uses seconds and fraction of seconds. Then there is the
whole document access issue.

>
> So, it makes sense to just hit these points, and give pointers to the underlying details, which don't matter.
>
> Warner
>
>> _______________________________________________
>> ntp mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/ntp
>
> _______________________________________________
> ntp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ntp

--
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: Warner Losh <[email-masked]>
Date: Mon, 1 Apr 2019 19:34:52 -0600

On Mon, Apr 1, 2019 at 7:14 PM Watson Ladd <[email-masked]> wrote:

> > The current definition doesn't help. A reference to the BIPM with the
> latest definition doc at the time of publication would suffice. The hard
> part is getting the right pointers.
> >
> > It doesn't matter what a SI second is. NTP counts them and exchanges
> that count. They could be anything. People have done GPS times (also SI
> seconds) and UT1 times (solar variable length seconds). They've even done
> Julian Time NTP.. Now obviously these different time scales won't
> interoperate, but they are fine so long as everybody exchanging time agrees
> on the rules for the time scale. That brings us to UTC. It's the odd man
> out. It ticks at a fixed rate, and occasionally has seconds inserted into
> it. UTC is related to TAI by a stepwise function that always increases or
> decreases by 1. This increase or decrease historically has always been
> December or June, but technically could be at the end of any month. UTC is
> synchronized to UT1 every so often based on a set of rules. The current
> rule is that | DUT1 | must remain < .9s so you move it when you think it
> might exceed .9 before the next chance to move it. Other rules are
> possible, but just like leaps at the end of any other month than December
> or June not likely.
>

The definition of an SI seconds has nothing to do with years, etc.

Warner

---

From: Watson Ladd <[email-masked]>
Date: Mon, 1 Apr 2019 19:06:19 -0700

> The definition of an SI seconds has nothing to do with years, etc.

Correct, but UTC timestamps are specified with a representation based
on hours, minutes, and seconds.

---

From: Warner Losh <[email-masked]>
Date: Mon, 1 Apr 2019 21:51:23 -0600

> on hours, minutes, and seconds.

There are two standards. What defines an si second. And also how UTC labels
the seconds. TF460 is for the latter. I don't have a good reference for the
former, but we don't have to repeat the current SI definition because that
is not relevant to NTP. It just exchanges counts of seconds, whatever they
are, and the number of hyper fine transitions is never an issue for the
protocol. It is an issue if you are building a cesium standard and almost
never at any other time.

The mapping of a number to the UTC labeling of the second is defined in the
NTP RFC. It is relevant because the mapping is unique to NTP. Also relevant
is how UTC labels the seconds, since it is totally weird to the
uninitiated.

Also, what to do around the leap second, where UTC is totally weird and
irregular compared to a traditional time scales is highly relevant. Those
details matter and are of critical importance to implementing NTP in an
interoperable way. Even the basics are worth repeating. All the more
because supposedly obvious details not worth mentioning in prior docs are
done in a diversity of ways.

Although I've been nitpicky, it comes from a decade of doing high precision
time and frequency work and having to interoperate with all the different
choices... I'm so glad to see this get spelled out. Thank you so much for
doing this...

Warner

---

From: Warner Losh <[email-masked]>
Date: Mon, 1 Apr 2019 22:00:22 -0600

On Mon, Apr 1, 2019, 8:27 PM Seaman, Robert Lewis - (rseaman) <
[email-masked]> wrote:

> On 4/1/19, 6:35 PM, Warner Losh wrote:
>
>
>
> If only this were true. The unnecessary confusion of the SI unit of
> interval time (the “second”) with the original definition of the second as
> a fraction of a solar day is what has created the lengthy debate about UTC
> and leap seconds. This then extends to the definition of the solar synodic
> day (
> http://hanksville.org/futureofutc/preprints/files/28_AAS_13-515_Seaman.pdf).
> By choosing the SI unit to masquerade as Universal Time, that is, as a
> fraction of the mean solar day, the precision timing community has
> generated diverse dependencies between what should be independent concepts,
> including between the SI unit and the physical concept of year. That many
> of these dependencies are esoteric does not make them thereby less real.
>
>
>
> By all means the SI second could have been “set by fixing the numerical
> value of the ground state hyperfine splitting frequency of the cesium 133
> atom to be equal to *[say]* exactly *10 000 000 000* when it is expressed
> in the SI unit s–1, which is equal to Hz” (or the corresponding original
> phrasing), but rather the frequency was selected to be *9 192 631 770*
> Hz. This wasn’t a coincidence, but an attempt to obscure the reality that
> atomic time and solar time are two different things. Civil time, and thus
> NTP which conveys it, has dependencies on two different time scales.
>

Historically  the number was chosen to replace the old definition. It is
relevant there. But it's not relevant for the purpose of implementing NTP.

NTP gets it's time from a reference clock, which may care about the details
of the definition, or from a peers, which doesn't. Either way, NTP doesn't
care. It can easily exchange times with a different notion of a second from
SI, so long as all the peers agree to follow the same standard. It could be
GPS which uses SI seconds without leaps. It could have an offset like
exchanging Julian dates. It could even be UT1. This document is not
relevant to any of those. I give them by way of example for why the details
of UTC should be repeated (since they are weird and unique to UTC) and why
the SI second definition need not be (since it will change to be more
precise, but such changes don't matter to NTP since those details don't
affect how you'd write an NTP client or server.)

Warner

> Rob Seaman
> University of Arizona

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 2 Apr 2019 09:40:58 +0200

> There was a leap second at 3692217600. Do you send:

NTP implementations I've seen just pick up the current system time, so
which timestamps are returned during a leap second depends on the way
the particular operating system handles the leap second, and e.g. ntpd
running on one type of operating system can behave differently than ntpd
running on a different type of operating system.

So the discussion which way is the "better" one had to be done with the
maintainers of the OS kernel clock implementation. ;-)

Letting an NTP daemon decide what kind of timestamp to return would
require that it

1.) uses the adjtimex()/ntp_adjtime() API calls to retrieve a timestamp
including some status information, and then modifies the time stamp in a
preferred way

2.) maintains its own clock beside the system clock so it could handle
the leap second in a different way than the kernel clock

Solution 1.) is normally not used because adjtimex()/ntp_adjtime() take
longer to execute than a simple clock_gettime()

Solution 2.) can cause additional problems if the daemon's internal leap
second handling differs from the leap second handling observed by
applications running on the particular system.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Kurt Roeckx <[email-masked]>
Date: Tue, 2 Apr 2019 19:04:11 +0200

> applications running on the particular system.

One option would be the allow both the cases, but I still think
that should get documented.

But to be compliant with the current draft, you would need to know
that a timestamp is within the leap second.

It can be that with the current kernel APIs, this is very hard to
properly do, but I don't see that as a problem of the draft.

Assuming it's not possible to properly implement the bits
indicating that a timestamp is within the leap second, maybe the
draft should mention that in that case you should either not reply
around the leap second or return that you're not synchronized.

Kurt

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 3 Apr 2019 10:41:34 +0200

> properly do, but I don't see that as a problem of the draft.

I agree, but on the other hand, what's the advantage of a draft if it is
not implemented, e.g. because the performance is degraded?

Many years ago Dave Mills suggested to implement kernel clocks in a way
that the system time doesn't have to be stepped back to insert a leap
second. Instead, it should basically be stopped during that second, but
this would result in always the same time stamp during the leap second,
so the final proposal was to increment the least significant bit of the
time stamp e.g. 1 microsecond or nanosecond whenever any application
retrieves a time stamp. This approach makes sure the system time is
always strictly monotonically increasing.

Thus, an approach like this at the kernel level would avoid many
problems with user space applications across a leap second, and some
time ago I've discussed this with one of the Linux kernel developers.

He told me it's simply too expensive (in terms of execution time) to let
the kernel check whether its time is inside a leap second, or not,
whenever any applications asks for the system time, just because this
may happen during a single second in a couple of years.

So the kernel in fact simply steps the time back at the beginning or end
of a leap second, which often causes problems with applications that
don't expect such step, but anyway it's less expensive for the kernel.

Basically the same problem arises if you call adjtimex()/ntp_adjtime()
whenever you want to get a current system timestamp. If a program like
ntpd did it the number of client requests it can handle per second would
probably be decrease, and on the other hand this wouldn't even improve
the situation of applications running on a client.

In my opinion the current mechanisms provided by the protocol to
announce and handle a leap second are sufficient, even across several
stratum levels.

What was generally missing (unless you used autokey) was a way to
propagate the current TAI/UTC offset over the network, so it's good to
have a replacement which can be used to do this in the absence of
autokey, if required, e.g. using an extension field.

A bigger problem than the protocol itself is to have a way to inject
information on an upcoming leap seconds, TAI offset, etc. reliably at
the top of the timing chain.

With reliably I mean that this must not only work as long as the OS
maintainers provide software update packages that contain a tzdata/leap
second file, but independently from firmware updates. We have discussed
this a couple of days ago.

> around the leap second or return that you're not synchronized.

Yes, the current behavior of ntpd to claim to be *not* synchronized for
a short interval across a leap second (as has been suggested by
Miroslav) doesn't hurt, but reduces potential problems especially with
simple clients, and I agree that *this* behavior should be documented.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Kurt Roeckx <[email-masked]>
Date: Wed, 3 Apr 2019 20:11:36 +0200

> may happen during a single second in a couple of years.

The other problem with a solution like that is that your time is
wrong, going from 0 to -0.5, then jump to +0.5 to go back to 0.
It would a case where you do not want to answer packets either.

> the situation of applications running on a client.

I can understand that you don't want to have such a check in
kernel code, because things that don't care about it are also
affected by it. But I don't see a problem doing such a check in
userspace as a problem. In case it's known that the kernel does a
step, it looks like you should be able known if you're in the
middle of a leap second or not without losing much performance.

An other solution is that the kernel provides a TAI interface, and
user space can then just adjust it with the TAI-UTC offset.

Anyway, I'm not arguing that we need the to be able to indicate
that a timestamp was during a leap second, just saying that there
might be solutions to properly implement the draft as it currently
is.

> announce and handle a leap second are sufficient, even across several

I've seen many broken ntp servers during each leap second, it takes
a long time for them to get fixed across the statum level. I think
one of the reasons is that the NTP RFC defines "00" as no leap
second, instead of not knowning it, and that the "00" wins even when
others servers have properly announced the leap second. I think that
the current draft making a distinction between the 2 cases is a good
thing, but I guess it needs to have guidance of how to actually
deal with it.

Kurt