Subject: [Ntp] Servers stepping back their clocks

From: Danny Mayer <[email-masked]>
Date: Tue, 12 Feb 2019 19:59:08 -0500

I'd just like to reiterate what I said in the virtual interim meeting
during the roughtime discussion that any server that steps their clock
backwards is fundamentally broken and breaks the NTP protocol. It does
not matter where it's broken, if Posix, the O/S, or elsewhere.

Any such server should not be used as a source of NTP packets and should
be avoided at all costs.

Danny

---

From: Watson Ladd <[email-masked]>
Date: Tue, 12 Feb 2019 17:22:21 -0800

What do you believe the NTP UTC timestamp should be of the event drawn
in Figure 3, Annex 3 of
https://www.itu.int/dms_pubrec/itu-r/rec/tf/R-REC-TF.460-6-200202-I!!PDF-E.pdf?
What do you believe the timestamp is one second before that?

The problem is not with server monotonicity (which raises questions
about where ntpd gets the time on Linux: we did have an outage from
using the usual time functions) but rather with monotonicity of the
timescale without reference to any given server: it should be
impossible for a casually later event to have an earlier timestamp.
The common interpretation of UTC, as you admit contained is POSIX,
doesn't support that. The idea that UTC NTP timestamps as defined in
NTP v4 are silently spread over a period given by the filter is a
reading not supported by the text: leap second appears exactly once in
it.

It may be that common implementations do this. There are solutions
using a proper hour, minute, second format to this problem. But I
could just use TAI and avoid all these issues: the relevant verbiage
in TF.460-6* is a continuous scale vs. the adjustments made to UTC.

Sincerely,
Watson Ladd

On Tue, Feb 12, 2019 at 4:59 PM Danny Mayer <[email-masked]> wrote:
> _______________________________________________
> ntp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ntp

---

From: Danny Mayer <[email-masked]>
Date: Tue, 12 Feb 2019 21:53:53 -0500

> What do you believe the timestamp is one second before that?

What you are seeing there is that when adding a leap-second into the
stream of ticks they add a 60 just before UTC midnight, they are not
moving the clock back one second. Standard displays and functions of
time cannot show 23:59:60 though they can do 23:59:59 and 00:00:00
(ignoring fractions of a second). This is where reality (or at least our
programmers' ideas of reality) meets theory. Similarly if they were ever
to declare a negative leap-second, there would be no 23:59:59, it would
just go from 23:59:58 to 00:00:00.

This is where things get tricky. To handle a leap-second insertion there
are various different schemes that have been implemented. Almost all of
the involve slowing down the clock, ie reducing the the frequency of the
next ticks from some point before the leap-second is to be inserted
until some point after when normal frequency is restored. The reference
implementation does this one way, Google implemented leap-smear which
does things differently. You can find Dave Mills's documentation on this
here: https://www.eecis.udel.edu/~mills/leap.html and Google's here:
https://developers.google.com/time/smear and there are a number of other
implementations of leap-second smearing.

Dave's document discusses the problems with POSIX's approach and why he
thinks it's wrong.

Interestingly enough RFC5905 has nothing to say about implementing
leap-seconds only that the protocol packet has an indicator that a
leap-second is imminent at the last minute of the current month. We
might want to consider adding guidance on this issue.

In the meantime I believe that at least one of the REFID drafts wants to
add a way of indicating that leap-smear is in effect.  The other issue
here is what if you don't want leap-smearing so you need to know if the
NTP server providing NTP packets is doing or not doing leap-smearng in
which case you want to drop those packets.

Does this help?

Danny

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Feb 2019 09:20:25 +0100

>>> Danny Mayer <[email-masked]> schrieb am 13.02.2019 um 03:53 in Nachricht

> just go from 23:59:58 to 00:00:00.

(while incrementing the day ;-))

> implementations of leap-second smearing.

I guess the decision against leap smearing was that "a second should still be a second". Otherwise we could continuously smear the second from one leap event to the next.

> might want to consider adding guidance on this issue.

Yes especially as NTPv3 had a clock model to handle that. So unless updates, I guess people will stick to that older model.

> add a way of indicating that leap-smear is in effect.  The other issue

Personally I think the purpose of REFID is different.

> which case you want to drop those packets.

Isn't there agreement that leap-smearing NTP servers actually are not RFC-compiant RFC servers?
So I think leap-smearing servers should show them selves as such (maybe via extension field), but not "pollutiong" the REFID.

Regards,
Ulrich

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 10:18:43 +0100

> Any such server should not be used as a source of NTP packets and should

If this is taken strictly then most Unix operating systems must not be
used as NTP servers since they simply *do* step the time back whenever a
leap second is inserted. :-(

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 10:48:33 +0100

> What do you believe the timestamp is one second before that?

The usual problem here is that all these nice documents tell you about a
second numbered '60', while in fact the system timestamp (at the kernel
level) is only a number of seconds since an epoch, and fractions of that
second.

IMO the main problem is that there's no commonly used API call that
provides some status information together with the system timestamp.

This is similar to changing the derived local time back by 1 hour at the
end of DST. Only if you know the DST status together with the time (e.g.
2:15 AM) you can distinguish whether "2:15" refers to the first (e.g
2:15 CEST) or second (e.g. 2:15 CET) occurrence.

It's similar with a leap second. Only if there was a leap second status
indicator associated with the system time stamp the runtime library
which converts system timestamps to a human readable time and calendar
date would be able to see that the system time was stepped back due to a
leap second, and could return second "60" for the duplicate second.

> in TF.460-6* is a continuous scale vs. the adjustments made to UTC.

True, but if your system runs on TAI which needs to be converted to UTC
then you have to maintain the TAI/UTC offset. Of course this can be done
by some kind of leap second file, but the problem is to keep this
information continuously updated.

The PTP protocol has been designed to broadcast that offset, but for NTP
this was only possible with an extension to the autokey protocol, which
is now considered obsolete.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 11:15:13 +0100

Danny Mayer wrote:
> (ignoring fractions of a second).

I think they could if they knew the second occurrence of the same second
number is a leap second. See my other email. The firmware used inside
Meinberg devices supports a leap second indicator flag, and thus can
distinguish second "60" from a duplicate "59".

> This is where reality (or at least our

This case can easily be handled by a conversion routine.

> until some point after when normal frequency is restored.

Hm, almost all? ntpd for Windows does this because it has explicitly
implemented in ntpd, and it's ntpd who slews the system time quickly
because the Windows system wouldn't do it at all.

On most Unix-like systems the OS handles a positive leap second by just
stepping the system time, and any application (including ntpd) just
observes that the time is stepped back.

> The reference
> implementation does this one way,

Please note the kind of leap smearing implemented in ntpd does *not*
smear its own internal system time. It just smears the time sent to its
clients.

> Google implemented leap-smear which

The POSIX approach is obviously wrong, or at least imperfect, but there
is not much you can do against it in an application.

> add a way of indicating that leap-smear is in effect.

In fact, this has been implemented in the current versions of ntpd. So a
client has at least a chance to determine that leap smearing is in
progress. I don't think, however, that this has been specified in any RFC.

> The other issue

If a client of a leapsecond-smearing ntpd checked the refid accordingly,
it could determine if the server is actually smearing.

On the other hand, if a client polls several servers and only one (or
few) of them smears then the smearing servers would be discarded anyway
as falsetickers.

Smearing is a hack to avoid known problems with clients that have
problems when the system time is stepped back due to a leap second, and
as Heiko wrote, it can hurt the requirement to use accurate legal time
in an application.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Feb 2019 12:22:26 +0100

>>> Martin Burnicki <[email-masked]> schrieb am 13.02.2019 um 10:18 in
Nachricht <[email-masked]>:

[...]
> leap second is inserted. :-(

I think it's due to time() returning seconds since the epoch, not counting leap seconds. There should be a separate leap seconds counter in UNIX, but there isn't.

So when looking at the return value of time() only you MUST see a value that's jumping back after having inserted a leap second, even in those system who "correctly" handle leap seconds, right?

So Martin, are you saying that most UNIX systems do not know about leap seconds at all?

Regards,
Ulrich

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Feb 2019 12:29:27 +0100

>>> Martin Burnicki <[email-masked]> schrieb am 13.02.2019 um 10:48 in
Nachricht <[email-masked]>:
[...]
> is now considered obsolete.

The decision to send the TAI offset in an extension field only if the autokey protocol is being used was a very poor design decision actually.  Actually it was a reason to use the autokey protocol ;-)

Ulrich

---

From: Watson Ladd <[email-masked]>
Date: Wed, 13 Feb 2019 04:00:58 -0800

> > What do you believe the NTP UTC timestamp should be of the event drawn
https://www.itu.int/dms_pubrec/itu-r/rec/tf/R-REC-TF.460-6-200202-I!!PDF-E.pdf
?
> just go from 23:59:58 to 00:00:00.

The UTC timestamp format is a single binary number defined in RFC 5905.
What should it be at those two times?

I don't think we can have roughtime without an answer to that question, and
it's clear that TAI sidesteps this whole
problem and was prefered over smearing.

The discussion below seems curiously resistant to answering my question on
this matter.

> Dave's document discusses the problems with POSIX's approach and why he

David's approach will not work for Roughtime. Could you explain a version
of UTC that would?
We did specify smeared UTC, people clearly want TAI instead.

-- 
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: Danny Mayer <[email-masked]>
Date: Wed, 13 Feb 2019 08:02:54 -0500

> distinguish second "60" from a duplicate "59".
There is no duplicate 59. and there can be no "same" second.

Danny

---

From: Danny Mayer <[email-masked]>
Date: Wed, 13 Feb 2019 08:06:34 -0500

On 2/13/19 7:00 AM, Watson Ladd wrote:
>
>
> On Tue, Feb 12, 2019 at 6:54 PM Danny Mayer <[email-masked]
> <mailto:mayer@ntp..org>> wrote:
> >
> >
> The UTC timestamp format is a single binary number defined in RFC
> 5905. What should it be at those two times?
>
> I don't think we can have roughtime without an answer to that
> question, and it's clear that TAI sidesteps this whole
> The discussion below seems curiously resistant to answering my
> question on this matter.
>
Sorry but I must have missed what question you asked. Can you phase it
in a way that I (and perhaps others) can understand?

Danny

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 15:23:47 +0100

> The decision to send the TAI offset in an extension field only if the autokey protocol is being used was a very poor design decision actually.  Actually it was a reason to use the autokey protocol…

I absolutely agree. And a replacement should be available if autokey
isn't used anymore.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 16:00:33 +0100

Ulrich Windl wrote:
> I think it's due to time() returning seconds since the epoch, not counting leap seconds.

True, and the same for clock_gettime( CLOCK_REALTIME, ...)

> There should be a separate leap seconds counter in UNIX, but there isn't.

Yes, there is.

At least the Linux kernel maintains a TAI offset value. However, some
time sync software has to set the initial TAI offset in the kernel after
reboot. ntpd with autokey or a valid leap second file can do this.

The Linux kernel increments its TAI offset value automatically whenever
it has to insert a leap second.

Unfortunately this even happens if the kernel's TAI offset value has not
been set before, so after a leap second the TAI value may have been
incremented from 0 to 1, and you can't simply check if the TAI offset is
really valid by just checking if it's 0 or not.

> So when looking at the return value of time() only you MUST see a value that's jumping back after having inserted a leap second, even in those system who "correctly" handle leap seconds, right?

Right, that's what I meant. You get the same numeric value twice, and if
you feed that time_t into the gmtime() function you get of course the
same date and time twice, with second "59". If gmtime() knew about the
leap second status then it could simply return "60" instead of "59", and
this would become visible to the user.

> So Martin, are you saying that most UNIX systems do not know about leap seconds at all?

Of course they do insert a leap second, but usually step the time back
by one second, but usually you won't ever see a second "60" in the user
interface.

Danny said systems that eventually step the time back are not suitable
to work as NTP servers, and my conclusion was that most Unix systems
weren't suitable since they just do this in case of a leap second.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: "Ulrich Windl" <[email-masked]>
Date: Wed, 13 Feb 2019 16:37:13 +0100

Hi!

Probably one should ask the "POSIX people" that made that standard how a
localtime() routine can ever return 60 in tm_sec if the decision is only made
by reading a time_t value. ;-)

       char *asctime(const struct tm *tm);
       char *asctime_r(const struct tm *tm, char *buf);

       char *ctime(const time_t *timep);
       char *ctime_r(const time_t *timep, char *buf);

       struct tm *gmtime(const time_t *timep);
       struct tm *gmtime_r(const time_t *timep, struct tm *result);

       struct tm *localtime(const time_t *timep);
       struct tm *localtime_r(const time_t *timep, struct tm *result);

       tm_sec    The number of seconds after the minute, normally in the
range
                 0 to 59, but can be up to 60 to allow for leap seconds.

There's a symmetric problem for "time_t mktime(struct tm *tm);", BTW.

Regards,
Ulrich Windl

>>> Martin Burnicki <[email-masked]> schrieb am 13.02.2019 um
16:00 in
Nachricht <[email-masked]>:
> Ulrich Windl wrote:
>>>>> Martin Burnicki <[email-masked]> schrieb am 13.02.2019 um
10:18 
> in
>> I think it's due to time() returning seconds since the epoch, not counting

> leap seconds.
>> So when looking at the return value of time() only you MUST see a value 
> that's jumping back after having inserted a leap second, even in those
system 
> who "correctly" handle leap seconds, right?
>

---

From: Miroslav Lichvar <[email-masked]>
Date: Wed, 13 Feb 2019 16:46:55 +0100

> by reading a time_t value. ;-)

It will return 60 when using one of the "right" timezones, which
include leap seconds. Of course, that requires the system clock to not
keep time in the Unix timescale, but rather in "TAI-10" and that can
break all kinds of applications.

-- 
Miroslav Lichvar

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 13 Feb 2019 18:55:14 +0100

> There is no duplicate 59. and there can be no "same" second.

Of course there is.

I've just duplicated a test I've already run some years ago and put the
results here:
https://kb.meinbergglobal.com/kb/time_sync/leap_second_handling_by_the_linux_kernel

So you can see exactly what happens on a Linux system.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Danny Mayer <[email-masked]>
Date: Wed, 13 Feb 2019 13:10:44 -0500

> So you can see exactly what happens on a Linux system.

That's someone broken implementation. The ITU-T document makes it clear
that it's a different second.

Let's at least differentiate between what is implemented and what the
standard has stated.

Danny

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 13 Feb 2019 13:14:21 -0800

> I absolutely agree. And a replacement should be available if autokey

"Let me get in to my Delorean."  Please see
draft-stenn-ntp-extended-information, which had this in -00 in October
of 2016.

-- 
Harlan Stenn, Network Time Foundation
http://nwtime.org - be a Member!

---

From: Warner Losh <[email-masked]>
Date: Wed, 13 Feb 2019 14:31:44 -0700

> There is no duplicate 59. and there can be no "same" second.

In POSIX there most certainly is. There is no unique encoding for the leap
second to distinguish 59 from 60 (or in some interpretations 60 from 00).
That is the fundamental problem with POSIX that is absolutely not fixable,
absent additional metadata. That is what Martin is talking about here:
providing that data to resolve an otherwise ambiguous case.

Warner

Danny

---

From: Harlan Stenn <[email-masked]>
Date: Wed, 13 Feb 2019 14:09:48 -0800

We have a responsibility to let the POSIX folks know about their problems.

We don't have a responsibility to *fix* their problems.

H

On 2/13/2019 1:31 PM, Warner Losh wrote:
> 
> 
> On Wed, Feb 13, 2019, 6:03 AM Danny Mayer <[email-masked]
> <mailto:[email-masked]> wrote:
> 
> 
> In POSIX there most certainly is. There is no unique encoding for the
> leap second to distinguish 59 from 60 (or in some interpretations 60
> from 00). That is the fundamental problem with POSIX that is absolutely
> not fixable, absent additional metadata. That is what Martin is talking
> about here: providing that data to resolve an otherwise ambiguous case.
> 
> Warner
> 
>     Danny
> 
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Warner Losh <[email-masked]>
Date: Wed, 13 Feb 2019 15:25:07 -0700

I agree. History has shown there is no political will to change this aspect
of POSIX. Hisory has also show leap seconds as a means to keep UTC in sync
are not loved by people having to implement them, but also not hated enough
in the public at large for there to be any  change. So any solution will
need to consider the two incompatible realities coexisting :(

Warner

---

From: "Ulrich Windl" <[email-masked]>
Date: Thu, 14 Feb 2019 08:24:39 +0100

>>> Danny Mayer <[email-masked]> schrieb am 13.02.2019 um 19:10 in Nachricht

>>>>>> What do you believe the NTP UTC timestamp should be of the event drawn
https://www.itu.int/dms_pubrec/itu‑r/rec/tf/R‑REC‑TF.460‑6‑200202‑I!!PDF‑E.pdf?

>>>>> What you are seeing there is that when adding a leap‑second into the
https://kb.meinbergglobal.com/kb/time_sync/leap_second_handling_by_the_linux_

> kernel
> That's someone broken implementation. The ITU‑T document makes it clear

Let's make it positive: Who can show an implementation that worked as
designed, i.e. showing the 61st second in the minute?

Regards,
Ulrich

---

From: Martin Burnicki <[email-masked]>
Date: Thu, 14 Feb 2019 09:13:58 +0100

> about here: providing that data to resolve an otherwise ambiguous case.

Exactly. Thanks!

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Martin Burnicki <[email-masked]>
Date: Thu, 14 Feb 2019 09:15:23 +0100

Harlan,

> draft-stenn-ntp-extended-information, which had this in -00 in October

I know, and I hope this becomes available for usage soon.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 14 Feb 2019 09:40:43 +0100

> providing that data to resolve an otherwise ambiguous case.

The question is how NTP should deal with this ambiguity. Clients can
ignore measurements with timestamps in those two seconds and servers
could perhaps respond with "not synchronized". I think some NTP
implementations do this. Maybe it should be added to the
specification.

-- 
Miroslav Lichvar

---

From: Harlan Stenn <[email-masked]>
Date: Thu, 14 Feb 2019 01:47:09 -0800

> implementations do this. Maybe it should be added to the

I don't think it's an issue.

If ntpd is running on the system, it knows what time it is, and it knows
if it's before, during, or after a leap second.

The "window of potential difficulty" gets smaller if either we or the
upstream have a valid leapsecond file, and newer versions of ntpd
already respond with LI=3 if a leap second is in progress.  Clients that
receive these packets should ignore these packets.

If ntpd is getting its time from Somewhere that doesn't have smarts
about the leap second, then there are a variety of scenarios that all
boil down to the worst case is there may be 0, 1, or 2 timestamps in
there that *may* have a problem.  The only reason there would be 2 is if
the client is doing an iburst volley during the leapsecond.  Otherwise
there *might* be one packet with an interesting timestamp if the poll
happens close enough to the leap event.

But NTP is pretty good about detecting and ignoring wonky packets.

So again, I don't think it's an issue.

But I could be missing something.  If so, I'd love to know what it is.

---

From: Martin Burnicki <[email-masked]>
Date: Thu, 14 Feb 2019 11:26:49 +0100

Miroslav,

> implementations do this. Maybe it should be added to the

NTP/ntpd *does* have the required information. If configured properly
with reliable sources then it *knows* that a leap second is approaching,
and it forwards this information to its own kernel (if the OS supports
this) as well as to its clients.

The problem is that the operating system steps the system time back by 1
s at UTC midnight. ntpd just picks up a timestamp of the current system
time when it sends a reply to a client.

If I remember correctly then it was you who made the proposal to let an
NTP server send "not synchronized" for a short interval across a leap
second, and I think this is a very good idea.

For those who don't know why this is a good thing:

If you have server and a client which both step the time back by 1
second when they insert the leap second, this will not happen *exactly*
at the same time but let's say 1 millisecond or 10 microseconds after
each other.

If a client happens to poll the server exactly at this moment then the
server may already have switched back, but the client has not, or vice
versa. In either case the client will compute a time offset of about 1 s
in this case, rather than the expected offset in the ms range or below.
This can mess up the filtering done at the client side.

However, if the server sends "not synchronized" then this single packet
is discarded, which doesn't hurt since the time control loop works
similar to a flywheel, and the previous as well as the next queries
yield consistent results and low offsets.

The kernel itself steps the system time back, and a daemon like ntpd
interacts with the kernel only in periodic intervals, when it is
scheduled to run, so it doesn't know at which point in time exactly the
kernel inserts the leap second. So it's a good idea to start the "not
synchronized" period early enough, and end it when it can be sure the
leap second has been inserted, so it can send a consistent status to its
clients.

In normal case each individual client polls the server only once every
64 seconds, or in even longer intervals, so for a single client it is
very unlikely that it even notices that the server sends replies with
"not synchronized" across a leap second, but *if* this accidentally
happens it helps to keep the client's time control loop more stable.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 14 Feb 2019 12:38:30 +0100

> This can mess up the filtering done at the client side.

NTP uses four timestamps, two on each side, so it also depends between
which timestamps the step happened. On the client side, it can happen
before T1, between T1 and T4, or after T4. On the server side, it can
be before T2, between T2 and T3, or after T3. So, there are 9
combinations and the error in the measured offset due to the leap
second can be -1, -0.5, 0, +0.5, +1.0 seconds.

> kernel inserts the leap second.

I think the daemon could use the ntp_gettime() call to read the clock
and check if the timestamp is before or after the step, assuming the
kernel reports this information reliably - which may not always be the
case, but I have not seen any implementation doing that. There may be
a performance loss.

It's probably safer and easier to just ignore any measurements that
have a timestamp in that ~3 second interval around midnight. It would
be nice if it was explained in the NTP specification.

-- 
Miroslav Lichvar

---

From: Martin Burnicki <[email-masked]>
Date: Thu, 14 Feb 2019 15:12:14 +0100

> second can be -1, -0.5, 0, +0.5, +1.0 seconds.

Yes, of course. I've simplified this a little bit since in worst case
the effect is the same.

> case, but I have not seen any implementation doing that. There may be

I'm assuming the problem with the ntp_gettime() call is that it takes
longer to execute than the clock_gettime() call, and it needs at least
some kind of critical section or mutex to make sure consistent data is
returned.

As I said earlier, the main problem is not with ntpd, ptpd, or whatever,
but with applications which randomly read the current system time to do
their work.

If all these applications called ntp_gettime() instead of clock_gettime(
CLOCK_REALTIME, ... ) whenever they needed an unambiguous time then

1.) in worst case this could severely affect the resulting accuracy if
the system time is read very often and accesses needed to be serialized, and

2.) AFAIK there is still no common runtime library function that could
evaluate the status and return a second number with value "60".

> be nice if it was explained in the NTP specification.

Indeed, it should be documented somewhere.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Tony Finch <[email-masked]>
Date: Thu, 14 Feb 2019 15:39:14 +0000

Miroslav Lichvar <[email-masked]> wrote:
> > absent additional metadata.
>
> The question is how NTP should deal with this ambiguity.

This discussion baffles me, because I thought all this had been settled
ages ago, as illustrated in Martin Burnicki's KB article:

i.e. the ntp_adjtime() system call provides the additional metadata that
Warner Losh refers to, which translates into the NTP leap indicator bits.

POSIX time is no worse than NTP time: there is a constant offset between
POSIX time and NTP time, so naive NTP has the same leap second ambiguity /
stepping backwards problem. But the leap indicator bits disambiguate so it
should be a non-issue in a correctly functioning system.

Look at the timestamps in RFC 5905:

   | 1 Jan 1970  | 40,587     | 0   | 2,208,988,800 | First day UNIX   |
   | 1 Jan 1972  | 41,317     | 0   | 2,272,060,800 | First day UTC    |
   | 31 Dec 1999 | 51,543     | 0   | 3,155,587,200 | Last day 20th    |
   |             |            |     |               | Century          |

2,208,988,800 is the constant offset between NTP and POSIX time.

POSIX time for 1999-12-31T00:00:00 is 946598400

2208988800 + 946598400 = 3155587200

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
The Minch: Southerly 6 to gale 8, occasionally 5 at first, perhaps severe gale
9 later in north. Slight or moderate just to the north of Skye, but elsewhere
moderate or rough. Drizzle then fair. Good, occasionally moderate at first.

---

From: Warner Losh <[email-masked]>
Date: Thu, 14 Feb 2019 11:19:54 -0700

On Thu, Feb 14, 2019 at 1:40 AM Miroslav Lichvar <[email-masked]>
wrote:

> leap

NTP solved this problems years ago by having a flag on the measurement
saying that it's during a leap second. So while the numbers used to
exchange the time can be ambiguous, there's enough metadata so that the
server knows what's going on.

POSIX never did, so ntp_adjtime and other interfaces are needed for ntpd to
cope on a POSIX system. Thankfully, those are nearly universally
implemented.

Warner

---

From: Miroslav Lichvar <[email-masked]>
Date: Fri, 15 Feb 2019 08:57:22 +0100

> server knows what's going on.

Which flag are you referring to? The leap indicator just says that the
last minute of the day or month (depending on where in RFC 5905 you
look at) has 61 or 59 seconds. It doesn't flip between the 60th and
61th second.

> cope on a POSIX system. Thankfully, those are nearly universally

ntp_adjtime/gettime are nice, but AFAIK nothing is using them to read
the clock, not even the NTP implementations.

-- 
Miroslav Lichvar

---

From: Warner Losh <[email-masked]>
Date: Fri, 15 Feb 2019 08:32:12 -0700

> look at) has 61 or 59 seconds. It doesn't flip between the 60th and

Ah, true. I'd recalled it was the upper bits of the timestamp, but can't
find that now. You know it could be a bad one if it's in the last two
seconds of the day. The negative leap second is unambiguous, though, if you
have the flag. But since we've never had negative leaps, and likely never
will, it's a moot point.

> the clock, not even the NTP implementations.

Systems that care about leap seconds use them so that the leap second is
properly flagged. But not enough systems do that, so everybody gets the
leap second wrong because the traditional POSIX interface encourages a
"well, it's just a second every few years, so why bother" attitude. :(

Warner

---

From: Tony Finch <[email-masked]>
Date: Fri, 15 Feb 2019 16:04:20 +0000

Warner Losh <[email-masked]> wrote:
> seconds of the day.

That implies NTP servers cannot correctly answer around a positive leap
second :-( You also have to avoid the first second following the leap.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
an equitable and peaceful international order

---

From: Warner Losh <[email-masked]>
Date: Fri, 15 Feb 2019 09:52:35 -0700

> second :-( You also have to avoid the first second following the leap.

Yes. I believe that's what's done since the time deltas for NTP differences
are small enough that sleep(1) is an effective "solution".

Warner

---

From: "Ulrich Windl" <[email-masked]>
Date: Mon, 18 Feb 2019 08:49:48 +0100

>>> Tony Finch <[email-masked]> schrieb am 15.02.2019 um 17:04 in Nachricht
<[email-masked]>:
> second :‑( You also have to avoid the first second following the leap.

I wonder: The LI seems to be an "input" leap indicator ("things to be done"),
but not an "output" leap indicator ("things actually done"): If the leap status
would be indicated during leap second processing, the time sent would be
correct, right?

Regards,
Ulrich
> ‑‑

---

From: Tony Finch <[email-masked]>
Date: Mon, 18 Feb 2019 14:41:33 +0000

> would be indicated during leap second processing, the time sent would be

Yes, although the leap indicator bits can also be "things just done but
still requiring cleanup". And, as you say, what is missing in NTP packets
is something like the return value from ntp_adjtime():

       TIME_OK     Clock synchronized, no leap second adjustment pending.

       TIME_INS    Indicates that a leap second will be added at the end of
                   the UTC day.

       TIME_DEL    Indicates that a leap second will be deleted at the end of
                   the UTC day.

       TIME_OOP    Insertion of a leap second is in progress.

       TIME_WAIT   A leap-second insertion or deletion has been completed.
                   This value will be returned until the next ADJ_STATUS
                   operation clears the STA_INS and STA_DEL flags.

       TIME_ERROR  The system clock is not synchronized to a reliable server.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Selsey Bill to Lyme Regis: West or southwest, 4 or 5. Slight or moderate,
occasionally rough at first. Occasional rain or showers, becoming fair. Good,
occasionally moderate.