Subject: Re: [Tzdist] [tzdist] #31 (service): Include leap seconds?

From: Daniel Migault <[email-masked]>
Date: Fri, 28 Nov 2014 11:31:52 +0100

Hi,

Walking through still opened issues, I would like to know whether we have
consensus on this issue, and eventually raise the impact on the URL format.

BR,
Daniel

On Thu, Oct 23, 2014 at 5:39 PM, Daniel Migault <[email-masked]> wrote:

> Adding comment from Paul to the list.
>
> I think that leap second discussions shoudl go wit that thread.
>
>
> The standard tz distribution has leap second information, and if we add
> its binary format as a media type it should all work.  Different
> distributions use different names for it, though; the default is the
> somewhat-awkward low-level naming convention TZ='../zoneinfo-leaps/America/
> Los_Angeles' to get Los Angeles time with leap seconds.  Presumably a
> better naming convention could be standardized.
>
>
> On Thu, Oct 23, 2014 at 3:37 PM, Lester Caine <[email-masked]> wrote:
>
>> On 23/10/14 14:10, Martin Burnicki wrote:
>> >> I dispute the suggestion that this is important for historic data.
>> >> It only has relevance if the historic data has only been stored using
>> >> unix epochs at which time the debate is to if the original 'clock' even
>> >> used leap seconds?
>> >
>> > Doesn't this depend on the kind of historic data you are evaluating?
>> >
>> > If you are postprocessing data which depends on time *intervals* then
>> > the number of leap seconds in this interval may well be important.
>>
>> That some historic data may have been created using unix epochs is a
>> fact, but that data has to be processed with the same software that
>> created it in order to return the correct time. Other more reliable
>> storage methods use date and time in a manor that there is less reliance
>> on establishing if the correct number of leap seconds were included in
>> the first place.
>>
>> It is perhaps the same problem as historic timeshots of the DST data. If
>> a system is reliant on only unix epochs with leapseconds added, then any
>> data post the addition of a new leap second needs correcting just as a
>> DST change will affect previously generated data.
>>
>> There has been a long discussion on another list about tidying up the C
>> time library and much of the sticking point is around making seconds
>> work as a base while a simple switch to time being a 'fraction' of a day
>> solves many of the problems, a timestamp becomes a simple pair of 32bit
>> numbers ...
>>
>> --
>> Lester Caine - G8HFL
>> -----------------------------
>> Contact - http://lsces.co.uk/wiki/?page=contact
>> L.S.Caine Electronic Services - http://lsces.co.uk
>> EnquirySolve - http://enquirysolve.com/
>> Model Engineers Digital Workshop - http://medw.co.uk
>> Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
>>
>> _______________________________________________
>> Tzdist mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/tzdist
>>
>
>
>
> --
> Daniel Migault
> Orange Labs -- Security
> +33 6 70 72 69 58
>

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Lester Caine <[email-masked]>
Date: Fri, 28 Nov 2014 11:16:56 +0000

On 28/11/14 10:31, Daniel Migault wrote:
> Walking through still opened issues, I would like to know whether we
> have consensus on this issue, and eventually raise the impact on the URL
> format.

>From my own point of view, leap seconds are secondary to any tz
identifier? They are a set of rules to apply to a particular clock so
should be added as an appendix to the draft rather than being a
required. leap second information can be obtained from other sources so
it should be optional for a publisher to include it with tzdist?

---

From: Martin Burnicki <[email-masked]>
Date: Fri, 28 Nov 2014 12:33:38 +0100

> it should be optional for a publisher to include it with tzdist?

What are you going to do with the information you get via the tzdist 
protocol?

Just imagine you need to compute the time span between 2 particular 
local date/time pairs. You surely try to find out if the 2 points in 
time were DST or not according to the DST rules for that year, but your 
computation may be wrong depending on whether a leap second has been 
inserted during that time interval, or not.

IMO a leap second table (list) should be made available by a 
full-featured tzdist server, so clients or applications which need this 
information can easily retrieve it in a consistent way, and clients or 
applications which don't need this can simply ignore it.

Being able to distribute leap second event dates via the tzdist protocol 
would very much simplify applications which otherwise had to try to 
collect this information from different locations on the internet, with 
the same limitations we have now when it's only possible to get updates 
e.g. of the TZ dist data base into embedded systems.

Martin

---

From: Eliot Lear <[email-masked]>
Date: Fri, 28 Nov 2014 12:53:36 +0100

Dear colleagues,

Before we include ANY support for leap seconds I would like to
understand whether or not non-stratum 1 implementation takes advantage
of the leapseconds file for purposes of setting the clock.  Personally I
cannot imagine this is the case.  And if it is not the case, then I
would argue that including leapseconds in the protocol goes beyond its
purpose, as specified in our charter, the relevant text of which is
really here:

> Time zone changes are
> often delivered as operating system updates, and are thus tied to
> release schedules that can trail the actual time zone changes by a
> significant period of time. A service is needed that can provide timely,
> reliable updates.

At this point, as co-chair, I will look dimly at *any* change that is
beyond that purpose, especially if it overly complicates the protocol. 
Put another way, I will close issues unless the case is clearly made at
this point that any new changes are satisfying the above goal.  I
apologize for not adhering a bit more strictly to the charter in the past.

Furthermore, I expect this group to come to consensus this on document
in the coming weeks.

Eliot

---

From: Lester Caine <[email-masked]>
Date: Fri, 28 Nov 2014 12:00:33 +0000

> inserted during that time interval, or not.

ONLY if you insist on using a clock that only counts seconds ...
AND it is being modified to be a particular type of 'seconds count'

We do not include dates and times a unix epochs in the standards, we use
a fully qualified value, so using a clock which counts days and handles
time as a fraction of a day will always give you the right answer ...
leap second is then just a variation on a days period.

---

From: Lester Caine <[email-masked]>
Date: Fri, 28 Nov 2014 12:10:43 +0000

> Furthermore, I expect this group to come to consensus this on document

Totally agree on your interpretation. My other post covers the relevant
point on where it is applicable, and I CAN see it being incorporated in
later drafts but not as a core requirement.

---

From: Paul Eggert <[email-masked]>
Date: Fri, 28 Nov 2014 08:47:34 -0800

Eliot Lear wrote:
> of the leapseconds file for purposes of setting the clock.

Many current clients of the tz database ship with tz's leap second support and a 
few applications use this. One example is chrony; see 
<http://listengine.tuxfamily.org/chrony.tuxfamily.org/chrony-dev/2012/02/msg00020.html>. 
On the Ubuntu desktop I'm using to type this message, I can get and set the 
clock using the leapseconds file by means of shell commands like the following:

export TZ='right/America/Los_Angeles'
date
date -s '2012-06-30 16:59:60'

The tzdist protocol must support this sort of use; otherwise, from Ubuntu's 
point of view tzdist won't be doing its job of transmitting time zone info in a 
timely way.

Presumably tzdist can do so by transmitting the tz database's binary file 
format, which reflects the contents of the leapseconds file. When tzdist 
transmits the binary file for 'right/America/Los_Angeles' it will be 
transmitting a file with leap seconds data. When it transmits the binary file 
for 'America/Los_Angeles' it won't. So there's no need to change the tzdist 
protocol to do this sort of thing, assuming servers are willing to ship the 
binary data (and why wouldn't they be?). Perhaps this should be mentioned in the 
tzdist document somewhere.

As I understand it the question about leap seconds is not whether tzdist can 
support leap seconds; it's whether the leap-seconds information should be part 
of the protocol or part of the data. I suggest the latter, since that already 
should work, and it shouldn't require changing the protocol.

[1] For example, chrony uses the tz database's leap second support; see 
<http://listengine.tuxfamily.org/chrony.tuxfamily.org/chrony-dev/2012/02/msg00020.html>.

---

From: Lester Caine <[email-masked]>
Date: Fri, 28 Nov 2014 17:02:42 +0000

On 28/11/14 16:47, Paul Eggert wrote:
> So there's no need to change the tzdist protocol to do this sort of
> thing, assuming servers are willing to ship the binary data (and why
> wouldn't they be?). Perhaps this should be mentioned in the tzdist
> document somewhere.

But the current draft has no provision for any binary data. If it did
then I would not have a problem with the data set being transferred.
iCalendar format can't easily be used to feed the existing tz code?
There will have to be some modification to handle the different rule
formats?

---

From: Paul Eggert <[email-masked]>
Date: Fri, 28 Nov 2014 09:16:00 -0800

Lester Caine wrote:
> But the current draft has no provision for any binary data.

Let's change it so that it does, then. That should be an easy fix. The binary 
data format is quite commonly used, and it ought to be called out in the 
standard anyway.  Leap-second problem solved.

---

From: Martin Burnicki <[email-masked]>
Date: Sat, 29 Nov 2014 00:16:25 +0100

Paul Eggert wrote:
> Let's change it so that it does, then. That should be an easy fix. The
> binary data format is quite commonly used, and it ought to be called out
> in the standard anyway.  Leap-second problem solved.

Hm, where is binary data coming into the game here?

Similar as the time zone information is passed as JSON data I think the 
list of leap second events can be distributed as JSON data.

After I had first made my proposal here on the list Ken Murchinson 
replied (Oct. 10) and reminded us that Steve Allen had already made a 
proposal:
http://www.ietf.org/mail-archive/web/tzdist/current/msg00070.html

I think the tzdist protocol should not transport binary data, at least 
not for this kind of data. Leap second events could be transmitted e.g. 
as a list of UTC dates at the end of which a leap second is inserted, or 
a date / TAI offset pair, as proposed by Steve, or the leap event times 
could be specified as the UTC number of seconds after an epoch, e.g. 
after 1900-01-01T00:00:00 UTC, as the NIST leap second file does it. 
None of these requires binary data, just a few JSON definitions.

An expiration date for the table would also be good so the client can 
determine if the server's data have been updated recently enough.

Similar as a tzdist client could receive time zone updates from a tzdist 
server and provide e.g. glibc with the updated DST rules, the client 
could also receive the leap second table in some kind of JSON format and 
convert it to whichever format is appropriate for the target system, 
e.g. to the binary format understood by glibc.

Martin

---

From: Lester Caine <[email-masked]>
Date: Sat, 29 Nov 2014 00:18:47 +0000

> Hm, where is binary data coming into the game here?
Well since the bulk of systems currently use the raw TZ data it would be
useful if we had something that could transfer the correct data. As has
already been said, iCalendar can not be converted transparently back to
the raw format TZ code currently uses. Paul was simply sugesting that
since the TZ binary format is already a standard that could be used.

> None of these requires binary data, just a few JSON definitions.
Paul is talking more about the full TZ data which includes leapsecond.
At the end of the day, leapseconds is only a list of timestamps, and
that could simply be a simple list ...

> determine if the server's data have been updated recently enough.
We HAVE to have a version identifier of some sort, but for leapsecond
data that is just the last entry. We do not know when the next one will
be added do we?

> e.g. to the binary format understood by glibc.
As long as the JSON format CAN be converted back to the same format as
is currently in general use. I still do not believe that an iCalendar
based rule set CAN as it changes the format and ordering defined in TZ?
The binary format will have to be modified and all the code updated -
won't it Paul?

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 28 Nov 2014 20:54:31 -0500

Hi Lester,

> since the TZ binary format is already a standard that could be used.

This is simple to solve via content negotiation. Here is what we can do:

1) Define a media type application/leapseconds+json as the required to 
implement format for retrieval of leapseconds via a URI 
{/service-prefix}/leapseconds.

2) Define a media type application/leapseconds+binary as an optional format 
that matches the IANA DB binary format.

3) Client can then use the HTTP accept header to get their preferred format 
(with the knowledge that a default JSON format is always available no 
matter what)

All that said, I believe this should be an extension to the base spec.

-- 
Cyrus Daboo

---

From: Paul Eggert <[email-masked]>
Date: Fri, 28 Nov 2014 18:32:52 -0800

Cyrus Daboo wrote:
> I believe this should be an extension to the base spec.

I don't see why it needs to be an extension. The tz tradition is that the 
leap-second-aware TZ settings use a 'right/' prefix, e.g., 
'right/America/Los_Angeles' rather than 'America/Los_Angeles'. The data entries 
themselves use the same format either way. Surely iCalendar can handle 
leap-second-aware data, and if so, it could do something similar, without having 
to change the tzdist protocol.

That being said, I don't see why tzdist doesn't also have a standard way to 
exchange binary tz data. The binary tz format is ubiquitous, portable, and just 
as commonly used as iCalendar if not more so. Billions of clients already use 
the binary format and won't be able to use iCalendar without serious upgrades 
and/or compatibility hassles. What's not to like?

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 28 Nov 2014 21:37:10 -0500

Hi Paul,

--On November 28, 2014 at 6:32:52 PM -0800 Paul Eggert <[email-masked]> 
wrote:

> That being said, I don't see why tzdist doesn't also have a standard way
> to exchange binary tz data. The binary tz format is ubiquitous, portable,
> and just as commonly used as iCalendar if not more so. Billions of
> clients already use the binary format and won't be able to use iCalendar
> without serious upgrades and/or compatibility hassles. What's not to like?

So all we need is a registration and formal definition of 
application/zoneinfo (or whatever is appropriate for the binary format) and 
clients can request that format in addition to iCalendar.

-- 
Cyrus Daboo

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 1 Dec 2014 06:26:10 +0000

> data that is just the last entry. We do not know when the next one will

No, but sometimes we know that it *won't* be added at the next opportunity.

Bulletin C 48 was issued on 2014-07-07. Since then we've known that the
next leap second will be no earlier than 2015-06-30 [1], so right now we
could have an expiry date of 2015-06-29 on the data, whereas on 2014-07-06
the expiry date would have been 2014-12-30.

[1] I'm assuming that Bulletin C implicitly excludes the March and
September options. If not, then adjust the proposed expiries accordingly.

-- 
Clive D.W. Feather          | If you lie to the compiler,
Email: [email-masked]     | it will get its revenge.
Web: http://www.davros.org  |   - Henry Spencer
Mobile: +44 7973 377646