Subject: Re: [Tzdist] Getting to closure Re: [tzdist] #31 (service): Include leap seconds?

From: Stephen Colebourne <[email-masked]>
Date: Mon, 1 Dec 2014 09:08:02 +0000

When developing JSR-310 for Java, we ended up omitting any direct support
for leap seconds. Part of that was because there wasn't a good source of
data -although early drafts of the spec did distribute the table of leap
second dates with the tz data. I think that providing the raw set of leap
second dates (plus some form of indicator as to whether one is added or
removed) would be a useful addition to this service. I also think it is
closely related.

The use of the "right" timezone IDs, as Paul suggests, is a different
thing. Those conflate two concepts - tz and leaps. I have no objection to
having the "right" timezone IDs published, however it would be a great
service to permit the leap second raw data to be sent as well (just as the
current tzdist spec has both the rules and the expanded form of tz data).

wrt the charter text, one could argue that leap seconds are an essential
part of determining the correct time, and thus in scope, or one could argue
that they are niche and so are out of scope. There is no doubt in my mind
that the data would be used if provided, but probably by only a small
fraction of all consumers of the protocol.

If it were added, something simple like having /leapsecs return a JSON
document would be sufficient.

Stephen

On 28 Nov 2014 11:54, "Eliot Lear" <[email-masked]> wrote:

>  Dear colleagues,
>
> Before we include ANY support for leap seconds I would like to understand
> whether or not non-stratum 1 implementation takes advantage of the
> leapseconds file for purposes of setting the clock.  Personally I cannot
> imagine this is the case.  And if it is not the case, then I would argue
> that including leapseconds in the protocol goes beyond its purpose, as
> specified in our charter, the relevant text of which is really here:
>
>  Time zone changes are
> often delivered as operating system updates, and are thus tied to
> release schedules that can trail the actual time zone changes by a
> significant period of time. A service is needed that can provide timely,
> reliable updates.
>
>
> At this point, as co-chair, I will look dimly at *any* change that is
> beyond that purpose, especially if it overly complicates the protocol.  Put
> another way, I will close issues unless the case is clearly made at this
> point that any new changes are satisfying the above goal.  I apologize for
> not adhering a bit more strictly to the charter in the past.
>
> Furthermore, I expect this group to come to consensus this on document in
> the coming weeks.
>
> Eliot
>
> _______________________________________________
> Tzdist mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tzdist
>
>

---

From: Lester Caine <[email-masked]>
Date: Mon, 01 Dec 2014 10:13:49 +0000

On 01/12/14 09:08, Stephen Colebourne wrote:
> When developing JSR-310 for Java, we ended up omitting any direct
> support for leap seconds. Part of that was because there wasn't a good
> source of data -although early drafts of the spec did distribute the
> table of leap second dates with the tz data. I think that providing the
> raw set of leap second dates (plus some form of indicator as to whether
> one is added or removed) would be a useful addition to this service. I
> also think it is closely related.
> 
> thing. Those conflate two concepts - tz and leaps. I have no objection
> to having the "right" timezone IDs published, however it would be a
> great service to permit the leap second raw data to be sent as well
> (just as the current tzdist spec has both the rules and the expanded
> form of tz data).
> 
> part of determining the correct time, and thus in scope, or one could
> argue that they are niche and so are out of scope. There is no doubt in
> my mind that the data would be used if provided, but probably by only a
> small fraction of all consumers of the protocol.
> 

I was not aware of the /right situation, but is there any situation
where leapseconds have any variation due to timezone? I can't see any
reason one would want more than a single set of leapsecond data?

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 01 Dec 2014 13:04:20 +0100

Paul Eggert wrote:
> Cyrus Daboo wrote:
>> I believe this should be an extension to the base spec.
>
> I don't see why it needs to be an extension. The tz tradition is that
> the leap-second-aware TZ settings use a 'right/' prefix, e.g.,
> 'right/America/Los_Angeles' rather than 'America/Los_Angeles'. The data
> entries themselves use the same format either way. Surely iCalendar can
> handle leap-second-aware data, and if so, it could do something similar,
> without having to change the tzdist protocol.
>
> That being said, I don't see why tzdist doesn't also have a standard way
> to exchange binary tz data. The binary tz format is ubiquitous,
> portable, and just as commonly used as iCalendar if not more so.
> Billions of clients already use the binary format and won't be able to
> use iCalendar without serious upgrades and/or compatibility hassles.
> What's not to like?

Using this table for the "right" timezones is only one single way to use 
the table, and I agree that it would be easier to implement this on the 
client side if a binary data set could be retrieved.

This presumes that the binary format is really portable across all types 
of clients, e.g. 32/64 bit types of operating systems, little endian and 
big endian architectures (e.g. Motoroly/Sparc vs. Intel/AMD), etc. I'm 
not familiar with the details of the binary format used by the TZ project.

Does it really work to use the same binary time zone or leap second file 
with
- Solaris or Linux or *BSD on Sparc
- Linux or *BSD or Solaris on Intel x86
- Linux or *BSD or Solaris on Intel Itanium

If this all works, why haven't TZDB files just been distributed as 
binary files at least to all *ix clients? This would have made it much 
simpler to deploy updated TZ rules to many clients instead of waiting 
for OS maintainers to provide new binary packages deployed as software 
updates.

Also, beside the "right" timzones there are other applications which 
could be interested in a leap second table, e.g. NTP or PTP daemon 
software acting as proimary server / grandmaster.

So IMO providing a leap second table in a generic format (e.g. JSON) is 
the most important thing from point of view of the tzdist protocol, and 
providing a file in binary format in addition is an option which may be 
helpful, but also requires to take care that the generic and binary data 
is always updated consistently and provide exactly the same information.

I'm sure one the tzdist protocol has been officially released there will 
be tzdist clients (similar to NTP or PTP clients) which will pull TZ 
updates from a tzdist server and provide the OS or libraries with the 
updated information, which could include leap seconds.

Martin

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 01 Dec 2014 13:27:52 +0100

Clive D.W. Feather wrote:
> Lester Caine said:
>>> An expiration date for the table would also be good so the client can
>>> determine if the server's data have been updated recently enough.
>> We HAVE to have a version identifier of some sort, but for leapsecond
>> data that is just the last entry. We do not know when the next one will
>> be added do we?
>
> No, but sometimes we know that it *won't* be added at the next opportunity.
>
> Bulletin C 48 was issued on 2014-07-07. Since then we've known that the
> next leap second will be no earlier than 2015-06-30 [1], so right now we
> could have an expiry date of 2015-06-29 on the data, whereas on 2014-07-06
> the expiry date would have been 2014-12-30.

This is similar to the way ntpd now uses the NIST leap second file.

Without an expiration date ntpd would never know if a decision has been 
made by the IERS to add another leap second at the next possible 
opportunity, or not, or if just the leap second file hasn't been updated 
by a newer version which may have been published by NIST.

If the expiration date has not yet expired a piece of software like ntpd 
or ptpd can be sure to handle leap seconds correctly since all 
information is available. However, if the file has expired the software 
can't be sure.

Coming back to the example I've recently mentioned: if you need to 
determine the exact interval (in seconds) between May 1, and August 3, 
for a given year you need to know if a leap second occurred at the end 
of June, 30, or not.

If your leap second data is current enough and there is no entry for 
June 30 there was no leap second, and you can compute the correct time 
interval. However, if the data set has expired you just don't *know* if 
there has been a leap second, or not, so your computation could be 
correct or not, but in any case a warning can be issued that it is 
impossible to make sure the computed difference is correct.

This should be similar to TZ rules which may be defined for a given 
period of time / number of years, or "until further notice", as used 
with the European TZ rules, AFAIK.

Martin

---

From: Matthew Bobowski <[email-masked]>
Date: Mon, 1 Dec 2014 12:04:21 -0500

Esteemed colleagues:

It is great to see the continued discussion of time zone distribution
service. Leap seconds is an important characteristic of overall time
management, and deserves a point of discussion.

Coming back to Eliot's point, gives reminder of the charter's purpose which
namely "allows timely updates of time zone data to be delivered to
clients." It seems important to consider the original input drafts by
Douglass and Daboo.

I don't know if leap seconds should or should not be included as part of
tzdist, but if it is I would question binary support and predilection
towards any particular operating system. Consider the possibilities that
this protocol may be used by divergent software systems. Additionally,
those systems are more likely to adopt and adhere to a protocol that is
unambiguous and provides discreet data. Perhaps, clarity of objectives can
help determine the outcome.

Thanks and with much appreciation,
-Matt

On Mon, Dec 1, 2014 at 7:27 AM, Martin Burnicki <
[email-masked]> wrote:

> opportunity, or not, or if just the leap second file hasn't been updated by
> a newer version which may have been published by NIST.
>
> or ptpd can be sure to handle leap seconds correctly since all information
> is available. However, if the file has expired the software can't be sure.
>
> determine the exact interval (in seconds) between May 1, and August 3, for
> a given year you need to know if a leap second occurred at the end of June,
> 30, or not.
>
> If your leap second data is current enough and there is no entry for June
> 30 there was no leap second, and you can compute the correct time interval.
> However, if the data set has expired you just don't *know* if there has
> been a leap second, or not, so your computation could be correct or not,
> but in any case a warning can be issued that it is impossible to make sure
> the computed difference is correct.
>
> This should be similar to TZ rules which may be defined for a given period
> of time / number of years, or "until further notice", as used with the
> European TZ rules, AFAIK.
>
> Martin
>
>

---

From: Paul Eggert <[email-masked]>
Date: Mon, 01 Dec 2014 10:41:09 -0800

On 12/01/2014 04:04 AM, Martin Burnicki wrote:
> This presumes that the binary format is really portable across all 
> types of clients, e.g. 32/64 bit types of operating systems, little 
> endian and big endian architectures (e.g. Motoroly/Sparc vs. 
> Intel/AMD), etc.

Yes, it is portable.  The same files can be used regardless of 
endianness, bit representation and the width of the machine word. For 
example, the reference tz code should work correctly even on Unisys 
ClearPath enterprise servers that use 36-bit ones' complement words.

> binary files at least to all *ix clients?

Because nobody asked :-).  Actually, I expect it's partly because we 
didn't want NIH (or IANA, now) to be burdened by billions of clients 
downloading the binary data.  tzdist should be much better for this sort 
of thing.

---

From: Martin Burnicki <[email-masked]>
Date: Tue, 02 Dec 2014 14:56:32 +0100

> ClearPath enterprise servers that use 36-bit ones' complement words.

Sounds great. I have to admit I haven't been aware of this fact.

> downloading the binary data.  tzdist should be much better for this sort

Of course I agree that it's not desirable to have every little client 
get a copy of the binary files directly from the TZ home server at IANA, 
similarly as it's not desirable that each embedded device with 
networking support from all over the world would send its NTP request 
directly to the primary servers at NIST, USNO, or PTB.

However, with the fact in mind that the binary version is portable it 
would have been much easier in the past years to provide systems with TZ 
or leap second updates, without having to deploy a specific software update.

There are many publicly accessible mirror servers providing copies of 
open source software images, etc., and I'm sure these servers could also 
have hosted copies of the binary TZ DB files.

Anyway, I think the new tzdist protocol is a very valuable resource to 
distribute this kind of things, but the information provided by this 
protocol should include *all* the basic things you need if you work with 
time, i.e. including the leap seconds table.

Martin

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Dec 2014 14:37:21 +0000

> time, i.e. including the leap seconds table.

The main thing *I* am looking at with tzdist is the 'expand' facility
using single id's for embedded device applications. That is something
that TZ would never be expected to provide, but is just the thing that
is needed in the real world. That the tzdist system also needs a
complete mirror of the underlying data is a given, and simply cloning
each binary version of TZ is an obvious base which should perhaps have
been considered before the current charter was frozen?

---

From: Rob Seaman <[email-masked]>
Date: Tue, 2 Dec 2014 07:52:33 -0700

On Dec 2, 2014, at 6:56 AM, Martin Burnicki <[email-masked]> wrote:

> Anyway, I think the new tzdist protocol is a very valuable resource to distribute this kind of things, but the information provided by this protocol should include *all* the basic things you need if you work with time, i.e. including the leap seconds table.

Yes.  One stop shopping!

Rob Seaman
National Optical Astronomy Observatory

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Dec 2014 17:42:13 +0000

>> Anyway, I think the new tzdist protocol is a very valuable resource to distribute this kind of things, but the information provided by this protocol should include *all* the basic things you need…

But one might say why isn't this part of the time distribution system ;)

---

From: Steve Allen <[email-masked]>
Date: Tue, 2 Dec 2014 09:52:06 -0800

> But one might say why isn't this part of the time distribution system ;)

The nitty gritty of the history isn't as important as a good way
forward, but the answer is basically that the original process for
setting up the current scheme did not include the rigorous safeguards
that the IETF requires.

--
Steve Allen                 <[email-masked]>                WGS-84 (GPS)
UCO/Lick Observatory--ISB   Natural Sciences II, Room 165    Lat  +36.99855
1156 High Street            Voice: +1 831 459 3046           Lng -122.06015
Santa Cruz, CA 95064        http://www.ucolick.org/~sla/     Hgt +250 m

---

From: Daniel Migault <[email-masked]>
Date: Fri, 12 Dec 2014 22:29:56 +0100

Hi,

Regarding the discussion on the leap second, I would suggest we make it
optional. If you strongly disagree or think that cannot be made optional,
then feel free to propose some text we agree for version 05.

Best Regards,
Daniel

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Steve Allen <[email-masked]>
Date: Fri, 12 Dec 2014 13:35:38 -0800

> then feel free to propose some text we agree for version 05.

Timezones with or without leap seconds are part of the data, not the
protocol.  Clients need more complexity to handle leap seconds, but
that's not part of the protocol.  So long as the namespace does not
forbid, and clearly describes, zones with leap seconds as distinct
from zones without leap seconds then the protocol should work fine.

---

From: Ken Murchison <[email-masked]>
Date: Fri, 12 Dec 2014 16:39:15 -0500

Hi Steve,

> from zones without leap seconds then the protocol should work fine.

Have you abandoned your idea distributing leapseconds as an Etc/TAI 
pseudo-timezone?  I thought that was a rather simple and elegant idea.

-- 
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University

---

From: Steve Allen <[email-masked]>
Date: Fri, 12 Dec 2014 13:56:24 -0800

> pseudo-timezone?  I thought that was a rather simple and elegant

That remains a possible way to do it.  Some clients might prefer to
handle the data in that format.  Other clients might prefer to handle
the data with zones named like the "right" nomenclature that is part
of the current tzdata.  I don't see where the protocol fails to handle
either method.

I do agree that a clear explanation of the nomenclature and difference
between "right"-style (zones which expect to count leap seconds) and
POSIX-style (zones which expect to not count leap seconds) is important.
I am happy to contribute to text of that sort, but it may be better to
derive it directly from text currently in the IANA tzdata files.

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 22:02:01 +0000

> pseudo-timezone?  I thought that was a rather simple and elegant idea.

Since leap seconds only has relevance where the processor is using a
clock based on a modified unix epoch style of clock, for systems that do
not work that way it has no effect on the TZ data used by these systems.
tzdist is not using unix epoch counts for timestamps, so a separate
'clock modifier' table is logical even if it is divergent from how TZ
data is - in some circumstances - distributed using epoch bsed binary
data. A separate set of data is the logical approach to allow clients to
make their own choices.

---

From: Ken Murchison <[email-masked]>
Date: Sat, 13 Dec 2014 11:39:22 -0500

For those that are interested, I have added an Etc/TAI time zone to my 
test server.

Full VTIMEZONE: https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc/TAI

Observances: 
https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc/TAI/observances?start=1970-01-01T00:00:00Z&end=2038-01-01T00:00:00Z

---

From: Stephen Colebourne <[email-masked]>
Date: Sun, 14 Dec 2014 08:45:59 +0000

While a slightly unusual approach, it is also quite neat. I'm happy
that approach meets any needs I've seen. Only question is how to
ensure that it is published from most servers. Include it in the
actual tzdb?
Stephen

On 13 December 2014 at 16:39, Ken Murchison <[email-masked]> wrote:
> For those that are interested, I have added an Etc/TAI time zone to my test
> server.
>

---

From: Steve Allen <[email-masked]>
Date: Mon, 15 Dec 2014 10:36:19 -0800

On Sat 2014-12-13T11:39:22 -0500, Ken Murchison hath writ:
> For those that are interested, I have added an Etc/TAI time zone to
> my test server.
>
> Observances: https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc/TAI/observances?start=1970-01-01T00:00:00Z&end=2038-01-01T00:00:00Z

That's nice to see that the tzdist protocol is handling the data.
It might be good to think about how that might be tweaked for
applications that want an answer other than zero between 1970 and
1972, but that is already problematic because TAI did not exist with
that name until 1971.  There is no right answer for proleptic cases
extending back as far as some of the zones which have been in tzdata.
And this is not relevant for tzdist.

I think that leaves the issue of including leap seconds in tzdist as a
name space issue, and that is explicitly outside the scope of the
draft.

However, it could be helpful to add a paragraph explaining that some
publishers may include zones where the time offsets are based on the
actual broadcast specification for the number of seconds which have
occurred in UTC as published by the ITU-R.  Those zones would be used
for systems where the underlying time scale does count leap seconds,
in contrast to POSIX time which does not count leap seconds.  Such a
paragraph should note that client programs using those zones will need
extra code to handle the leap second information.

If such a paragraph be included then it should include references to
1) the IANA tzdata and tzcode for the "right" zones
2) the ITU-R document defining UTC.  There is already an example for that
in informative reference 6 of RFC 7164
 [6]  ITU, "Standard-frequency and time-signal emissions", ITU-R
      TF.460-6, February 2002,
      <http://www.itu.int/rec/R-REC-TF.460/>.
3) the IERS publication Bulletin C which announces leap seconds, also
as seen in reference 12 of RFC 7164
 [12]  IERS Earth Orientation Centre, "Bulletin C - Product metadata",
       <http://datacenter.iers.org/web/guest/eop/-/somos/5Rgv/product/16>.

But in any case I think that tzdist issue #31 is already handled by the
protocol in the draft.

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 17 Dec 2014 15:55:23 +0100

Hi,

Daniel Migault wrote:
> optional. If you strongly disagree or think that cannot be made
> optional, then feel free to propose some text we agree for version 05.

I'm actually a little bit disappointed that all of the discussion on 
leap seconds is only with regard to the "right" time zones.

I've been hoping the new tzdist protocol would be a good chance to 
simplify all kind of applications which have to deal with times, and 
rely on regular updates coming from human intervention, like changed DST 
rules for particular time zones determined by individual countries, or 
new leap second events scheduled by the IERS.

In my opinion the "right" time zones are basically a good idea, but this 
is only *one* specific application which uses a list of leap second 
dates, and a big minus here is that an expiration date is missing from 
the TZ DB's leap second table representation.

As a summary:

1.) We have the leap second table file provided by NIST, which is 
directly used by NTP (for now). This table *does* include an expiration 
date, and this is important since programs like ntpd can emit a 
*warning* to the user or admin if the file is outdated. It is *not* 
possible to generate such a warning only based on some version info, if 
you (ntpd in this example) can't figure out if a newer version is available.

2.) We have the leap second file from TZ DB. This file has a different 
format than the NIST file, but as far as I can see is simply generated 
from a file in NIST format by a script which is part of the TZ DB 
package. Unfortunately the expiration date is *not* migrated into the TZ 
DB's leap second file, so some important information is lost.

By the way, do I recall correctly that TZ DB should only be *one* of the 
possible providers for the data distributed via the tzdist protocol?

3.) We have a leap second file from IERS, which is once more in a 
different format than the ones mentioned above.

In my opinion the IERS should be the authoritative source for a leap 
second file (or table) since this is the institution deciding whether a 
leap second is to be scheduled, or not. I've proposed this to the IERS 
folks some time ago, and also proposed to include an expiration date.

If you look at the IERS web page at
http://hpiers.obspm.fr/iers/bul/bulc/

then actually there is an outdated Leap_Second_History.datsave file 
dated 2014-01-28
http://hpiers.obspm.fr/iers/bul/bulc/Leap_Second_History.datsave

and also a newer Leap_Second_History.dat file dated 2014-09-30
http://hpiers.obspm.fr/iers/bul/bulc/Leap_Second_History.dat

which contains some more comments in its header, plus an *expiration date*.

So TZ DB's leap second file can alternatively be derived from the IERS 
file, but neither a file in NIST format nor in IERS format can be 
derived from a file in TZ DB format some information being lost.

On the other hand it should be easy for a tzdist client program to 
retrieve a generic table of leap second events plus expiration date from 
a tzdist server, and convert it to one or more formats required by 
applications like the "right" timezones, ntpd, ptpd, or the Network Time 
Forum's General Timestamp (GTS) API.
http://nwtime.org/projects/timestamp-api/

The leap second table expiration date may not be too important for the 
API calls which have actually been implemented to deal with the "right" 
timezones since existing API calls may not provide a way to alert the 
user or application if the table is outdated.

However, programs like ntpd (which already does so) or ptpd could emit a 
warning if they see that the expiration date is approaching, and the GTS 
API could set a warning flag if this is the case.

In real life it is perfectly possible that a system doesn't receive 
updates automatically, maybe in an isolated network, or because a 
caching server has not been updated and provides its clients with 
outdated information.

So my proposal is to let tzdist provide a list of leap second events 
plus an expiration date, as a simple table of UTC dates like 2014-12-31. 
This should not be too hard to be implemented but provides a huge 
benefit for future timekeeping applications, even though this may not be 
directly related to iCalendar applications.

Providing the "right" timezones with its leap second file, eventually in 
binary format, can be a good supplement but in my opinion the primary 
goal should be to make this data available in a generic format.

Best regards,

Martin

---

From: Ken Murchison <[email-masked]>
Date: Wed, 17 Dec 2014 10:15:12 -0500

On 12/17/2014 09:55 AM, Martin Burnicki wrote:
> rely on regular updates coming from human intervention, like changed 
> DST rules for particular time zones determined by individual 
> countries, or new leap second events scheduled by the IERS.
>
> In my opinion the "right" time zones are basically a good idea, but 
> this is only *one* specific application which uses a list of leap 
> second dates, and a big minus here is that an expiration date is 
> missing from the TZ DB's leap second table representation.
>
>
> As a summary:
>
> directly used by NTP (for now). This table *does* include an 
> expiration date, and this is important since programs like ntpd can 
> emit a *warning* to the user or admin if the file is outdated. It is 
> *not* possible to generate such a warning only based on some version 
> info, if you (ntpd in this example) can't figure out if a newer 
> version is available.
>
> package. Unfortunately the expiration date is *not* migrated into the 
> TZ DB's leap second file, so some important information is lost.
>
> By the way, do I recall correctly that TZ DB should only be *one* of 
> the possible providers for the data distributed via the tzdist protocol?
>
> second file (or table) since this is the institution deciding whether 
> a leap second is to be scheduled, or not. I've proposed this to the 
> IERS folks some time ago, and also proposed to include an expiration 
> date.
>
> retrieve a generic table of leap second events plus expiration date 
> from a tzdist server, and convert it to one or more formats required 
> by applications like the "right" timezones, ntpd, ptpd, or the Network 
> Time Forum's General Timestamp (GTS) API.
> API calls which have actually been implemented to deal with the 
> "right" timezones since existing API calls may not provide a way to 
> alert the user or application if the table is outdated.
>
> However, programs like ntpd (which already does so) or ptpd could emit 
> a warning if they see that the expiration date is approaching, and the 
> GTS API could set a warning flag if this is the case.
>
> plus an expiration date, as a simple table of UTC dates like 
> 2014-12-31. This should not be too hard to be implemented but provides 
> a huge benefit for future timekeeping applications, even though this 
> may not be directly related to iCalendar applications.
>
> Providing the "right" timezones with its leap second file, eventually 
> in binary format, can be a good supplement but in my opinion the 
> primary goal should be to make this data available in a generic format.

I have already done this on my demo server and I posted a couple of 
links a few days ago.  The first link provides the leap data in 
iCalendar format and the TZUNTIL property shows the expiration.  The 
second link provides the leap data in the tzdist JSON format for 
observances and the toplevel "end" string shows the expiration.

https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc%2FTAI
https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc%2FTAI/observances?start=1970-01-01T00:00:00Z&end=2038-01-01T00:00:00Z

---

From: Martin Burnicki <[email-masked]>
Date: Wed, 17 Dec 2014 17:24:03 +0100

Hello Ken,

> https://cyrus-test.andrew.cmu.edu/tzdist/zones/Etc%2FTAI/observances?start=1970-01-01T00:00:00Z&end=2038-01-01T00:00:00Z

Thanks. I had seen your email but had misunderstood it.

Great that is is now supported.

Martin

---

From: Paul Eggert <[email-masked]>
Date: Wed, 17 Dec 2014 10:11:57 -0800

> TZ DB's leap second file, so some important information is lost.

Thanks for reporting this.  I have installed the attached fix in the 
experimental tz sources on github and it should appear in the next tz 
release.  I am cc'ing the tz mailing list to give them a heads-up.

> a leap second is to be scheduled,

I would prefer this also.  However, the IERS file is copyrighted. That 
is why the tz database reproduces the NIST file (the NIST file is public 
domain).  Other distributors of the IERS file might want to keep this in 
mind.

Amusingly enough, the NIST file's expiration date disagrees with the 
IERS's.  As a practical matter the NIST's date is more conservative and 
is a better choice for applications like tz and tzdist and this is 
another argument for using the NIST file.

Perhaps someday the IERS will address these two concerns.
>From 951395058c6d9e069ccbf767669bddc60bcbe0c8 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Wed, 17 Dec 2014 10:02:57 -0800
Subject: [PATCH] * leapseconds.awk, NEWS: Also output expiration date comment.

Problem reported by Martin Burnicki in:
http://www.ietf.org/mail-archive/web/tzdist/current/msg01132.html
---
 NEWS            | 3 +++
 leapseconds.awk | 9 +++++++++
 2 files changed, 12 insertions(+)

diff --git a/NEWS b/NEWS
index b86822e..674d353 100644
--- a/NEWS
+++ b/NEWS
@@ -33,6 +33,9 @@ Unreleased, experimental changes
 
   Changes affecting commentary
 
+    The leapseconds file commentary now mentions the expiration date.
+    (Problem reported by Martin Burnicki.)
+
     Update Mexican Library of Congress URL.

diff --git a/leapseconds.awk b/leapseconds.awk
index bb81e96..756391e 100644
--- a/leapseconds.awk
+++ b/leapseconds.awk
@@ -37,6 +37,11 @@ BEGIN {
 }
 
 /^ *$/ { next }
+
+/^#\tUpdated through/ || /^#\tFile expires on:/ {
+    last_lines = last_lines $0 "\n"
+}
+
 /^#/ { next }
 
 {
@@ -64,3 +69,7 @@ BEGIN {
     }
     old_TAI_minus_UTC = TAI_minus_UTC
 }
+
+END {
+    printf "\n%s", last_lines
+}
-- 
1.9.3

---

From: Steve Allen <[email-masked]>
Date: Thu, 18 Dec 2014 06:50:23 -0800

> Thanks. I had seen your email but had misunderstood it.

Yes, thanks again to Ken.  In addition to this I think it would be
instructive to include one example of a "right" zone on each side of
the Prime Meridian.  Ken, Martin, and I should probably engage about
getting the correct signs in all the data for those.  We can do the
nitty gritty details off list and announce the results when we agree
that they are working.

But again, as far as I am concerned, this issue #31 can be closed
because the protocol already handles it.

---

From: "tzdist issue tracker" <[email-masked]>
Date: Thu, 18 Dec 2014 16:54:56 -0000

#31: Include leap seconds?

Comment (by [email-masked]):

 Nice work done by Ken with proof of concept with a pseudo-timezone.
 Question remains whether to codify this in some way.  Discussion to occur
 on list.  People to get their views out onto list by the 9th of January.

 Should we have considerations for leap seconds in the document?
 Should we handle leap second with a pseudo timezone or some other way?

 Maybe not /zones/etc/TAI but /leapsecond.

 We need a proposal.
 Ken has done a JSON representation

-- 
------------------------------------------+-------------------------------
 Reporter:  [email-masked]  |       Owner:  [email-masked]
     Type:  defect                        |      Status:  new
 Priority:  major                         |   Milestone:
Component:  service                       |     Version:
 Severity:  -                             |  Resolution:
 Keywords:                                |
------------------------------------------+-------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/tzdist/trac/ticket/31#comment:1>
tzdist <http://tools.ietf.org/tzdist/>

---

From: Ken Murchison <[email-masked]>
Date: Thu, 18 Dec 2014 12:04:59 -0500

>   Ken has done a JSON representation

Below is the JSON observances representation from my demo server by 
doing an "expand" operation on Etc/TAI (clipped in the middle for 
brevity).  Obviously, if we decide on using a separate URI namespace 
such as /leapseconds, the "tzid" and "name" elements wouldn't make much 
sense, so we could either drop them or just substitute "leapseconds".

{
   "tzid": "Etc/TAI",
   "start": "1972-01-01T00:00:00Z",
   "end": "2015-06-28T00:00:01Z",
   "observances": [
     {
       "name": "TAI",
       "onset": "1972-01-01T00:00:00Z",
       "utc-offset-from": 0,
       "utc-offset-to": 10
     },
     {
       "name": "TAI",
       "onset": "1972-07-01T00:00:00Z",
       "utc-offset-from": 10,
       "utc-offset-to": 11
     },

...

     {
       "name": "TAI",
       "onset": "2009-01-01T00:00:00Z",
       "utc-offset-from": 33,
       "utc-offset-to": 34
     },
     {
       "name": "TAI",
       "onset": "2012-07-01T00:00:00Z",
       "utc-offset-from": 34,
       "utc-offset-to": 35
     }
   ]
}

---

From: Arthur David Olson <[email-masked]>
Date: Thu, 18 Dec 2014 12:39:17 -0500

While things have changed subsequently, on appropriately configured systems
you used to be able to set the environment variable TZ to...
    posix/America/New_York
...if you wanted time handling guaranteed to omit leap seconds or...
    right/America/New_York
...if you wanted time handling guaranteed to reflect leap seconds or...
    America/New_York
...if you were prepared to take whatever the system was configured to
deliver.

This approach might be used with tzdist; an indication of leap second
preference (or lack thereof) could be part of the URL used to request data.

    @dashdashado

On Thu, Dec 18, 2014 at 12:04 PM, Ken Murchison <[email-masked]>
wrote:
> Below is the JSON observances representation from my demo server by doing
> an "expand" operation on Etc/TAI (clipped in the middle for brevity).
> Obviously, if we decide on using a separate URI namespace such as
> /leapseconds, the "tzid" and "name" elements wouldn't make much sense, so
> we could either drop them or just substitute "leapseconds".
>
>
> {
>   "tzid": "Etc/TAI",

---

From: Lester Caine <[email-masked]>
Date: Thu, 18 Dec 2014 19:19:00 +0000

> preference (or lack thereof) could be part of the URL used to request data.

In the case of iCalendar format data, there would be no difference. One
ONLY needs a different view if one is using a simple binary dump using
only second values? Display everything as date and time and there is no
difference between the two views?

---

From: Matthew Bobowski <[email-masked]>
Date: Fri, 19 Dec 2014 11:21:39 -0500

I thought the goal of this endeavor was open text format (i.e. XML, JSON).
How does a binary dump help here? If this is proprietary to *Nix systems,
then it won't be adopted. Is the proposal to solve leap seconds to include
a binary transfer?

---

From: Steve Allen <[email-masked]>
Date: Fri, 19 Dec 2014 08:29:03 -0800

> ...if you were prepared to take whatever the system was configured to

And the sparse description of the "right" zones in tzdata and tzcode
has led to more than one situation like this
http://stackoverflow.com/questions/22686328/why-does-the-php-date-function-returns-a-wrong-time-off-by-24-seconds

> preference (or lack thereof) could be part of the URL used to request data.

I think that the tzdist protocol cannot forbid a publisher from
providing zone information like this.  The question is whether to
acknowledge that in the tzdist draft, and if so, what language to use.

A forewarning, because different authorities have chosen different
interpretations of what "UTC" means, there is no answer that satisfies
everyone and appeals to conform to the different authorities result in
flame wars.  Nevertheless, I will document my understanding of the
interpretation used in the "right" zones of tzdata/tzcode and ways
that tzdist might describing such use.

---

From: Martin Burnicki <[email-masked]>
Date: Fri, 19 Dec 2014 17:49:39 +0100

> release.  I am cc'ing the tz mailing list to give them a heads-up.

Great, thanks.

> domain).  Other distributors of the IERS file might want to keep this in

We (maybe I) could contact the IERS folks once more. Maybe they are just 
not aware that this copyright limits the files usage for TZ DB and/or 
tzdist.

On the other hand, the intention of the file should be to use it, or the 
information it provides, so why should there a problem extracting data 
from it using a script and feeding the result to TZ DB and/or tzdist?

> another argument for using the NIST file.

Agreed. If the expiration date is only e.g. 2 days before the next 
potential leap second then alerting may be too late, except if the 
application alerts let's say 2 weeks (or whatever) before the current 
date reaches the expiration date.

> Perhaps someday the IERS will address these two concerns.

Shall I contact the IERS folks with regard to this?

Martin

---

From: Martin Burnicki <[email-masked]>
Date: Fri, 19 Dec 2014 17:50:49 +0100

> nitty gritty details off list and announce the results when we agree

Yep.

Martin

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 19 Dec 2014 12:23:58 -0500

Hi Matthew,

> then it won't be adopted. Is the proposal to solve leap seconds to include

The default for tzdist should be a JSON format. However, if someone wants 
to register a media type for a binary format, then servers can also support 
that and clients could retrieve the binary data via use of the Accept 
header. If we anticipate the need for this, then we will need to add a list 
of supported media types for /leapseconds into the capabilities response - 
that would be trivial.

As for the JSON format - Ken's solution of using the existing expand action 
response is convenient because it already exists. However, we would 
probably want to tweak that to make the "oname" (observance name) optional 
so that it could be skipped in the /leapseconds response since it is 
redundant there. Similarly the "tzid" member in the root JSON object is 
redundant.

The alternative is to introduce a new JSON format for leapseconds that can 
be tailored for the exact schema we need:

{
  "changes":
  [
       "offset": 10
       "offset": 1
       "offset": 1,
    }
  ]
}

I think that is a little cleaner and it would avoid having to adapt the 
existing expand schema (and associated description). So I am going to 
propose we use a new JSON format for leapseconds (on the assumption that we 
can get quick consensus on that).

One question, the IANA/Olson data has things like this:

# Leap	YEAR	MONTH	DAY	HH:MM:SS	CORR	R/S
Leap	1972	Jun	30	23:59:60	+	S

In Ken's data he signifies the transition at T00:00:00, whereas the IANA 
data uses T23:59:60 (which is where the extra second gets added). What is 
better here? Do the other leapseconds data sources use the same format as 
the IANA data? What would happen in the (what I think is very unlikely) 
case that a second needs to be removed? Would the IANA data show T23:59:58?

-- 
Cyrus Daboo

---

From: Ken Murchison <[email-masked]>
Date: Fri, 19 Dec 2014 13:30:52 -0500

On 12/19/2014 12:23 PM, Cyrus Daboo wrote:
> The default for tzdist should be a JSON format. However, if someone 
> wants to register a media type for a binary format, then servers can 
> also support that and clients could retrieve the binary data via use 
> of the Accept header. If we anticipate the need for this, then we will 
> need to add a list of supported media types for /leapseconds into the 
> capabilities response - that would be trivial.
>
> As for the JSON format - Ken's solution of using the existing expand 
> action response is convenient because it already exists. However, we 
> would probably want to tweak that to make the "oname" (observance 
> name) optional so that it could be skipped in the /leapseconds 
> response since it is redundant there. Similarly the "tzid" member in 
> the root JSON object is redundant.
>
> The alternative is to introduce a new JSON format for leapseconds that 
> can be tailored for the exact schema we need:
>
> {
> I think that is a little cleaner and it would avoid having to adapt 
> the existing expand schema (and associated description). So I am going 
> to propose we use a new JSON format for leapseconds (on the assumption 
> that we can get quick consensus on that).

I'm fine with a new JSON format.

---

From: Paul Eggert <[email-masked]>
Date: Fri, 19 Dec 2014 10:55:21 -0800

Matthew Bobowski wrote:
> How does a binary dump help here? If this is proprietary to *Nix systems

The tz binary format is not proprietary: it's public domain, and is very widely 
adopted, more so than the text-based alternatives that are being proposed for 
tzdist.  It's also well-documented, portable, more compact and easier to process 
than the text-based forms, etc.  Its main downside, as I understand it, is that 
it's not an official format, something that I suppose could be easily rectified 
but I haven't the faintest idea how to go about it (and frankly was hoping that 
Someone Else would do it :-).

---

From: Paul Eggert <[email-masked]>
Date: Fri, 19 Dec 2014 11:39:53 -0800

On 12/19/2014 08:49 AM, Martin Burnicki wrote:
> On the other hand, the intention of the file should be to use it, or 
> the information it provides, so why should there a problem extracting 
> data from it using a script and feeding the result to TZ DB and/or 
> tzdist?
>

Yes, we can do that in the United States.  But we cannot reproduce the 
input file from the IERS; all we can do is reproduce the output data.  
Which, effectively, is what we're doing now, using the NIST file as an 
intermediary.

 From our point of view it would be better if the IERS's file were 
explicitly placed in the public domain or under a free-software license 
like the GPL or the BSD license, so that we could distribute the file as 
part of the tz distribution.

> Shall I contact the IERS folks with regard to this?

Sure, please feel free; you can cite this thread.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 19 Dec 2014 20:22:46 +0000

> from it using a script and feeding the result to TZ DB and/or tzdist?

There is no copyright issue in using the data; only the presentation in the
IERS file is copyright.

>> IERS's.

Since IERS set the rules, that's the date that matters.

-- 
Clive D.W. Feather          | If you lie to the compiler,
Email: [email-masked]     | it will get its revenge.
Web: http://www.davros.org  |   - Henry Spencer
Mobile: +44 7973 377646

---

From: Eliot Lear <[email-masked]>
Date: Sat, 20 Dec 2014 10:18:10 +0100

For what it's worth, I am considering opening an issue to include a new
media type for the tz database binary.  I would like to hear views on
this, but expect slow response as I will be out of the office from today
until the 5th.

Regards,

Eliot

On 12/19/14 7:55 PM, Paul Eggert wrote:
> The tz binary format is not proprietary: it's public domain, and is
> very widely adopted, more so than the text-based alternatives that are
> being proposed for tzdist.  It's also well-documented, portable, more
> compact and easier to process than the text-based forms, etc.  Its
> main downside, as I understand it, is that it's not an official
> format, something that I suppose could be easily rectified but I
> haven't the faintest idea how to go about it (and frankly was hoping
> that Someone Else would do it :-).
>

---

From: Daniel Migault <[email-masked]>
Date: Tue, 23 Dec 2014 10:25:51 +0100

I see the binary format as an additional Time Zone Format which could be
added later. I am just wondering whether having a single spec for
distributing leap seconds and tz would not require that format must be
valid for leap second and tz. I have no idea whether it is an awful idea or
not, but maybe we could work around this issue with a separate document for
"adaptation of tz dist service for leap seconds", making these two services
independent.  -- of course it could also be a section in the current
document.

BR,
Daniel

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58