Subject: Re: [Tzdist] Ticket#21 Semantics of year fields

From: Tim Parenti <[email-masked]>
Date: Tue, 2 Sep 2014 01:30:40 -0400

On 31 August 2014 14:24, Daniel Migault <[email-masked]> wrote:

> Of course, this issue does not seem completely solved to me.

That's because http://trac.tools.ietf.org/wg/tzdist/trac/ticket/21 and its
ensuing discussion have touched on several different questions, some of
which could arguably stand on their own as independent tickets:

*(21a)* Should truncation points and expansion boundaries be treated as
universal time?  Paul's proposal and my follow-up handle this nicely, I
think, and allow for times before the introduction of UTC by deferring to
RFC 3339.
http://www.ietf.org/mail-archive/web/tzdist/current/msg00274.html
http://www.ietf.org/mail-archive/web/tzdist/current/msg00276.html

*(21b)* Should truncation points and expansion boundaries have
second-precision, rather than just year-precision?  Paul noticed this
issue, and I suggested using ISO 8601 dates, as profiled by RFC 3339, so
that clients can issue requests with any appropriate level of precision.
We would still need to define, e.g., that dates like "1980" are to be
treated as equivalent "1980-01-01 00:00:00Z" for this purpose.
http://www.ietf.org/mail-archive/web/tzdist/current/msg00280.html
http://www.ietf.org/mail-archive/web/tzdist/current/msg00294.html

*(21c)* Should truncation points and expansion boundaries be returned in
the response?  I believe this is important so that a client can verify the
extent of the validity of the data it is receiving.  For example, knowing
that a request for [1900,2050) has only returned valid data for [1901-12-13
20:45:52Z, 2038-01-19 03:14:08Z).  I don't think there has been any
substantive discussion on this.  I did note, however, that this might be
partially covered by ticket 15 for VTIMEZONE; we would need something
similar for JSON responses.
http://www.ietf.org/mail-archive/web/tzdist/current/msg00276.html
http://trac.tools.ietf.org/wg/tzdist/trac/ticket/15

*(21d)* If yes to (21c), should truncation points and expansion boundaries
returned in the response be expressed in universal time per RFC 3339, or in
local time?  Paul suggests universal, and I am inclined to agree with him,
but Cyrus said he had some reasons to share in favor of using local times,
which I would like to see.
http://www.ietf.org/mail-archive/web/tzdist/current/msg00330.html
http://www.ietf.org/mail-archive/web/tzdist/current/msg00339.html

*(21e)* Should *all* times returned *anywhere* in the response be expressed
in universal time per RFC 3339?  I hadn't raised this question, but it
seems to me that (21d) might easily be confused for this (and perhaps this
is what Cyrus is actually getting at).  This would include transition onset
times and the like.  I'd like to hear more from Cyrus on this, too, before
firming up my opinion.

There are a few other small bits here and there that have come up, but
these are the major questions I've seen (so far) in this thread which I
believe should be addressed before closing this ticket.

--
Tim Parenti

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 09:28:56 +0100

On 02/09/14 06:30, Tim Parenti wrote:
> *(21d)* If yes to (21c), should truncation points and expansion
> boundaries returned in the response be expressed in universal time per
> RFC 3339, or in local time?  Paul suggests universal, and I am inclined
> to agree with him, but Cyrus said he had some reasons to share in favor
> of using local times, which I would like to see.
> *(21e)* Should *all* times returned *anywhere* in the response be
> expressed in universal time per RFC 3339?  I hadn't raised this
> question, but it seems to me that (21d) might easily be confused for
> this (and perhaps this is what Cyrus is actually getting at).  This
> would include transition onset times and the like.  I'd like to hear
> more from Cyrus on this, too, before firming up my opinion.

This precise point was one of my own introductions to the problem. For
archival data ALL that should be used is a UTC type base. Anything
involving local time is always client display related. It is the only
way to ensure consistency. In the case of data, the problems started
arising where servers were running on local time and trying to store
that along with it's TZ setting in the database, but they were ACTUALLY
working with times originated in a second TZ - that of the real client.
So basically what is 'local time'? Of cause the main problem was the use
of the browser offset to identify a client local time! The original
coders did not live in a DST area, so did not understand the problem. :(

The client knows the local time to display with the correct DST, which
would have to be pass to the server in order to change the data
supplied. So what is the point? This is not intended to be displayed
material?

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 09:40:09 +0100

> supplied. So what is the point? This is not intended to be displayed

I realised that I'd also missed the problem that is the current
discussion on tz. For a simple clock when a DST change happens, you get
the same hour twice and only using the underlying UTC time allows the
clock to work anyway.

---

From: Tim Parenti <[email-masked]>
Date: Tue, 2 Sep 2014 10:05:52 -0400

On 2 September 2014 04:28, Lester Caine <[email-masked]> wrote:

> Anything
> involving local time is always client display related.
>

Not necessarily; for example, "2014-09-02T10:05:52-04:00" is a
fully-specified RFC 3339 date which reflects my current local time, while
not being at all ambiguous dependent on your understanding of the rules of
my zone.

This is not an argument in favor of using local times, just pointing out
that the two options can be equivalent.  I think that, whichever option we
choose, we should keep the times fully and unambiguously specified.
(Perhaps this is a 21f to go alongside 21d and 21e.)  Obviously doing so is
fairly trivial with universal time, but it's completely possible with local
offsets, too.

--
Tim Parenti

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 15:47:16 +0100

On 02/09/14 15:05, Tim Parenti wrote:
> 
> On 2 September 2014 04:28, Lester Caine <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     Anything
> fully-specified RFC 3339 date which reflects my current local time,
> while not being at all ambiguous dependent on your understanding of the
> rules of my zone.
> 
> that the two options can be equivalent.  I think that, whichever option
> we choose, we should keep the times fully and unambiguously specified.
> (Perhaps this is a 21f to go alongside 21d and 21e.)  Obviously doing so
> is fairly trivial with universal time, but it's completely possible with
> local offsets, too.

No it's not 'completely possible' since I have no idea if -04:00 is in a
DST zone or not? This is where the process falls over in my book as
people then pass these 'fully-specified RFC 3339 dates' without
thinking. Exactly the problem I flagged.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 2 Sep 2014 15:45:11 +0100

> not being at all ambiguous dependent on your understanding of the rules of

Actually, that's fully specified irrespective of the rules of your zone.
It's an unambiguous specification even if you're in Hawaii or Shanghai.

> fairly trivial with universal time, but it's completely possible with local

I think we merely need to require a zone designator, either "Z" or an
offset.

-- 
Clive D.W. Feather          | If you lie to the compiler,
Email: [email-masked]     | it will get its revenge.
Web: http://www.davros.org  |   - Henry Spencer
Mobile: +44 7973 377646

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 2 Sep 2014 15:50:19 +0100

> thinking. Exactly the problem I flagged.

There seems to be some confusion as to what "the problem" is.

"2014-09-02T10:05:52-04:00" is umambiguous. It's the same as
"2014-09-02T14:05:52Z" and as
"2014-09-03T01:47:52+11:42".

None of them say anything about time zones in the IANA TZ DB sense, nor do
they say anything about whether DST is currently in effect or not. But I
don't understand why that's relevant. Unless I'm confused, the point here
is to specify a moment of last change, or something like that. For that,
all you need is an unambiguous time stamp.

---

From: Tim Parenti <[email-masked]>
Date: Tue, 2 Sep 2014 10:56:26 -0400

> It's an unambiguous specification even if you're in Hawaii or Shanghai.

That's exactly what I meant; apologies if that wasn't clear.  It refers to
a single unambiguous point in time, regardless of (a) the rules of my zone,
(b) the rules of your zone, (c) what your understanding of the rules of my
zone may or may not be.

> I think we merely need to require a zone designator, either "Z" or an

Exactly.  For (21f), I think this is fairly clear.  Whether that designator
is "Z" or an offset is what (21d) and (21e) are asking.

--
Tim Parenti

---

From: Tim Parenti <[email-masked]>
Date: Tue, 2 Sep 2014 11:00:04 -0400

On 2 September 2014 10:47, Lester Caine <[email-masked]> wrote:

> I have no idea if -04:00 is in a
> DST zone or not?
>

How is that relevant to determining at what instant to apply an offset?

My local UTC offset will go from -04:00 to -05:00 in a few months.  I could
say that that will happen at 2014-11-02T02:00:00 local time, but that's
ambiguous for a number of reasons.

Instead, I could say that it happens at any of:
2014-11-02T02:00:00-04:00
2014-11-02T01:00:00-05:00
2014-11-02T06:00:00Z
...and you would know exactly when to apply the change.

--
Tim Parenti

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 16:08:07 +0100

> all you need is an unambiguous time stamp.

In THIS case perhaps the question should be 'why add some random
number?' ... The problem I keep seeing is people posting dates with a
local 'offset' time which is actually wrong because it's during the next
DST period. It's 'fully specified', but it's wrong :(

---

From: Tim Parenti <[email-masked]>
Date: Tue, 2 Sep 2014 11:18:47 -0400

On 2 September 2014 11:08, Lester Caine <[email-masked]> wrote:

> The problem I keep seeing is people posting dates with a

"Wrong" in the sense that a wall-clock in the region in question will never
correspond to that time and offset, yes.  Take the next America/New_York
shift, for example.  Yours is a very valid concern against using
2014-11-02T02:00:00-04:00.  A counterargument in favor is that this is
typically how humans interpret and talk about the times at which
transitions occur.

My concern against using 2014-11-02T01:00:00-05:00 also appeals to human
readability.  Although it accurately represents the time and offset
immediately after the onset of the transition, it could easily be confused
with the equivalent time before the transition.  (That is, someone might
see this and think the clock should change *at* 01:00, rather than changing
*to* 01:00.)

That leaves universal time 2014-11-02T06:00:00Z, which avoids these issues
and seems simplest to me, in re: (21d) and (21e).

--
Tim Parenti

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 2 Sep 2014 16:16:53 +0100

Lester Caine said:
> number?' ...

Because people want to write "now" with their own local offset.

I can argue either way on this one. I'm slightly on your side, but for the
reason that there's no need for syntactic sugar on a feature that only
machines will normally read, so let's require the use of "Z". But it has to
be for a reason like that, not because of something spurious about DST.

> DST period. It's 'fully specified', but it's wrong :(

But people get the conversion to UTC equally wrong. People get things wrong
the whole time!

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 16:39:05 +0100

On 02/09/14 16:16, Clive D.W. Feather wrote:
>> It's 'fully specified', but it's wrong

Which is what tzdist should strive to correct :)

My point was that people use offsets currently without the mechanism
that ensures UTC conversion would be correct. If the tzdist side is
working right, with a UTC time, then client display of the local time
should show a problem while other local mechanisms may not. Sticks out
like a saw thumb when the meetings move an hour following the local
clock DST changes!

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 15:43:50 -0400

Hi Tim,
Thanks for this nice summary of the issue(s).

> http://www.ietf.org/mail-archive/web/tzdist/current/msg00276.html

The reason we originally went with just a "year" was to do with the 
expected "granularity" at which servers would be prepared to truncate data, 
on the basis that they will likely want to cache such data rather than 
truncate on-the-fly for each client request. Given that, we felt yearly 
intervals would be sufficient, and that midnight on January 1st was the 
correct point in the year. Obviously we missed the local time vs UTC 
distinction when specifying "midnight". I do think we should pick one of 
those and allow only that one. That will allow us to keep the query 
parameter to just the year number rather than a full RFC3339 date-time 
stamp.

> http://www.ietf.org/mail-archive/web/tzdist/current/msg00294.html

As per above, I don't think we should allow arbitrary precision/granularity 
in the truncation or expand boundaries as it can significantly impact 
server implementation. I think clients will find it easier to just deal 
with the year precision too, as they will also want to cache the result so 
it covers the likely range of dates that a user or device might "scan" over 
when viewing a calendar.

> that a request for [1900,2050) has only returned valid data for
> [1901-12-13 20:45:52Z, 2038-01-19 03:14:08Z).  I don't think there has
> been any substantive discussion on this.  I did note, however, that this
> might be partially covered by ticket 15 for VTIMEZONE; we would need
> something similar for JSON responses.

I am leaning towards defining new iCalendar properties for use in VTIMEZONE 
components to explicitly indicate the range of validity of the time zone 
data described by the component. We can do that in the tzdist spec and 
mandate the presence of those properties in the data delivered by tzdist.

> returned in the response be expressed in universal time per RFC 3339, or
> in local time?  Paul suggests universal, and I am inclined to agree with
> him, but Cyrus said he had some reasons to share in favor of using local
> times, which I would like to see.

Let's take expand first: clients are most likely to use local time as their 
"basis" for the events they are manipulating or displaying - definitely 
true for any client presenting event data to users. As such, its more 
convenient for them to get the expansion observances data relative to that 
chosen local time. Also, when a client has events in multiple time zones to 
display, it will have to map each of those to the time zone used for the 
display. That is likely done by a conversion to UTC, then a conversion from 
UTC to the display time zone basis. Given that I think using local time for 
the observances is the most convenient choice. I would also note that is 
what iCalendar uses for its DTSTART value in the STANDARD and DAYLIGHT 
components.

If we stick with local time for expand, the next question is how to express 
that in the JSON response. Currently we have:

      {
        "name": "Daylight",
        "onset": "2008-03-09T02:00:00",
        "utc-offset-from": -18000,
        "utc-offset-to": -14400
      },

where onset is the local time with the -from offset (i.e. the wall clock 
time when the transition occurs). That follow the iCalendar model of using 
DTSTART/TZOFFSETFROM/TZOFFSETTO. However, I could also envision using this:

      {
        "name": "Daylight",
        "localtime-from": "2008-03-09T02:00:00-05:00",
        "localtime-to": "2008-03-09T03:00:00-04:00",
      },

which is nicer from a human readability standpoint (as others have noted) 
because it clearly shows the "direction" in which the local time shift 
occurs (whereas the "utc-offset-from/to" variant requires doing some mental 
math to figure that out). For a computer it is trivial to convert between 
the two.

> expressed in universal time per RFC 3339?  I hadn't raised this question,
> but it seems to me that (21d) might easily be confused for this (and
> perhaps this is what Cyrus is actually getting at).  This would include
> transition onset times and the like.  I'd like to hear more from Cyrus on
> this, too, before firming up my opinion.

My preference is to stick with local times for the observances because 
there is no ambiguity there given the way the -from/-to is specified. For 
the expand/truncation years, there is the possibility of ambiguity if a DST 
transitions occurs very close to the chosen date+midnight. So there I think 
UTC should be used (year-01-01T00:00:00Z). Now that does mean a client has 
to ask for an expansion range that covers the previous year if it wants the 
expansion to cover local midnight for 1st January of a particular year. 
i.e. if the client only cares about 2014, it should ask for expansion for 
2013->2015 just to be sure it catches any transitions that might occur 
around 1st January. That shouldn't be a significant amount of extra data.

-- 
Cyrus Daboo

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 12:52:37 -0700

On 09/01/2014 10:30 PM, Tim Parenti wrote:
 > http://trac.tools.ietf.org/wg/tzdist/trac/ticket/21 and its ensuing 
discussion have touched on several different questions, some of which 
could arguably stand on their own as independent tickets:

Thanks for organizing these points.  I wish that the Trac tickets and/or 
the wiki were as easy to follow as your email.  I have gotten lost in 
the breadcrumb-trails of emails whose topics are poorly connected to 
Trac ticket numbers.

 > (21a) Should truncation points and expansion boundaries be treated as 
universal time?

The consensus seems to be "yes" here.

 > (21b) ... We would still need to define, e.g., that dates like "1980" 
are to be treated as equivalent "1980-01-01 00:00:00Z" for this purpose.

We could allow both forms, i.e., both RFC 3339 date-fullyear (the former 
form) and date-time (the latter).  The former would have an implied 
"-01-01 00:00:00Z" appended.  However, it'd be simpler to allow just 
date-fullyear, and that would be my mild preference.

 > (21c) Should truncation points and expansion boundaries be returned 
in the response?  ... I don't think there has been any substantive 
discussion on this.

You and I have both been in favor of this, and nobody has demurred. What 
further discussion would be helpful?  Are there some unresolved issues 
that you're thinking of?

 > (21d) ... should truncation points and expansion boundaries returned 
in the response be expressed in universal time per RFC 3339, or in local 
time?  ... Cyrus said he had some reasons to share in favor of using 
local times, which I would like to see. ...

Me too!

 > (21e) Should *all* times returned *anywhere* in the response be 
expressed in universal time per RFC 3339?

I would say "yes".  Allowing local-time time stamps in the metadata 
would be unnecessary complexity, with costs likely exceeding benefits.

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 20:56:26 +0100

On 02/09/14 20:43, Cyrus Daboo wrote:
> time when the transition occurs). That follow the iCalendar model of
> using DTSTART/TZOFFSETFROM/TZOFFSETTO. However, I could also envision
> using this:
> 
>      {
>        "name": "Daylight",

Sorry this is just an unacceptable joke!
Just what local time do you use for the 'extra' hour?

Just how do you expect to handle server situations where several clients
exit in many timezones?

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 21:03:42 +0100

> Just how do you expect to handle server situations where several clients
EXIST ... must get a new keyboard - worn this one out ...

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 16:15:57 -0400

Hi Lester,

> Just what local time do you use for the 'extra' hour?

What extra hour? The localtime-from represents the current local time at 
which a DST transition occurs tied to the offset in effect at that time 
(i.e., it unambiguously identifies a UTC value).

Here is the more complete example that does the -to value jumping back - 
but its still unambiguous because of the utc offset associated with it:

      {
        "name": "Daylight",
        "localtime-from": "2008-03-09T02:00:00-05:00",
        "localtime-to": "2008-03-09T03:00:00-04:00",
      },
      {
        "name": "Standard",
        "localtime-from": "2008-11-02T02:00:00-04:00",
        "localtime-to": "2008-11-02T01:00:00-05:00",
      },

The above just represent a point in time at which a transition occurs. 
Dealing with extra or missing hours as a result of that is up to the client 
using that data.

> exi(s)t in many timezones?

Not sure what this means. The client always has to handle the 
"implications" of the time zone data - all the tzdist server needs to do is 
send them the data, and in an unambiguous form.

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 21:25:02 +0100

> local times, which I would like to see. ...

Using local time may be a special case for simple single timezone
devices, but even then it has a number of difficulties. In situations
where one is dealing with more than one timezone it just does not work.

My previous post was a little vague, but normally I'm serving pages up
to many timezones, and so the server local time IS UTC ... anonymous
clients where I only have an offset only get UTC text, while identified
clients get their own local times or UTC depending on their choice.
Offset is a client display function ... and trying to run the server on
anything other than UTC just risks confusion.

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 21:35:42 +0100

On 02/09/14 21:15, Cyrus Daboo wrote:
> Dealing with extra or missing hours as a result of that is up to the
> client using that data.

So the first thing you do is convert it back to UTC? As the local clock
is running UTC anyway, it just needs the offset and the UTC time at
which to change it. Making the data packages POSH just makes things
confusing when you have several timezones being used, and the first
thing the simple single timezone device has to do is pull the whole lot
apart again anyway.

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 16:38:29 -0400

Hi Lester,

> where one is dealing with more than one timezone it just does not work.

Not true - all the calendar clients I use cope perfectly well with events 
in multiple time zones - even events where the start time is in one time 
zone, and the end in another, and the display (calendar "canvas") set to 
yet a different one, and the OS system clock yet another! All of that works 
well today provided the client has the proper time zone data.

> anything other than UTC just risks confusion.

At this point the debate is mostly about the action=expand response. That 
response has to include an unambiguous timestamp for each observance in the 
requested expand range. That can be done using UTC, or local time + offset 
- i.e., an ISO8601/RFC3339 value. The other piece of information needed is 
how the offset is changing. Note that in this case, the local time + offset 
is relative to the time zone being expanded - it is not an arbitrary choice 
for the server. So in the example I sent, I should have included the "tzid" 
value as part of the response snippet to make it clear that the observance 
was for that tzid and that the local time + offset being used was the one 
specific to that time zone.

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 21:42:32 +0100

On 02/09/14 20:43, Cyrus Daboo wrote:
> As per above, I don't think we should allow arbitrary
> precision/granularity in the truncation or expand boundaries as it can
> significantly impact server implementation. I think clients will find it
> easier to just deal with the year precision too, as they will also want
> to cache the result so it covers the likely range of dates that a user
> or device might "scan" over when viewing a calendar.

Ticket #1 - start event is not on a 1st January transition. Making the
start truncation on that boundary then needs ANOTHER transition for the
real start event. On an untruncated data set, every timezone has a start
event which needs a full timestamp, and by making the truncation point
that value removes the need to flag the period prior to the real start
date/time as not really the timezone.

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 16:49:30 -0400

Hi Lester,

> thing the simple single timezone device has to do is pull the whole lot

No - my whole point of using local time is that that is what most clients 
are doing anyway. My calendar client is currently set to a display time 
zone of America/New_York. Most of my events use that time zone. When 
drawing the events on the calendar canvas the client will likely have 
midnight local time (America/New_York) at the top, so to calculate the 
pixel offset for drawing an event, the client will want the time of that 
event relative to the display time zone.

So, my point is that using localtime+offset is a (minor) convenience for 
the client, and is functionally no different from using UTC. Clearly the 
use of (localtime+offset) needs careful explanation to indicate that it is 
tied to the specific time zone used for the expand. That seems to me to be 
causing some confusion - which is definitely a good argument for instead 
using UTC.

-- 
Cyrus Daboo

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 13:57:23 -0700

On 09/02/2014 12:43 PM, Cyrus Daboo wrote:

> *(21b)* ... I think clients will find it easier to  just deal with the year precision too, as they will also want to cache 
the result so it covers the likely range of dates that a user or device 
might "scan" over when viewing a calendar.

This would seem to overlook the point raised in 
<http://www.ietf.org/mail-archive/web/tzdist/current/msg00280.html>, 
which is that it's quite common for clients to be interested in time 
stamps in the range [-2**31, 2**31) seconds since 1970 -- this is the 
longstanding 32-bit POSIX time_t.

If the boundary granularity is 1 year, these clients cannot query for 
time stamps in their desired range.  I suppose they could ask for time 
stamps in the year range [1901,2039) (i.e., a bit more data than they 
need), but if the *server* is 32-bit it won't be able to supply the 
requested data and will reject the query or warn about a truncated 
response or something, and even if the server can service the request 
without problems then the *client* will then need to process the 
response's time stamps which in some cases will be outside its normal 
32-bit range.

So, in this common case, it is quite a mess for the protocol to insist 
on year granularity.  It's much simpler for both client and server if 
the protocol supports the natural boundaries of the client and/or 
server, which are rarely on year boundaries.

>> in local time?
> ... clients are most likely to use local time as their  "basis" for the events they are manipulating or displaying - 
definitely true for any client presenting event data to users. As such, 
its more convenient for them to get the expansion observances data 
relative to that chosen local time.

If the time stamp has an accurate UTC offset (e.g., "2008-03-09 
02:00:00-05:00"), then whether to allow non-UTC time stamps is purely a 
question of notational convenience within the protocol.  Clients and 
servers will need to be able to convert between UTC and localtime 
anyway, and the protocol's time stamps are typically not shown to users 
directly, so there is little benefit to requiring or even allowing 
support for localtime time stamps in the metadata.

However, not every time stamp in the protocol can have an accurate UTC 

>        "localtime-to": "2008-03-09T03:00:00-04:00",

does not work for the most recent transition in Africa/Monrovia, whose 
UTC offset for "localtime-from" is -2588 seconds, which is not a 
multiple of 60 and therefore cannot be expressed in ISO 8601 or RFC 3339 
format.

In summary, a client or server that wants to work in general must 
support UTC metadata.  That being the case, why should the protocol 
encourage clients or servers to use localtime metadata? It's extra 
complexity that provides no significant benefit and which messes up in 
some cases.

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 22:02:58 +0100

On 02/09/14 21:38, Cyrus Daboo wrote:
> So in the example I sent, I should have included the "tzid" value as
> part of the response snippet to make it clear that the observance was
> for that tzid and that the local time + offset being used was the one

Then one has to mentally convert those values between different
timezones to check which change happens first. If the base is already
UTC one knows straight away the order they will happen in and so which
part of a schedule will have which offset as the timetable rolls through
or moves between physical timezones.

Why do you need the content to be 'easily readable' when it is unlikely
the users will ever see it, and if we are looking at it, it's because
something is going wrong with the flow and we are more likely to need to
check physical information.

I'll expand on calendar rendering in your other post ....

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 17:15:14 -0400

Hi Paul,

--On September 2, 2014 at 1:57:23 PM -0700 Paul Eggert <[email-masked]> 
wrote:

> So, in this common case, it is quite a mess for the protocol to insist on
> year granularity.  It's much simpler for both client and server if the
> protocol supports the natural boundaries of the client and/or server,
> which are rarely on year boundaries.

One could argue the "mess" is caused by 32-bit time_t!

Let's take a step back and answer some questions:

1) Do we want servers to indicate they have time zone data for fixed 
truncation intervals? e.g., only data after 1970, only data after 2000 
(with some upper limit).

2) Do we want servers to support an arbitrary range of truncation as 
requested by the client?

3) Do we want the precision for the client requested ranges to be as low as 
one second (e.g., 1974-04-01T12:34:56Z)?

Personally I don't like the idea of #3 as it requires more work on the part 
of the server and also potentially represents a denial of service vector 
for a malicious clients. #1 is a definite must. #2 is a nice to have.

> In summary, a client or server that wants to work in general must support
> UTC metadata.  That being the case, why should the protocol encourage
> clients or servers to use localtime metadata? It's extra complexity that
> provides no significant benefit and which messes up in some cases.

OK, you have convinced me. So then the proposal is to have the 
"observances" member use a UTC time stamp for the "onset", e.g.:

      {
        "name": "Daylight",
        "onset": "2008-03-09T07:00:00Z",
        "utc-offset-from": -18000,
        "utc-offset-to": -14400
      },

instead of:

      {
        "name": "Daylight",

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 22:17:39 +0100

On 02/09/14 21:49, Cyrus Daboo wrote:
> No - my whole point of using local time is that that is what most
> clients are doing anyway. My calendar client is currently set to a
> display time zone of America/New_York. Most of my events use that time
> zone. When drawing the events on the calendar canvas the client will
> likely have midnight local time (America/New_York) at the top, so to
> calculate the pixel offset for drawing an event, the client will want
> the time of that event relative to the display time zone.

I don't see anything here that dictates using local time ... other than
the selection of the starting time for the calendar. I do the same thing
but can display either UTC base or local base, and naturally the local
base results in 23 or 25 hours on the relevant days ... with the correct
display of time including the extra hour.

> use of (localtime+offset) needs careful explanation to indicate that it
> is tied to the specific time zone used for the expand. That seems to me
> to be causing some confusion - which is definitely a good argument for
> instead using UTC.

No it's an even better argument for not introducing the potential
confusion at the distribution level with different offset times
appearing all over the place. The client will never see it in any case
which is an even better case not to use an assortment of local times.

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 22:59:39 +0100

> response's time stamps which in some cases will be outside its normal

For a 32bit client what would the 'full year' range actually be?
(1902,2038)? Loosing a bit of the first and last years is probably not a
problem if one does round to the nearest year?

I'm mixing up 'selected truncation' with 'historic truncation'. That
timezones started and may even end on random dates with start events
involving potentially sub second accuracy gives a structural limit
outside which different rules apply. The physical truncation due to 32
bit limits could be modelled the same way, giving errors for the
inaccessible parts of the first or last years?

Limiting 'selection' to a year boundary is not incompatible with the
real limits which would give an error if the selection is outside the
physical capability? However I don't see any difference in generating a
truncated view with random start and stop over one simply rounded to a
year. All the same calculations are required so why is one a DOS risk?
If someone tries to overload a website we just limit their access.

I'm easy on a year rounding, but I'd still use a full date stamp to
describe it so one only saves the time element, but standardizing
everything on a single date/time stamp just makes sense everywhere.

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 15:37:33 -0700

Lester Caine wrote:

> Loosing a bit of the first and last years is probably not a

Unfortunately losing data would be a problem, because it would mean 
clients getting data via tzdist would disagree with clients getting data 
via current means (typically operating-system distributions): the 
current clients get the data correctly but the tzdist clients would have 
screwups near the 32-bit boundaries.  These disagreements could cause 
problems, and are unnecessary if we design the protocol correctly.  This 
point was first raised in 
<http://www.ietf.org/mail-archive/web/tzdist/current/msg00302.html>.

> (1902,2038)?

For signed 32-bit hosts the full range is [1901-01-13 20:45:52Z, 
2038-01-19 03:14:08Z).  For unsigned 32-bit hosts (less common, but they 
do exist) the full range is [1970-01-01 00:00:00Z, 2106-02:07 
06:28:16Z).  In both cases I'm assuming POSIX time, which ignores leap 
seconds.

> The physical truncation due to 32

Sure, that could work.

---

From: Lester Caine <[email-masked]>
Date: Wed, 03 Sep 2014 00:13:57 +0100

> <http://www.ietf.org/mail-archive/web/tzdist/current/msg00302.html>.

No a 32bit tzdist client would only ever get 1902 to 2037, or 1970 to
2105 depending on how it is built? Limits that match the 64bit builds.
I agree that this is not necessary, but may be easier to manage at a
compatibility level? Rather than handling the error approach for the
missing bits?

>> inaccessible parts of the first or last years?

I'm just glad Firebird uses two 32 bit fields for date+time even on
32bit platforms. Gets around all these problems and how all my own data
has been stored since 1992 :) AND we still had 16bit machines back then.
Getting the same results from 64bit builds to the older 32bit kit is
important and some of the limits are avoidable other ways.

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 16:18:30 -0700

Cyrus Daboo wrote:> Hi Paul,
> One could argue the "mess" is caused by 32-bit time_t!

Yes, we should all switch to 64-bit time_t right away. :-)  Actually 
NetBSD has switched to 64-bit time_t even on 32-bit hosts, but most 
32-bit platforms have not and they'll be quite common for the next 
decade or two.

> truncation intervals? ...
>
> 3) Do we want the precision for the client requested ranges to be as low
> as one second (e.g., 1974-04-01T12:34:56Z)?
>
> Personally I don't like the idea of #3 as it requires more work on the
> part of the server and also potentially represents a denial of service
> vector for a malicious clients. #1 is a definite must. #2 is a nice to
> have.

I agree that #1 is a definite must because *all* practical servers must 
truncate their time data *somewhere*, and they should indicate the 
truncation boundaries.  If we assume #1, a common situation will be 
servers that support time stamps in the traditional signed 32-bit range 
[1901-01-13 20:45:52Z, 2038-01-19 03:14:08Z) along with clients that 
support the same range.  These servers cannot arbitrarily truncate their 
range to [1902, 2038) because that will mess up the clients.  And the 
servers cannot advertise [1901, 2039) because they don't support all 
time stamps in that range.  The only reasonable option is for these 
servers to advertise the time stamps that they actually support.  Hence 
server-advertised truncation ranges must not be limited to year boundaries.

I agree that #2 is nice to have but is not required.  A client can 
simply ask for whatever truncated data the server has precomputed.

#3 is similar to #1, in that the protocol should allow a server to 
advertise (say) (a) data truncated to signed 32-bit time_t, (b) data 
truncated to unsigned 32-bit time_t, (c) data truncated to the years 
2013 through 2016, (d) data truncated to the years 0000 through 9999, 
(e) whatever.  Many of these truncations will involve boundaries that 
are not January 1, and the tzdist protocol should support these boundaries.

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 16:27:36 -0700

> compatibility level? Rather than handling the error approach for the

Sorry, I'm having trouble parsing this comment; perhaps you meant to 
write "So" instead of "No"?

But yes, a 32-bit signed POSIX time_t client is interested only in time 
stamps in the range [1901-01-13 20:45:52Z, 2038-01-19 03:14:08Z). 
Presumably if the server advertises data truncated precisely to this 
range, the client will ask for that data.  Otherwise, the client will 
ask for a smallest advertised truncation range that contains the client 
range, and discard info that's outside the client's range.  If there 
isn't any such advertised range, I suppose the client will get whatever 
the server has, and either interpolate the missing eras, or report an 
error when asked to convert in the missing eras.

---

From: Lester Caine <[email-masked]>
Date: Wed, 3 Sep 2014 00:45:17 +0100

I meant 'No' ...
If you only serve up the full years that are complete, one can get away with the year rounding. Only advantage I see though is a perhaps a little easier documentation. Making the fine detail usable leaves out the resulting 'what the' when the missing dates are hit on the client.

Sent from my android device so the quoting is crap!

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 17:31:19 -0700

Lester Caine wrote:
> If you only serve up the full years that are complete, one can get away with the year rounding.

Sorry, I don't agree that one can "get away" with it.  It's not 
reasonable for (say) a tzdist-based Ubuntu client to have time stamps 
that differ from a conventional Ubuntu client that uses the exact same 
tz database.  If the tz databases are the same, the resulting time 
stamps should be the same too.  Otherwise provenance would go out the 
window, among other things.

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 02 Sep 2014 20:51:44 -0400

Hi Paul,

--On September 2, 2014 at 5:31:19 PM -0700 Paul Eggert <[email-masked]> 
wrote:

> that differ from a conventional Ubuntu client that uses the exact same tz
> database.  If the tz databases are the same, the resulting time stamps
> should be the same too.  Otherwise provenance would go out the window,
> among other things.

I wonder if we should support something like a "symbolic" truncation range, 
e.g. "truncation=posix-32bit" - that might make it a lot more obvious 
exactly what a non-integral year truncation range is all about. We would 
need a registry for such tokens that described the actual fully 
specified/precise UTC range.

-- 
Cyrus Daboo

---

From: Arthur David Olson <[email-masked]>
Date: Tue, 2 Sep 2014 21:02:24 -0400

> I wonder if we should support something like a "symbolic" truncation
range, e.g. "truncation=posix-32bit" - that might
> make it a lot more obvious exactly what a non-integral year truncation
range is all about.
> We would need a registry for such tokens that described the actual fully

The registry might be obviated by using a pair of time_t values. (for
example, -2147483648 and 2147483647).
This sacrifices (typical) human readability.

    @dashdashado

On Tue, Sep 2, 2014 at 8:51 PM, Cyrus Daboo <[email-masked]> wrote:

> I wonder if we should support something like a "symbolic" truncation
> range, e.g. "truncation=posix-32bit" - that might make it a lot more
> obvious exactly what a non-integral year truncation range is all about. We
> would need a registry for such tokens that described the actual fully
> _______________________________________________
> Tzdist mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tzdist
>

---

From: Paul Eggert <[email-masked]>
Date: Tue, 02 Sep 2014 19:14:10 -0700

Cyrus Daboo wrote:
> obvious exactly what a non-integral year truncation range is all about.

There could be a lot of potential symbols, including:

32-bit signed time_t, no leap seconds
32-bit signed time_t, with leap seconds (a widely supported extension)
same as above, but with pre-1970 data discarded.
32-bit unsigned time_t, no leap seconds
32-bit unsigned time_t, with leap seconds
35- or 36-bit version of the above (for Unisys 2200 series machines)
Microsoft Windows (1601 through past 9999)
OpenVMS timestamps (1858 through unknown)
DECdts timestamps (1582 through unknown)
64-bit unsigned time_t (1970 through past 9999)
etc., etc.

40- and 64-bit POSIX combinations would do the full 0000-9999 range, so 
there would be little point to distinguishing them, and I've omitted 
them from the above list.

Leap seconds would throw a monkey wrench into all this.  Each time a new 
leap second is announced, servers that support leap seconds and that do 
not go past the year 9999 would have to subtract one second from the 
upper bound of their supported range.  Which would mean that, strictly 
speaking, the number of symbolic names would grow every year or two, for 
each symbolic flavor that supports leap seconds and is limited. 
Alternatively the servers could arbitrarily reserve room for future leap 
seconds at the top of their supported range, which would be (there's no 
kind way to put it) a kludge.

This all feels like way overkill.  Isn't it simpler to characterize a 
supported range by its start and stop times?  Easier to explain, too.

---

From: Lester Caine <[email-masked]>
Date: Wed, 03 Sep 2014 08:50:29 +0100

> supported range by its start and stop times?  Easier to explain, too.

Totally agree Paul.
This is just like the 'use local time' question. There may be a small
gain but there are considerably more edge cases which I knew vaguely and
which you have now documented much better than I could have.

As you have nicely explained, while clipping to whole years may hide a
few of the discrepancies it IS at the expense of how the client sees
things. The only piece of this jigsaw that can be relied on IS the
client and only that can determine what it can actually work with. With
the lead second problem being a moving target as well.

Cyrus ... Paul and I do not get on over some points, but in this case,
he is far more knowledgeable given the code base that goes with tz. That
is key to making much of the tz data even work on many 'legacy' systems,
and we ARE stuck with having to support many in the short term. Clipping
to year boundaries may seem like a cheap fix, but has has been
demonstrated it then opens another can of worms :(

To try and reword what I was trying to work out ...
Some client systems may have limited capability, so the truncation of
data needs to be controlled by them. Limiting this to second accuracy
should be good enough, but if a server is incapable of providing a
suitable subset of data then it should not give any, and the client
needs to find a compatible source.

I was trying to investigate if we COULD use year rounding as a fallback,
but I think Paul ha managed to blow that out of the water?