Subject: Re: [Tzdist] Fwd: [tzdist] #32 (service): managing historical data

From: Daniel Migault <[email-masked]>
Date: Fri, 5 Dec 2014 22:35:06 +0100

Dear Friends and Colleague,

Please provide your opinion on adding a version number. I would like this
issue to be solved soon so we can have text in the next version.

BR,
Daniel

On Sat, Nov 29, 2014 at 4:20 PM, Lester Caine <[email-masked]> wrote:

> On 29/11/14 13:55, Daniel Migault wrote:
> > So I am trying to summarize the use we are addressing. Suppose we are
> > planning an event at date D0 using UTC as a reference, for people in
> > different time zone. This event is planned at date D1. For one of the
> > participant, its time zone has been updated between D0 and D1. This
> > updates results in having D1 (UTC) to move from D1_LOCAL_0 before the
> > change occurs to D1_LOCAL_1 after the change.
> >
> > If one receives the event indicating that D1 is D1_LOCAL_0. I should be
> > able to say, this local time has been derived from an tz that is not
> > relevant at D1, so so the actual event is at D1. Is that the or at least
> > a use case we intend to address?
>
> I need someone more 'proficient' with Islamic Time to fill in some of
> the fine detail, but the problem arises when a meeting scheduled for say
> 10AM local time in a day or so's time is published with a UTC time of X
> but then there is a revision to the local time due to the observed
> events which may start or stop Daylight Saving differently to expected
> so the local time moves and hour relative to UTC. That a video linkup
> will now happen differently was the problem I have had in practice, and
> the local time of the event SHOULD have been changed since the link was
> booked on the UTC time! Once you know the problem has arisen then there
> are several ways to resolve it but all involve change one time or another.
>
> The same problem will arise at short notice if a published DST
> transition does not ACTUALLY happen. We have had a couple of examples of
> this recently, and these might affect events longer into the future!
>
> > If so, using version could indicate which 'tz' has been used. For my
> > information how do we specify, the reference time? In our example, I
> > chose UTC, but, if we had chosen local time of the changing time zone,
> > then D1 would have been changed for almost all other time zones.
>
> The event is published using the current TZ data, and we have to publish
> the version information with the event data. When a delegate looks at
> the announcement for the event the local time advertised will be using a
> particular publishers set of TZID's and the current version of that
> data. By the time the delegate is looking at the event timetable, the
> latest TZ data may already be 'current+1' as so the data presented may
> be wrong. He then needs to know that for his particular TZID there is a
> discrepancy between the data he is looking at and the CURRENT situation
> on the ground. HOW the problem is to be fixed will depend on just what
> can be moved ... the local time activity retaining the UTC time, or does
> the UTC time need to change to reflect the new local time. *WE* can't
> answer that via tzdist, but by proper use of the version we can identify
> that there are two possible times now for this event and that the
> delegate needs to perhaps update their data for the event to match the
> LATEST version of the tz data or if no later version of event data is
> available - contact the organizers.
>
> > If the mentioned use case is correct, then we may have to look at:
> >     1) whether we have consensus that a version should be added
> I do not see how you can avoid a race condition without it?
>
> >     2) How version can be inserted in the URL model
> In order TO avoid the race condition a user must be able to access the
> data that matches that used to publish some event. Secondary to that
> this fixes all of the historic material since as new versions are
> published, all of the historic data is retained, and there may be
> several versions of tz between publishing an event calendar to the event
> happening ... changedsince can not be used to ascertain if any of the
> material has changed ... you either check out an updated version of the
> event calendar with a later version of tz - if available, or
> alternatively one can check if there is a difference between the version
> used to publish the data and the current data.
>
> >     3) What are the version speciifc operation. Maybe we should be able
> > to ask for the lastest version?
> What happens when the latest version has a short notice change in DST
> data which is not yet handled in the material one is looking at? In
> particular, LOCAL delegates may well be aware that there is some
> uncertainty when an event will happen, but the people using UTC time are
> generally totally unaware and in the current state of flux on many DST
> transitions, one does need to know that there has been a change rather
> than simply assuming that the 'latest' version is being used for the
> data we are looking at! The local time of an event may well change if a
> global link up has been booked based on UTC time.
>
> --
> Lester Caine - G8HFL
> -----------------------------
> Contact - http://lsces.co.uk/wiki/?page=contact
> L.S.Caine Electronic Services - http://lsces.co.uk
> EnquirySolve - http://enquirysolve.com/
> Model Engineers Digital Workshop - http://medw.co.uk
> Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
>
> _______________________________________________
> Tzdist mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tzdist
>

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Lester Caine <[email-masked]>
Date: Sat, 06 Dec 2014 00:39:51 +0000

On 05/12/14 21:35, Daniel Migault wrote:
> Please provide your opinion on adding a version number. I would like
> this issue to be solved soon so we can have text in the next version.

I'm still waiting for an explanation on how a version number can be avoided!

> On Sat, Nov 29, 2014 at 4:20 PM, Lester Caine <[email-masked]
> <mailto:[email-masked]>> wrote:
>

---

From: Stephen Colebourne <[email-masked]>
Date: Tue, 9 Dec 2014 22:44:10 +0000

Just to note that I am in favour of including an optional version
number in the spec using the previously discussed /versions/{version}
approach.

Stephen

---

From: Lester Caine <[email-masked]>
Date: Tue, 09 Dec 2014 22:56:06 +0000

> number in the spec using the previously discussed /versions/{version}

But if it is made optional, how do you ensure that you are using the
same data as another information source. Pray that both are published
with the same version?

---

From: Eliot Lear <[email-masked]>
Date: Wed, 10 Dec 2014 11:43:08 +0100

I would agree with Lester that this shouldn't be optional, but I also
don't know that one should be able to query a version.  I wonder why the
entire schema past the point of the name of the publisher should be
specified at all, since the semantics beyond that point will depend on
the structure of the information provided, and we are not in the
business of specifying that structure.

It is true that VCALENDAR is not a perfect match for response in all
cases.  This is where I think Accept: comes in, when it matters.

Eliot

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 10 Dec 2014 10:39:52 -0500

Hi Lester,

> same data as another information source. Pray that both are published

Today one might need to pray, but once tzdist is deployed and in use the 
goal is that users can rely on the data they have to be up to date and 
accurate (just as much as one can rely on an NTP server to give your device 
the correct time, or rely on the user having set the correct local time 
zone for the device etc).

We can't put a version property into the time zone data served by tzdist 
because that would mean that every time zone resource would change every 
time the overall database version is updated - even though only a small 
number of "real" changes took place. That would defeat the purpose of the 
"changedsince" behavior for the list action. Note, that if it is important 
for clients to know the version number, then that is available via the 
capabilities, or if the tzdist server supports alternate versions, via the 
/versions URI path behavior. That said, we could add some guidance/warnings 
about this:

    Until tzdist services are widely deployed and used, it is possible that
    users of calendar data may have different versions of time zone data on
    their devices, with those different versions having differing UTC
    offset and daylight saving time rules. This could lead to apparent
    differences in local or UTC times for meetings, potentially resulting
    in meetings being missed. For better reliability in such an
    environment, clients would need a way to ensure that all devices
    viewing a particular event are synchronized with regard to the same
    time zone data version for any time zones referenced by that event. One
    approach is to include the time zone data used for an event directly in
    that event (e.g., including an iCalendar "VTIMEZONE" component in the
    iCalendar object - the default behavior for iCalendar today).
    Alternatively, clients can include a URI that describes the source (and
    relevant version information) for the time zone data used by the
    creator of the meeting, allowing other users and devices to verify that
    the time zone data at that source matches what they have for the same
    time zone identifier.

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 17:26:33 +0000

On 10/12/14 15:39, Cyrus Daboo wrote:
> accurate (just as much as one can rely on an NTP server to give your
> device the correct time, or rely on the user having set the correct
> local time zone for the device etc).

This STILL misses the race hazard that exists with CURRENT data! First
it assumes that everybody s using the one published set of data. That
has been thrown out, so several sets will exist. Once one looks at a new
event diary one also has to establish just which publisher it is using!
One then gets the 'current' data for that publisher, but if the diary
was published today, and there has since been a change of data, then
what was used by the diary is out of sync.

OK one way of making your system work is to INSIST that ever published
dairy is always updated to the current data set for their publisher? Not
going to work, it takes time to find that what was published now needs
updating, and the material published may well already be cached around
the network, so an update will take time to propagate.

Historic material archived today will tag the publisher and the version,
and as long as in 10 years time the matching data can be recovered there
is not a problem. But the same simple process provides the necessary
security to CURRENT material since 'latest' may well not be the version
used five minutes ago ...

---

From: Tim Parenti <[email-masked]>
Date: Wed, 10 Dec 2014 12:32:47 -0500

On 10 December 2014 at 12:26, Lester Caine <[email-masked]> wrote:

> has been thrown out, so several sets will exist.
>

If some sort of identifier for the version is packaged with the response —
and I believe this is the intent of "changedsince" — then I don't see the
hazard.  If a particular publisher is slow to fetch and distribute
newly-available data, the old data they distribute will still be tagged as
such and obsoleted as normal once the updates become available from that
publisher.

Where I *do* see a potential race hazard is in defining exactly what value
"changedsince" takes on.  Is it the time at which a tzdist publisher
receives the new data, or begins distributing it, even if it was
technically available to them hours or days beforehand?  Or should this
date be fixed at the source (e.g., tz release timestamps) and propagate
somehow throughout the tzdist system?

--
Tim Parenti

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 10 Dec 2014 12:42:23 -0500

Hi Tim,

> somehow throughout the tzdist system?

No need. Users will congregate to tzdist servers that provide accurate and 
timely updates because those will ensure they don't miss meetings due to 
time zone data mismatches or update latencies. That is one of the key 
benefits of tzdist - it frees the user from depending on slow OS updates 
and gives them an independent path to a reliable system. Now maybe I am 
being far too optimistic in how I think tzdist will actually be used, but 
we are giving everyone the opportunity to have something they can rely on 
and that does not have to involve all kinds of "version reconciliation" 
between clients which would otherwise vastly complicate the clients 
themselves (and which I doubt most client authors will bother with given 
that they have typically not done so to date in the existing unreliable 
environment).

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 10 Dec 2014 12:50:24 -0500

Hi Lester,

> the network, so an update will take time to propagate.

As per my reply to Tim, users will opt to use tzdist servers that do 
provide timely and accurate updates if they start missing meetings because 
the ones they are using (or their reliance on just OS updates) fail them.

> used five minutes ago ...

So in ten years time the historians can just look at what data was current 
on the date of the event. They can also look and see if there was a time 
zone change prior to that time and use that to determine the accuracy of 
the actual time stamp (e.g., if the tzid being used was last changed one 
year prior to the event, then the probability of that event's actual time 
being correct is very high, if it was changed only a day or two before the 
event, then the probability is a lot lower, though if they know a reliable 
tzdist server was in use, then they can probably bump the probability up).

My key point here is that if version information really is important to the 
client, then they can insert that themselves from the data already provided 
by tzdist - e.g., include the actual URI for the tzdist data and the 
"primary-source" value from the /capabilities response, if the URI did not 
include /publisher or /versions segments.

-- 
Cyrus Daboo

---

From: Tim Parenti <[email-masked]>
Date: Wed, 10 Dec 2014 12:51:51 -0500

On 10 December 2014 at 12:42, Cyrus Daboo <[email-masked]> wrote:

> --On December 10, 2014 at 12:32:47 PM -0500 Tim Parenti <
> [email-masked]> wrote:
>
> and gives them an independent path to a reliable system.
>

I do agree that, given the intended use of tzdist and the intended speed of
update propagation, the potential hazard here is negligible.

> Now maybe I am being far too optimistic in how I think tzdist will
> actually be used, but we are giving everyone the opportunity to have
> something they can rely on and that does not have to involve all kinds of
> "version reconciliation" between clients which would otherwise vastly
> complicate the clients themselves (and which I doubt most client authors
> will bother with given that they have typically not done so to date in the
> existing unreliable environment).
>

I think this is a laudable goal.  That said, if we move toward allowing
arbitrary versions of data to be queried, I would see no harm in putting
the applicable version number in responses, to assist clients, and indeed
that would help to shift these kinds of (minor) issues away from the tzdist
protocol itself.  (I don't currently have a strong opinion on the larger
question.)

--
Tim Parenti

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 17:52:28 +0000

On 10/12/14 17:32, Tim Parenti wrote:
> Where I /do/ see a potential race hazard is in defining exactly what
> value "changedsince" takes on.  Is it the time at which a tzdist
> publisher receives the new data, or begins distributing it, even if it
> was technically available to them hours or days beforehand?  Or should
> this date be fixed at the source (e.g., tz release timestamps) and
> propagate somehow throughout the tzdist system?

It has been stated already that "changedsince" is managed by the server
and so even for the same publisher a different server will give a
different value. So that can't be used to identify WHICH "changedsince"
matrial was used by the information source since it is more than likely
using a different source? IF "changedsince" was the published date for a
version ... but it's being able to access an older version that is the
problem here.

Of cause if we WERE defining a single set of data with a single update
sequence then things would be a little easier, and personally I will
rely on TZ and it's well defined versions for my own source of data to
publish. But no doubt other sources will keep their own slightly
different views of things.

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 18:11:33 +0000

On 10/12/14 17:50, Cyrus Daboo wrote:
> provide timely and accurate updates if they start missing meetings
> because the ones they are using (or their reliance on just OS updates)
> fail them.

I am assuming that the only source is tzdist ... and by mirroring TZ
exactly as is I have accurate material to work with. Do you even accept
that there IS a providential race condition where data is published and
cached prior to a short notice change and when a user looks up this data
a few hours later there is the potential for miss information? It is
BECAUSE we have already had problems due to this very problem that I'm
trying to avoid it being an inherent flaw in anything replacing what we
currently use. Designing this race condition into tzdist is something
that strongly object to.

Probably 90% of users don't even need tzdist since their offset will
never change, but for some it is important to know that the data they
are looking at in a diary may well change tomorrow. Some of that
information is hidden in notes in TZ, but often it is not until a change
happens that we know that dates we had have a problem. That the diary
publisher should be flagged by a "changedsince" indication identifying a
problem allows the diary to be updated promptly, but that requires that
every cached copy also gets updated, and it's THAT part of the
distribution process that 'version' ensures the integrity off? Simply
assuming every copy matches is not a safe assumption?

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 18:34:03 +0000

On 10/12/14 17:51, Tim Parenti wrote:
> I do agree that, given the intended use of tzdist and the intended speed
> of update propagation, the potential hazard here is negligible.

Apparently a lot of ISP's cache pages when they read them, and do not
necessarily update them even if the source page changed. That caching
may even be the browser on your computer! The 'potential hazard' is far
from negligible!

---

From: Ken Murchison <[email-masked]>
Date: Wed, 10 Dec 2014 13:55:59 -0500

> may even be the browser on your computer! The 'potential hazard' is far

Section 4.1.4 gives recommendations on the interval for servers and 
clients to poll for updates.  If a client or server caches data for 
longer than this before updating and users start missing meetings, the 
users will stop using those clients and servers.  This is an 
implementation problem, not a protocol problem.

If short term changes to published TZ data occur within these polling 
windows, a version number won't solve the problem anyways. And even if 
the short term changes are published outside of this window, there is no 
guarantee that a primary server will be updated by an admin outside of 
the polling window, so again, a version doesn't solve the problem 
(unless you expect client to know what the expected version should be 
via some OOB mechanism).

Some of this depends on the definition of "short term change".  If 
"short term" means that new TZ data gets published less than a day prior 
to a change, then I don't see how tzdist can be expected to handle 
this.  If "short term" means a few days, then I would fully expect that 
well-behaved admins would update primary servers ASAP and that 
well-behaved secondary servers and client would gets updates during a 
polling interval that takes place prior to the impending time zone change.

-- 
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University

---

From: Doug Royer <[email-masked]>
Date: Wed, 10 Dec 2014 12:03:38 -0700

VTIMEZONE has a last-mod property. When that value changes, it should
invalidate the cache. It should be valid for version control.

Broken proxy servers seem out of scope and if they exist, a version ID
would not change anything.

-
Doug Royer
[email-masked]
(714)989-6135

On Wed, Dec 10, 2014 at 11:55 AM, Ken Murchison <[email-masked]>
wrote:

> clients to poll for updates.  If a client or server caches data for longer
> than this before updating and users start missing meetings, the users will
> stop using those clients and servers.  This is an implementation problem,
> not a protocol problem.
>
> windows, a version number won't solve the problem anyways. And even if the
> short term changes are published outside of this window, there is no
> guarantee that a primary server will be updated by an admin outside of the
> polling window, so again, a version doesn't solve the problem (unless you
> expect client to know what the expected version should be via some OOB
> mechanism).
>
> Some of this depends on the definition of "short term change".  If "short
> term" means that new TZ data gets published less than a day prior to a
> change, then I don't see how tzdist can be expected to handle this.  If
> "short term" means a few days, then I would fully expect that well-behaved
> admins would update primary servers ASAP and that well-behaved secondary
> servers and client would gets updates during a polling interval that takes
> place prior to the impending time zone change.
>
>
> --
> Kenneth Murchison

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 19:17:13 +0000

> implementation problem, not a protocol problem.
The protocol needs to be reliable and version is needed to plug this
hole, and also then solves the historic archive problem.

> (unless you expect client to know what the expected version should be
THAT is the point ...
When I publish a diary of events I have to add the name of the publisher
of the TZ data I've used along with the version of data. That is then
fixed and anybody accessing the diary can also access the TZ data used
to create it. If one of the archive services freezes a copy of the pages
then when ever they are looked at the user knows what data is needed to
view it. If a user has a copy of the diary cached locally, and then
finds that the 'latest' TZ version has changed they can easily assess if
it affects anything they are looking at.

> polling interval that takes place prior to the impending time zone change.
I am currently receiving diaries for next year published using todays TZ
version. I have a copy of each TZ version so can view the data, but if
some of that data changes in the new year by what mechanism do I check
that the iary has been updated on the source site? I check the version
of TZ used ... This has nothing to do with keeping a CURRENT set of data
up to date, but rather to ensure that the data being used matches that
used when material using dates and times normalized to UTC is published.

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 10 Dec 2014 14:22:55 -0500

Hi Lester,

> it affects anything they are looking at.

No! I don't want to send an event to attendees with a tzid that is fixed to 
a particular publisher or version, because otherwise, when the version 
changes and my client starts using the new version, I will have to re-send 
the event invitation with the new version detail to all the attendees (and 
I may have 1000's of events with 1000's of attendees) - that is exactly 
what we don't want to do (and exactly what happens today with iCalendar 
clients that are currently required to send the VTIMEZONE data in their 
invites). The current process is arguably broken - nearly all iCalendar 
clients throw away the VTIMEZONE data they receive from outside sources if 
the TZID matches one they already have locally. What we want to ensure is 
that clients can continue to do that and be able to rely on the fact that 
their local data is in sync with everyone else.

-- 
Cyrus Daboo

---

From: Ken Murchison <[email-masked]>
Date: Wed, 10 Dec 2014 14:32:50 -0500

Hi Lester,

> hole, and also then solves the historic archive problem.

Can you please explain how version plugs this hole?  I'm obviously 
missing your point.

> it affects anything they are looking at.

Maybe I'm just being thick, but why would I care what TZ data was used 
when events were published?  All I care about is what UTC offset was in 
effect at the exact instant of the event.  The most recent version of TZ 
data should always have the correct offsets, at least back as far as 
1970.  This should be the case regardless of the publisher.  If this 
isn't the case for TZ data moving forward, then people will stop using 
publishers of incorrect data.

If I invite multi-national colleagues to a conference call today using 
IANA 2014j TZ data, and at some point in the future I want to know when 
our call was, I should be able to use IANA 2020a or whatever to find 
out.  I shouldn't need to dig up a copy of 2014j.

The historic case that you mention (and possible use of incorrect data?) 
still seems like an extension to base tzdist to me.

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 20:02:46 +0000

On 10/12/14 19:22, Cyrus Daboo wrote:
> No! I don't want to send an event to attendees with a tzid that is fixed
> to a particular publisher or version, because otherwise, when the
> version changes and my client starts using the new version, I will have
> to re-send the event invitation with the new version detail to all the
> attendees (and I may have 1000's of events with 1000's of attendees) -
> that is exactly what we don't want to do (and exactly what happens today
> with iCalendar clients that are currently required to send the VTIMEZONE
> data in their invites). The current process is arguably broken - nearly
> all iCalendar clients throw away the VTIMEZONE data they receive from
> outside sources if the TZID matches one they already have locally. What
> we want to ensure is that clients can continue to do that and be able to
> rely on the fact that their local data is in sync with everyone else.

Cyrus ... I rest my case!

*I* want a single publisher and a single set of TZid's and I will be
using backzoneTZ to provide that data. But other users will probably not
have backzone so we have a broken system already. *YOU* will publish
events using your own publisher but your clients will have to use the
correct publisher to get the matching TZid's and data! Ideally these
will exactly match my data, but that is out of scope for the charter :(

If the details of an event change because someone has decided that what
was published is now wrong, then how do you identify the problem? Your
example is the exact explanation of the current problem, but how is your
current draft actually preventing the problem from happening? The event
invitation has to be updated to reflect the agreed change, but if I am
looking at the old version, and I see that there has been a variation
for this particular TZid then at least I know to ask.

---

From: Doug Royer <[email-masked]>
Date: Wed, 10 Dec 2014 13:11:04 -0700

> Maybe I'm just being thick, but why would I care what TZ data was used
when events were published?

Only when looking at old calendars. For the average user - no big deal.
Looking at some historical calendar - it might matter. (Like looking at a
DAYLIGHT/STANDARD VTIMEZONE before the state of AZ stopped using daylight
standard time - everything would be off by 1 hour for a few months of each
year before that date).

The VTIMEZONE component MAY contain multiple DAYLIGHT/STANDARD components
that DO serve this historical purpose. Over time their may be many
DAYLIGHT/STANDARD components that have non-overlapping time ranges, along
with last-mod property solving this problem without the need for an
additional version property.

-
Doug Royer
[email-masked]
(714)989-6135

On Wed, Dec 10, 2014 at 12:32 PM, Ken Murchison <[email-masked]>
wrote:

> Can you please explain how version plugs this hole?  I'm obviously missing
> your point.
>
>
>
> data should always have the correct offsets, at least back as far as 1970.
> This should be the case regardless of the publisher.  If this isn't the
> case for TZ data moving forward, then people will stop using publishers of
> incorrect data.
>
> IANA 2014j TZ data, and at some point in the future I want to know when our
> call was, I should be able to use IANA 2020a or whatever to find out.  I
> shouldn't need to dig up a copy of 2014j.
>

---

From: Doug Royer <[email-masked]>
Date: Wed, 10 Dec 2014 13:38:52 -0700

The organizer of an event specifies the timezone info. Even if it does not
match yours, its the time they will be hosting the event.

Why does it have to match another event from another organizer?

-
Doug Royer
[email-masked]
(714)989-6135

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 21:43:36 +0000

On 10/12/14 20:34, Ken Murchison wrote:
> Don't know if you intended to take this off-list, but I left it as such.
Should have been on list ;)

> On 12/10/2014 02:57 PM, Lester Caine wrote:
>> On 10/12/14 19:32, Ken Murchison wrote:
>> PLEASE read the various posts on the race problem with some TZ offset
>> changes!
> 
> I have, and I understand the race.

>> Yes a large majority of users will not even know there is a problem, but
>> for a small number of cases, DST offsets are changed at short notice due
>> to astronomical observations. So a meeting may move an hour at short
>> notice. 2014j becomes 2014k and the change currently slowly limps out so
>> no one knows until they log in at 10am to see then end of the meeting
>> because it started an hour earlier. My own example had video links
>> booked against UTC times, so the local meeting time had to change
>> retaing the UTC bookings. If the published timetable was tagged with
>> 2014j and tzdist is now saying 2014k a client can at least flag the
>> difference and warn that something has to change!
> 
> As I stated earlier, if a change is so short notice as to not be able to
> propagate prior to an affected meeting, then neither versions nor
> timestamps will solve the issue.  In this case, I would expect the
> user(s) in the affected time zone to be aware of such an impending
> change and coordinate the times manually with other participants.

It is getting a change out faster that is the key to tzdist actually
working. I can see some servers providing additional versions of TZ data
which are geared to support problem locations, but as well as the short
notice changes, longer time frame changes to previously published TZ
offsets also create a race condition.

>> Some DST events are changing almost annually, so a longer term event
>> calendar may well see an update in a years time. My own meeting
>> calendars run into 2016 already and even the UK DST can't be relied on
>> being retained.
> 
> Understood, which is why the actual UTC offsets should be calculated by
> the consumer with current TZ data near the time of an event.
> As Mike Douglass posted, this is why CalDAV clients and servers, store
> the TZID with an event.  As you note, pre-calculating UTC offsets is
> going to result in incorrect future times in some cases.

An international event will have meetings at different local times in
different countries. This is why normalizing to UTC is important, and it
IS the normalizing using the TZ data at the time of publication which
will be used to book video links, rooms and the rest. CURRENTLY when the
local time changes there is nothing flagging the problem and so problems
arise. Simply using a new set of TZ data is what causes the problem,
where as the system should report that there IS a problem at least. It
can't provide a fix, but that there IS a conflict at least allows the
organisers to see that and decide on a solution, which would normally be
to change the local time of the meeting leaving the UTC times
unaffected, that is if the facilities are available. Again, longer term
events may be affected if an unexpected change to DST rules are made for
political reasons. Assuming that you are looking at current times on
what may be a cached copy of the timetable creates a bigger window for
the race condition.

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 21:53:46 +0000

On 10/12/14 20:38, Doug Royer wrote:
> The organizer of an event specifies the timezone info. Even if it does
> not match yours, its the time they will be hosting the event.
> 
If they are organising linked meetings such as video conferencing then
it would be useful if they are using the same set of offsets?

I can keep adding examples of where a change to the local time of a
meeting may impinge on other activities in different time zones. Booking
flights is another problem if you have allowed an hour to get your
flight and the meeting is moved back an hour ...

If all you need is the local time then you probably have no interest in
UTC anyway which is why most systems could not even handle an event
which happens across several time zones. It's putting the system in
place that ensures that when booking a multi timezone event everybody is
using the same set of offsets.

---

From: Doug Royer <[email-masked]>
Date: Wed, 10 Dec 2014 15:06:34 -0700

You use the organizers TZ rules. If your user agent wants to map that into
your time, then great. But it still has to match what they will host.

Its not local time. Its organizers time.

-
Doug Royer
[email-masked]
(714)989-6135

---

From: Lester Caine <[email-masked]>
Date: Wed, 10 Dec 2014 23:42:14 +0000

On 10/12/14 22:06, Doug Royer wrote:
> You use the organizers TZ rules. If your user agent wants to map that
> into your time, then great. But it still has to match what they will host.
> 

Multiple time zone events are managed using UTC time with local
conversions to plan at each location. The local organisers schedule
around the UTC planned base. But this is irrelevant when it comes to
ensuring that what was published 6 months ago is still valid today.
Without the version process tzdist is just a waste of time. We will need
a replacement that actually works for archived data, be that historic or
current calendars, and events that may be affected by changes in offsets
in the future. Todays published data is tomorrows history and requires
access to todays normalizing data to ensure the date is not corrupted later.

---

From: Doug Royer <[email-masked]>
Date: Wed, 10 Dec 2014 17:13:47 -0700

That is one way to do that. However in this debate, if the organizer has a
different TZ definition than you, you can map it to and from UTC all you
want, you might not show up at the correct time.

Not all user agents store the data in UTC, many store the original event
and VTIMEZONE and map it as needed. If you do not use the TZ they specify,
you could show up, call, watch, whatever at technically the correct time,
but not when they are expecting you.

-
Doug Royer
[email-masked]
(714)989-6135

---

From: Lester Caine <[email-masked]>
Date: Thu, 11 Dec 2014 08:44:54 +0000

On 11/12/14 00:13, Doug Royer wrote:
> and VTIMEZONE and map it as needed. If you do not use the TZ they
> specify, you could show up, call, watch, whatever at technically the
> correct time, but not when they are expecting you.

Which is EXACTLY the problem that tzdist SHOULD be addressing. Just
perpetuating the current mess is simply not acceptable, and any draft
which does that should not be allowed forward.

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 09:47:12 -0700

> which does that should not be allowed forward.

Its not a problem unless you ignore what the organizer sends. Its more 
relevant that
everyone agree with the organizer, than with some other source.

  Yes to creating an authority. The authority and version information is 
useless if the organizer
uses another source.

-- 

Doug Royer - (http://K7DMR.us / http://DougRoyer.US)
[email-masked]
714-989-6135

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 11 Dec 2014 12:00:28 -0500

Hi Doug,

> everyone agree with the organizer, than with some other source.

Except that many clients DO ignore what the organizer sends - they just 
assume that if the TZID in the iCalendar data matches one they have 
locally, then the local definition is used in place of any sent in a 
VTIMEZONE. i.e., they throw away the organizer's copy of the VTIMEZONE in 
favor of there own.

One of the goals for tzdist is to ensure that when clients do that, they 
have a reasonable chance that their local time zone data matches what the 
organizer had (i.e., organizer and attendees are in sync wrt the VTIMEZONE 
data).

-- 
Cyrus Daboo

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 10:52:55 -0700

On 12/11/2014 10:00 AM, Cyrus Daboo wrote:
> Except that many clients DO ignore what the organizer sends - they 
> just assume that if the TZID in the iCalendar data matches one they 
> have locally, then the local definition is used in place of any sent 
> in a VTIMEZONE. i.e., they throw away the organizer's copy of the 
> VTIMEZONE in favor of there own.

Their code will have to change ether way or they will be just as broken.

They have to add code to fetch the correct organizers TZ data and 
recompute it themselves and hope they get it from a compatible source . 
Or they have to add/update code to use the organizers included data.

If the organizer is responsible for redistribution, then irrelevant or 
out of event time changes do not need to be fetched or recalculated by 
every CUA for every event every time the CUA starts. The organizer 
simply iTIP dispatches on relevant changes only.

> One of the goals for tzdist is to ensure that when clients do that, 
> they have a reasonable chance that their local time zone data matches 
> what the organizer had (i.e., organizer and attendees are in sync wrt 
> the VTIMEZONE data).
>

Yes, and and that is great separate point!!

When translating from organizer TZ to the local-TZ, the CUA can 
determine if it needs to get the latest local-TZ information for 
accurate results.

--

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 11 Dec 2014 13:11:07 -0500

Hi Doug,

--On December 11, 2014 at 10:52:55 AM -0700 Doug Royer 
<[email-masked]> wrote:

> every CUA for every event every time the CUA starts. The organizer simply
> iTIP dispatches on relevant changes only.

And how does an organizer know what is relevant? The organizer has no 
knowledge about which time zone a particular attendee might be in at the 
time of the meeting. If the attendee's time zone changes between when they 
accepted the meeting and the actual meeting time then that change is 
relevant to the attendee, but the organizer has no idea that is the case 
and so will not know to send out an update.

The point is that BOTH the organizer and attendee need to track changes to 
time zones so they can each decide what may or may not be relevant to each 
one of them. When they detect a change that may cause a problem, then yes 
they can use iTIP to communicate appropriate changes to the event. Just 
relying on the organizer to do that is not sufficient.

Note, that even what I describe above is problematic because few, if any, 
calendar clients actually have a concept of what time zone a user is likely 
to be in at some particular point in the future. Most clients allow users 
to change the "view" time zone to view events relative to a specific zone, 
but they don't let you say "next week I will be in London, so use 
Europe/London when viewing any of the days next week). Ultimately clients 
may do something like that if they need to fix these types of problem. (And 
as an aside, the proposed VAVAILABILITY component that is part of the 
calext WG, is one possible way to potentially indicate where one might be 
at some point in the future).

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Thu, 11 Dec 2014 18:28:03 +0000

On 11/12/14 18:11, Cyrus Daboo wrote:
> Note, that even what I describe above is problematic because few, if
> any, calendar clients actually have a concept of what time zone a user
> is likely to be in at some particular point in the future. Most clients
> allow users to change the "view" time zone to view events relative to a
> specific zone, but they don't let you say "next week I will be in
> London, so use Europe/London when viewing any of the days next week).
> Ultimately clients may do something like that if they need to fix these
> types of problem.

Calendaring systems still do not cover cross timezone activities well,
and I don't think iCalendar is particularly friendly in solving the
problems that need to be handled? For genealogical data we often flag a
time range covering the uncertainty in when an event happened so that
when time lining events one can see if something could have been
concurrent which can then can be researched in further detail. While
planning new events where a site may have a variation of local offset it
would be useful that the tzdist data actually flagged that there is
uncertainty in a forthcoming event if only to allow an organiser to
block bookings on that day or period. Often the organisers have no
understanding that these events may happen, so in addition to cleanly
publishing a short notice change, publishing a potential race problem is
an equally valid objective?

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 12:59:26 -0700

On 12/11/2014 11:11 AM, Cyrus Daboo wrote:
>> simply
> And how does an organizer know what is relevant?

Thank you - organizers CUA, not organizer. :-)

As in:  The CUA would know that the event is not in 2016 and ignore the 
TZ change, and not redistribute it because this specific TZ change is 
irrelevant.

> The organizer has no knowledge about which time zone a particular 
> attendee might be in at the time of the meeting. If the attendee's 
> time zone changes between when they accepted the meeting and the 
> actual meeting time then that change is relevant to the attendee, but 
> the organizer has no idea that is the case and so will not know to 
> send out an update.

Fetching your own TZ data is important. Updating your CUA with what you 
think the events TZ should be or should have been, is not the same subject.

Perhaps some kind of a 'do you really want that old TZ' message could be 
used. Both ways someone needs to notice/notify the other of an error. 
Without sending the organizers TZ, how would anyone know?

>
> The point is that BOTH the organizer and attendee need to track 
> changes to time zones so they can each decide what may or may not be 
> relevant to each one of them. When they detect a change that may cause 
> a problem, then yes they can use iTIP to communicate appropriate 
> changes to the event. Just relying on the organizer to do that is not 
> sufficient.

Yes, however the attendee needs to compare their latest TZ with what the 
organizer uses for TZ, even when the organizer is wrong. As that is when 
the organizer will have the event.

How does updating your view of what you think organizers TZ should be 
help? You would then be relying on the attendees to notice the organizer 
TZ is incorrect and informing the organizer manually. If no one notices 
or manually notifies the organizer (iTIP or phone) - you still are not 
on the same time slot. It could be that no one would have a clue until 
the event starts.

Both ways there may be an error. One way you break existing compliant 
CUAs. The other way you do not.

--

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 13:19:37 -0700

On 12/11/2014 11:28 AM, Lester Caine wrote:
> problems that need to be handled? For genealogical data we often flag 
> a time range covering the uncertainty in when an event happened so 
> that when time lining events one can see if something could have been 
> planning new events where a site may have a variation of local offset 
> it would be useful that the tzdist data actually flagged that there is 
> publishing a short notice change, publishing a potential race problem 
> is an equally valid objective? 

Is that not what BUSY-TENTATIVE is used for?
Or maybe PARTSTAT=TENTATIVE or STATUS=TENTATIVE ?
It would have to be up to the CUA/user/organizer to decide if they want 
to book one of those times.

As to TZ data, are you saying 'it may have been this time range' with 
respect to past time zone entries?

--

---

From: Lester Caine <[email-masked]>
Date: Thu, 11 Dec 2014 20:54:29 +0000

> on the same time slot. It could be that no one would have a clue until

That is exactly what happens today :(

Currently there is no reliable base to work off. As far as I am
concerned, different devices may well currently not have the same set of
offsets for a group of events. tzdist is at least a staring point in
providing a base that we can work from, and I would expect a single base
publisher to be used rather than the current disjointed standards.

For genealogical data it HAS to be a fully un-truncated set of material,
and this is an area where each version of TZ currently adds a growing
number of fixes to historic material - if that is actually included in
the published set.

For current calendars we now have a fairly reliable tracking of changes
happening via the TZ list, and in most cases while these changes have
not yet been published in a formal release, the information is available
to be passed down the chain. One would hope that a local venue might
update their clients that there is a problem with a booking but it
simply does not happen, so a more organised method is needed to make
people aware when something changes. It is a requirement that tzdist
provides timely notification of changes and there is little problem in
carrying that out, but USING that data is reliant on client systems
updating what they have published equally promptly. Using a simple
version tag allows every third party user to be advised if there is a
potential problem, and YES they may then ring the organisers to clarify
what is going to happen - and that may be the first the organiser knows
of the change! This can only happen if the third party knows that the
data is presented using the old version while there has been a change to
that data in the mean time. Be that a six month old calendar that has
now been messed up because subsequently a country has had a change of
government and DST has been changed drastically, or an astronomical
event has change the next weeks offsets from the initial assumption.

Does it matter if some data is wrong ... YES ... That the TZ data we are
using today may in fact be inaccurate is a fact, either going forward as
DST changes are currently assumptions, and going backwards as more and
more archival material is being uncovered to fill in the known gaps in
the backzone data. So we need to know what data was used at the time it
was used, and then we can review the material and update it as required.
This currently fails simply because we now have no idea just what data
was used to build the original data set.

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 11 Dec 2014 16:01:16 -0500

Hi Doug,

> Without sending the organizers TZ, how would anyone know?

A key benefit of tzdist is to avoid the need to send around a whole bunch 
of messages trying to "negotiate" the proper time zone. Clients should 
simply not have to do that. As it is clients should not have to include 
VTIMEZONE objects in iTIP messages - its a waste of bandwidth and storage 
(and it is exactly why we have draft-ietf-tzdist-caldav-timezone-ref which 
does do away with VTIMEZONEs in the data exchanged between clients and 
servers).

Having everyone agree to use a reliable and timely tzdist service 
eliminates many (not all) of the big issues that exist today. Yes, clients 
will still need to be smarter about dealing with time zone changes when 
they occur - and that is what this debate needs to stay focussed on. What 
information does tzdist need to provide in the data it provides to ensure 
that clients have the necessary information to do more advanced 
reconciliation of events when a time zone change occurs.

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Thu, 11 Dec 2014 21:07:10 +0000

> to book one of those times.

I don't use iCalendar which is why I would prefer a clean TZ dataset
rather than one that is processed to what I still view as a non-standard
format for other calendaring systems. So I'm looking at the information
being transferred rather than a single usage. So I'd welcome any input
on just how some of these things could be presented.

> respect to past time zone entries?
For the historic material that is currently relegated to backzone that
may well be the case, but for current data TZ publishes the 'best guess'
for the next transition on many TZid's and the notes advise what event
may change that guess. I've covered this some time back in that we could
add comments to an offset to indicate that a transition may change at
short notice. Currently there is no mechanism in TZ to flag that there
are two potential values for a period but it is something that at the
calendaring level would be useful to display.

---

From: Lester Caine <[email-masked]>
Date: Thu, 11 Dec 2014 21:32:02 +0000

On 11/12/14 21:01, Cyrus Daboo wrote:
> eliminates many (not all) of the big issues that exist today. Yes,
> clients will still need to be smarter about dealing with time zone
> changes when they occur - and that is what this debate needs to stay
> focussed on. What information does tzdist need to provide in the data it
> provides to ensure that clients have the necessary information to do
> more advanced reconciliation of events when a time zone change occurs.

I feel the case has been made for the ability to manage each published
version of a published data set? Using the version tagging. That a
server only needs to maintain versions for those TZid's which have
changed keeps the volume of data down. Probably 90% of data will never
change and so the very first version will still be used to provide that
material. Many updates will only be perhaps three or four TZid's and
potentially clients may never even read those as they are out of scope
for their calendars.

As new material is published using the current data, the correct version
can be tagged, so an archive can maintain a consistent view of it's
content. SO that only needs a server which is able to provide the
versioned views. As far as I am concerned, genealogical data needs this
facility and so has to use a server capable of it - with a backzoneTZ
base probably complimented by a historic location mapping database as well.

The problem is if event calendars published today need this versioning?
If they only contain data that is never likely to be part of a new
version, then obviously not, but it's what level of 'error' is
acceptable for material which is subsequently retained in archives and
used later in other cross references. That a meeting time is an hour
adrift may not matter so we can simply say 'tough' and ignore
discrepancies, but I'd still like a better explanation on how one
ensures that when a change in offsets are applied which invalidates
existing forthcoming events, this is notified without the use of a version?

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 15:00:58 -0700

> That is exactly what happens today :(
Compliant CUAs use the organizers TZ data that is shipped with the 
event. They do not go get one from their complied internal code or from 
some other source. Even when it is out of date, it may be what the 
organizer intended. Everyone sees the same time slot, correct or 
incorrect. Then they can debate or COUNTER the time.

NON-Complaint CUAs use random versions of TZ data.  This is the fix that 
needs to be done. To their code. Attendees and organizers might not see 
the same time slot, and they do not know because they do not compare it 
to any organizer data. Removing the TZ data or ignoring it sent from 
compliant CUAs causes the same problem because you would not know it was 
incorrect.

Breaking compliant CUAs will still not fix non-compliant CUAs.

A TZ database is needed. There is no reason to break existing good code.

And how would the organizer be informed they have old TZ data if it is 
not sent with the event? You have only ensured that only the organizer 
will be on time to the event on TZ changes. No one would know what 
'version' of TZ data was intended as it would no longer be sent and 
attendees might show up and hour early or late.

If all CUAs were compliant, the organizers CUA would get the correct TZ 
data. It would be forwarded to attendees CUAs as the version to use as 
it is document in iCal and iTIP. The attendee CUAs get the correct TZ 
data for their TZ. All works. Upon TZ updates, the organizer CUA would 
re-distribute an update. As it is now with compliant CUAs.

Current compliant implementations: If a organizer CUA does not see the 
update and an attendee does, there is currently no way to notify the 
organizer.  Messed up. Possibly the attendee CUA could see the change 
and send a COUNTER.

With the proposal, if the organizer CUA does not see the update and an 
attendee does, The attendee still has no way to notify the organizer 
*and* no way to know the problem happened.  The attendee CUA would not 
know that the organizer CUA did not see the change - nothing will be 
done - people show up at incorrect time. Still messed up.

The problem has not been solved when non-compliant CUAs are used. They 
must update their code.

The problem has not been solved when one or the other CUA misses a TZ 
update.

*Please* fix my flow below if I misunderstood the proposed methods:

For a PUBLISH event:

     Using the existing method:

         Only one CUA is really in play as other CUAs must periodically 
check for updates
         (REFRESH or reload a file).

         If the organizer CUA does not see the TZ update or update the 
TZ data in the event,
         things are busted and there is no documented way to solve the 
problem.

         Attendee CUAs could detect there may be a problem because the 
TZ data is different.

         If an attendee CUA ignores the old TZ data in the event, they 
may or might not be on time.

         Existing CUA PUBLISHed events will are compliant with the new 
methods.

         Existing compliant CUAs need no code changes to work now.

         As CUAs are updated, they can add code to check servers for new 
TZ data.

     Using proposed method:

        Only one CUA is really in play as other CUAs must periodically 
check for updates
         (REFRESH or reload a file).

         If the organizer CUA does not see the TZ update and the TZ data 
is NOT in the event,
         things are busted and there is no documented way to solve the 
problem.

         Attendees could NOT detect their may be a problem because there 
is nothing to compare to.

         Attendees would have no clue if the organizer scheduled the 
event at an incorrect UTC time
         or correct UTC time. They MUST assume the organizer CUA used a 
correct TZ version.

         Existing CUA PUBLISHed events will not be compliant with the 
new methods.

         All CUAs need to be re-written to comply with pulling TZ data 
for organizer TZ
         in addition to them already pulling TZ data for their local zone.

         All CUAs must re-check their TZ data for each event TZ used, 
each time they start up
         or when they view a calendar data/time range. Much more traffic.

For scheduled events - REQUEST:

     Using the existing method:

         The organizer sends out a REQUEST with TZ data.

         Each time there is a change, the organizer CUA sends  a new 
REQUEST also with any
         new TZ data.

         The attendee CUA process the request, does TZ -> UTC -> Local 
TZ view for user.
         Any iTIP negotiation takes place.

         There is no way for the attendee CUA to inform the organizer 
CUA they have an
         old view of TZ data. (perhaps a COUNTER will work ? Have not 
tried this)

         Existing compliant CUAs need no code changes to work now.

         As CUAs are updated, they can add code to check servers for new 
TZ data.

     Using the proposed method:

         The organizer sends out a REQUEST without TZ data.

         Each time there is a change, the organizer CUA sends  a new 
REQUEST also without any
         new TZ data.

         The attendee CUA process the request, does TZ -> UTC -> Local 
TZ view for user.
         Any iTIP negotiation takes place.

         There is no way for the attendee CUA to inform the organizer 
CUA they have an
         old view of TZ data AND because it is not detectable any more.

         All CUAs need to be re-written to comply with pulling TZ data 
for the organizers TZ
         in addition to them already pulling TZ data for their local zone.

         All CUAs must re-check their TZ data for each event TZ used, 
each time they start up
         or when they view a calendar data/time range. Much more traffic.

--

---

From: Doug Royer <[email-masked]>
Date: Thu, 11 Dec 2014 15:16:09 -0700

On 12/11/2014 02:01 PM, Cyrus Daboo wrote:
> A key benefit of tzdist is to avoid the need to send around a whole 
> bunch of messages trying to "negotiate" the proper time zone. Clients 
> should simply not have to do that. As it is clients should not have to 
> include VTIMEZONE objects in iTIP messages - its a waste of bandwidth 
> and storage (and it is exactly why we have 
> draft-ietf-tzdist-caldav-timezone-ref which does do away with 
> VTIMEZONEs in the data exchanged between clients and servers).

There is no negotiate involved in ether method, if one is out of date in 
both models, things are busted.

It gives the illusion of not being needed. But offline CUAs exist, 
limited connectivity CUAs exist, and non-compliant CUAs will still 
exist. How will you tell if nothing is sent?

And it BREAKS ALL currently compliant CUAs.

I don't use caldav. The issue belongs in the base iCal/iTIP methods then 
it works for all transports. Most of my events come from MIME objects in 
email, some from URLs I download into my calendar. Other than testing, I 
have never gotten a caldav event.

Would the proposed method still send TZ data in email and file/url 
PUBLISH methods? if not you are proposing two different iCal object 
sets, caldav and non-caldav. That seems a bigger red flag.

> focussed on. What information does tzdist need to provide in the data 
> it provides to ensure that clients have the necessary information to 
> do more advanced reconciliation of events when a time zone change occurs.
>

I disagree with it solving the error issue. It hides those issues and 
breaks existing code.

The issue it solves is with creating new or updating events. And for 
local time zones.

--

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 01:55:47 +0000

>         (REFRESH or reload a file).
Please point me to some RFC for 'CUA' ... all may calendar material is
published using TZ as distributed via PHP, and it is ensuring that
clients have the same version of TZ that I'm working towards currently.

>         If the organizer CUA does not see the TZ update or update the TZ
> data in the event,
>         things are busted and there is no documented way to solve the problem.

VTIMEZONE has a 'last-modified' entry which would match the timestamp of
a published version of TZ data. As has been said, the vast majority of
published events that timestamp would be 'day one' and offsets will
never change. It is only a small number of events that may be a problem
and even without tzdist we can document a way to manage that problem.

>         Attendee CUAs could detect there may be a problem because the TZ
> data is different.

I'm having trouble working out how an Attendee identifies the TZid being
used looking at iTIP spec. Like many older standards, the basic
functions had timezone information patched in and are not ideal for
planing events that happen concurrently in several locations, but I am
sure that if the Attendee has the right information, then the version
from 'last-modified' will allow checking that an unpublished change may
apply. Just as re-coding TZ information into VTIMEZONE format looses
some useful material, other conversions seem to be going on here, and
I'm sure the same comparisons can be made in this application area again
even without tzdist as a distribution method.

> may or might not be on time.

If the event is published as a local time only and they arrive at the
local time then things are probably OK. The problem is not with
individual events, but with ones where the same event is also happening
in another timezone and SHOULD happen at the same time. THAT is what
iCalendar does not handle safely currently? Planning based on a UTC time
base with only displayed information showing a local view is not
something that seemed to work using iCalendar processes? But it was a
good few years back I ruled out iCalendar as a base for UTC normalized
event planning. At that time for transport timetables cross Europe.

>         Existing CUA PUBLISHed events will are compliant with the new methods.
> 
>         As CUAs are updated, they can add code to check servers for new TZ data.

I feel we need to properly identify the various holes in the current
processes and either accept they will not be fixed, or agree a process
which if used can prevent them happening. If people do not want to use
that process then fine. Like the useless time offset provided in a
browser packet, some things have been broken a long time so why bother
fixing them?

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 11 Dec 2014 21:31:56 -0500

Hi Doug,

--On December 11, 2014 at 3:16:09 PM -0700 Doug Royer 
<[email-masked]> wrote:

> limited connectivity CUAs exist, and non-compliant CUAs will still exist.
> How will you tell if nothing is sent?
>

No - the whole point is most CUAs use their own built-in time zone database 
- typically those provided by the system apis they use. That is certainly 
the case for the majority of CalDAV clients. Most of those clients simply 
throw away any VTIMEZONE in the in-coming data if the TZID matches one they 
have internally. Some will even try matching the incoming TZID with one in 
their local cache by expanding observances and trying to find the best 
match. All of those clients could be termed "offline" clients.

I am not proposing changing iTIP right now - even though I suspect many 
clients will work fine without them. The goal is to get tzdist deployed and 
in use and then eventually change iTIP.

-- 
Cyrus Daboo

---

From: Mike Douglass <[email-masked]>
Date: Thu, 11 Dec 2014 23:17:31 -0500

On 12/11/2014 05:16 PM, Doug Royer wrote:
>> should simply not have to do that. As it is clients should not have 
>> to include VTIMEZONE objects in iTIP messages - its a waste of 
>> bandwidth and storage (and it is exactly why we have 
> There is no negotiate involved in ether method, if one is out of date 
> in both models, things are busted.
>
> I don't use caldav. The issue belongs in the base iCal/iTIP methods 
> then it works for all transports. Most of my events come from MIME 
> objects in email, some from URLs I download into my calendar. Other 
> than testing, I have never gotten a caldav event.
>
It already appears to be the case that most server and client 
implementations - caldav or non-caldav - don't trust the data sent to 
them via iTip. That information is often truncated and/or out of date.

We managed to get away with it in the US because the rules changed so 
infrequently. Also there were many fewer centralized calendar services.

By providing an easily accessible and frequently updated timezone source 
hopefully we can reduce the errors.

At the moment the vtimezone information sent with iTip is mostly 
(expensive) noise and we (server and client implementors) treat it as such.

We ran tests on a number of current clients and servers - both through 
CalDAV and simply importing events - and dropping the VTIMEZONE 
information worked fine.

If an organizer sends an event with out of date tz rules I'm not going 
to present that to an attendee as if it is correct.

I'd argue that the iCalendar spec is deficient in that it ought to send 
the local time AND the UTC time as calculated by the organizers and 
attendees - that would give us all an opportunity to check the 
calculation using our (recently fetched) tz data. No version is needed - 
nor do we have to derive our tz rules from the same source. They just 
have to agree. That though is an extension to iCalendar and not 
something for tzdist.

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 08:26:58 +0000

> have to agree. That though is an extension to iCalendar and not

I thought we had previously established that while there are
deficiencies in a number of current processes, VTIMEZONE could be ade
more reliable with the currently planned additions? That users then need
to be persuaded that this is now the right way to work is another matter.

Please can someone indicate how the iCalender mechanisms currently
handle meetings across multiple locations in different time zones? I've
seen linked events, but these normally seem to be manually generated?

---

From: Martin Burnicki <[email-masked]>
Date: Fri, 12 Dec 2014 11:24:13 +0100

> hopefully we can reduce the errors.

I absolutely agree, and I'm trying to summarize some cases.

Maybe we should look at a number of different cases which should be 
handled greatfully. Let's assume an appointment is to be made in 
Morocco, or Egypt, where we have seen that decisions about start or end 
of DST can be published only 3 days before this actually happens.

A local event is planned for a specific local time. If the DST rules 
change before the event actually happens the organizer (the person) has 
to decide whether the UTC start time should be kept the same, and thus 
the local start time should change, or if the local start time should be 
kept unchanged and thus the UTC start time changes.

In my opinion the usual case would be to keep the local start time 
unchanged to keep the relative time (e.g. "after lunch").

For international events basically the same facts apply. If the TZ rule 
changes for the organizer, the organizer has to decide whether he would 
keep his local time, and thus all other participant in different time 
zones had to change or update their start times, given in local time 
according to their selected time zone, or if the organizer prefers to 
keep the "international" time and thus nothing changes for the 
international participants.

Obviously the decision whether to change the UTC start time or the local 
time depends on the organizer.

If the organizer is an airline and the event is a flight then I'd expect 
the UTC start time is kept unchanged.

If it's a phone conference if may depend on the how attentive the 
organizer is. If he's  the boss and the other ones are employees he may 
insist on keeping his local time. If the participants are like partners 
he might want to reduce the overall effort, keep the international start 
time and adjust only his local time.

If you are going to go to the cinema in the country where the TZ rule 
changes then surely the local time when the film starts will be kept the 
same, even if the TZ rules change.

So obviously there is one point which can't be handled by a protocol 
like tzdist, i.e., which decision is made by the organizer.

On the other hand, regardless which decision is made by the organizer, 
the tzdist protocol should be able to help getting this handled as 
smoothly as possible in each of these cases.

Personally I would find it most helpful if I (my smartphone, or my PC) 
receive a notification if something in the time zone I've selected 
changes. Hoewever, of course this is not part of the protocol but of the 
client implementation.

Martin

---

From: Eliot Lear <[email-masked]>
Date: Fri, 12 Dec 2014 12:05:43 +0100

On 12/11/14, 7:28 PM, Lester Caine wrote:

> problems that need to be handled

There are some limitations that are inherent, when dealing with
timezones, but they are relatively few (like handling recurring meetings
during transitions).  None of that is relevant here.  Complaints about
vendor products are best dealt with in the market place and not in the
working group, unless there is a specific relevant interoperability
concern that impacts the protocol we are developing.

> For genealogical data we often flag a
> block bookings on that day or period.

Variations in local offset are not always predictable.  Moreover, there
is an interaction with recurring events that would make such a feature
virtually impossible to get right.

Eliot

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 11:15:28 +0000

> smoothly as possible in each of these cases.

The tzdist process is chartered to provide prompt updates of TZ data and
that is all it needs to do. The result of a variation in that
information is not something that can always be handled automatically.
The best we can do is flag a problem if it arises. Human intervention
will be required as there is no way of of guessing which is the best fix
to a problem.

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 11:27:48 +0000

> concern that impacts the protocol we are developing.

Using the process that I envisage is a matter for individual products. I
am hoping that what is currently developing is an understanding by
others that 'version' provides a clean solution to all various problems.
I'm not interested in any particular implementation - it's not relevant
- only ensuring that users of tzdist have a chance of flagging
complications that way arise even if that means manual actions.

> virtually impossible to get right.

This is the very situation I'm trying to cover. Schedules are published
for the coming year and often roll on each month. Simply tagging that
schedule with the current TZ version ensures that the current view can
be retained. When an update of version on tzdist happens one can
identify where a change to that view is causing a problem. It is not up
to tzdist to FIX the problem ... simply identify that there is a problem
to fix. Assuming that the information is always correct is not a
solution to the race problem?

---

From: Doug Royer <[email-masked]>
Date: Fri, 12 Dec 2014 11:22:26 -0700

On 12/11/2014 07:31 PM, Cyrus Daboo wrote:
> No - the whole point is most CUAs use their own built-in time zone 
> database - typically those provided by the system apis they use. That 
> is certainly the case for the majority of CalDAV clients. Most of 
> those clients simply throw away any VTIMEZONE in the in-coming data if 
> the TZID matches one they have internally. Some will even try matching 
> the incoming TZID with one in their local cache by expanding 
> observances and trying to find the best match. All of those clients 
> could be termed "offline" clients.
>
> I am not proposing changing iTIP right now - even though I suspect 
> many clients will work fine without them. The goal is to get tzdist 
> deployed and in use and then eventually change iTIP.
>

     Are you saying that CALDAV compliant CUA implementations are 
already REQUIRED to ignore the VTIMEZONE included?

         If YES, I see your point.
         If NO, then your breaking them - correct?

     Or are you saying that *ALL* CALDAV implementations currently 
ignore the VTIMEZONE (as in they are all broke) so the point is mute ?

--

---

From: Doug Royer <[email-masked]>
Date: Fri, 12 Dec 2014 11:48:36 -0700

> clients have the same version of TZ that I'm working towards currently.

CUA - Calendar User Agent: RFC 5545.

>>          things are busted and there is no documented way to solve the problem
If your *not* sending the VTIMEZONE with the VEVENT,  then there is no 
last-modified to see or compare to. Your just guessing and hoping the 
other end is using the latest version.

Many of my customers have off-line secure systems and the updates to the 
TZ database will be manual. An outsize URL *only* is useless to me. I am 
sure I am not the only one that will need to work with mostly off-line 
systems.

They can get email, so when I talk calendaring I think iTIP and want to 
make sure the data TZ data that the organizer intended is included.

Currently RFC 5546 includes VTIMEZONE. I have never used CALDAV, I had 
no idea it was non-iTIP compliant.
Is CALDAV iTIP compliant? Or are all CALDAV implementations just 
non-compliant?

--

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 12 Dec 2014 13:50:54 -0500

Hi Doug,

--On December 12, 2014 at 11:22:26 AM -0700 Doug Royer 
<[email-masked]> wrote:

>      Are you saying that CALDAV compliant CUA implementations are already
> REQUIRED to ignore the VTIMEZONE included?
>
>      Or are you saying that *ALL* CALDAV implementations currently ignore
> the VTIMEZONE (as in they are all broke) so the point is mute ?

This is all explained in the current draft: 
draft-ietf-tzdist-caldav-timezone-ref-00. Nothing currently requires 
clients to ignore time zones. The proposed extension will allow that in 
some circumstances.

Strictly speaking clients are not "ignoring" the time zone - they are 
matching the TZID with ones in their local cache and using the data in 
their local cache in place of the VTIMEZONE that was sent. As Mike has 
pointed out that is often a necessity since the transmitted VTIMEZONE may 
not be complete (only covers a very small range of time as applicable to 
the event it is attached to).

As the draft explains, we did a lot of interop testing and discovered that 
server could unilaterally decide to not return VTIMEZONE data in iCalendar 
objects read by the client (at least for those with "well known" TZIDs). 
Nothing broken as a result of that. The spec also makes it clear that this 
"timezones-by-reference" approach is only scoped to CalDAV interactions and 
does not apply to other protocols exchanging calendar (i.e., iTIP or iMIP).

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 21:21:46 +0000

> other end is using the latest version.

All that is needed with the event is a version tag ... a single extra
element. This then assumes VTIMEZONE is populated by other means - such
as tzdist :)

> sure I am not the only one that will need to work with mostly off-line

This is precisely what is happening with the bulk of the  historical
archive material. Once one has a version tag, everything can be
processed off-line. The only reason for accessing the on-line service is
to establish if the latest version has changed from your off-line cache,
and you only need to do anything if that update affects a TZid that is
currently being used by you. If all your material is in an area where
there is no active change from year to year then essentially nothing
changes for you. If you find that a problem has arisen tzdist provides
an indication of the problem, but humans may have to decide what changes
are required. My MAIN reason for wanting a managed version tracking
process is exactly because I may well be working off-line for hours
while travelling, and then using a different tzdist server at my
destination ... which has to mirror the data I already have cached and
will advise if I need to update the cached material. A system reliant on
a permanent synchronised update process does not work especially if
'changedsince' is not consistent between different servers. One can't
assume that the original server continues to be accessible when moving
between different access mechanisms?

---

From: Daniel Migault <[email-masked]>
Date: Fri, 12 Dec 2014 22:22:27 +0100

Hi,

I am incline to understand that tagging tzid with version solves this issue
has almost reached consensus. Now we should move forward to close this
issue and agree on whether the version is optional with a default value and
where the version is indicated.

BR,
Daniel

On Fri, Dec 12, 2014 at 7:50 PM, Cyrus Daboo <[email-masked]> wrote:
>
> Hi Doug,
>
> --On December 12, 2014 at 11:22:26 AM -0700 Doug Royer <
> [email-masked]> wrote:
>
> This is all explained in the current draft: draft-ietf-tzdist-caldav-timezone-ref-00.
> Nothing currently requires clients to ignore time zones. The proposed
> extension will allow that in some circumstances.
>

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Lester Caine <[email-masked]>
Date: Fri, 12 Dec 2014 21:50:19 +0000

On 12/12/14 21:22, Daniel Migault wrote:
> I am incline to understand that tagging tzid with version solves this
> issue has almost reached consensus. Now we should move forward to close
> this issue and agree on whether the version is optional with a default
> value and where the version is indicated.

The sticking point seems to be an acceptance that versions should be a
required since in addition to providing a solution to managing
historical data it also removes the race hazard in handling current
material. Once the mechanism is documented is there any reason to leave
an inherent race problem in the main distribution process? It may be
optional that a server provides the full range of versions however I
still feel this is only permissible if the protocol is documented
explaining the inherent race problem in the current sync process.

-----------------
I have a problem with the proposed conference call next week since I
will be driving down to London at that time, ready for a flight out to
Malta on the 19th. I think that we do have an understanding on why
managed versions are required for consistent operation and the agreement
is just required as to IF the race problem is something that is to be
avoided in the first draft or will have to come later? I think we had
already got agreement that anything done in day one has to reliably
support material published using tzdist, for which version is a
required. So it can't easily be optional in other processes?

---

From: Doug Royer <[email-masked]>
Date: Fri, 12 Dec 2014 15:22:52 -0700

> element. This then assumes VTIMEZONE is populated by other means - such

For those of you that have been very active on this list for a while
  - I applaud you patience as I try to understand the draft for the 
first time.
   Thank You All!

(Not talking CALDAV - talking the process). It will not do any good if 
there is no way to
refer to everything that is needed and I am going through the process 
step by step
in my head to make sure I have everything I need.

I agree that TZID is not sufficient unless it contains source 
information about the source
of the TZID. TZURL is in 5545 and it seems a logical usage for tzdist, 
yet I can not find it
in the draft-service or draft-caldav. (maybe this is to be covered in a 
later draft?)

Was it the intention that TZURL be mandated in events when not including 
VTIMEZONE
so you can find the TZID definition used by the originator? In some 
later draft?

For version, how about add a parameter to TZURL that contains the 
last-modified value
from that URL as the version?

TZURL;LAST-MODIFIED=20121212T201329Z:http://zones.example.com/tz/America-New_York.ics

*Or* another way it could be done is to add it as a parameter to the URL:

TZURL:http//zones.example.com/tz/America-New_York.ics?last-modified=20121212T201329Z

The URL parameter would not require an update to the TZURL ABNF.

Then at least you know the VTIMEZONE the organizer intended, the source 
of the TZID
and their version. It allows for checking against your cache. And it 
allows for some
out of band discussion about the organizer not having the newest version 
of the TZ data.

When checking against your cache, you check against the URL host+path 
and last-modified
in your cache. You can always fetch the newest version from the URL to 
determine if there have
been any TZ data updates.

The TZURL would also allow for disambiguation of TZID values as described
in draft-service, section 3.5

The HTTP GET could be modified to allow for fetching by last-modified.
Would this ever need to be done? It would allow for anyone to get the 
version
the organizer was using. Or is it sufficient to know they (or you or 
both) are
out of date and renegotiate the meeting times?

     GET /zones/America%2FNew_York
                    ?last-modified=20121212T201329Z

> ... My MAIN reason for wanting a managed version tracking
> destination ...

Yes!

--

---

From: Doug Royer <[email-masked]>
Date: Fri, 12 Dec 2014 18:13:00 -0700

On 12/11/2014 02:01 PM, Cyrus Daboo wrote:

> ...its a waste of bandwidth and storage (and it is exactly why we have 
> VTIMEZONEs in the data exchanged between clients and servers). ...

I do agree tzdist is the best way!

Transitionally, CUAs that are currently compliant are going to send them 
and some will choke if they are not included. The WebEx meeting to this 
working group is an example. It send the VTIMEZONE and everyone that 
uses it will call in at the same UTC time. Assuming their CUA uses it :-)

Replacing it with TZID + TZURL + some version info is also acceptable if 
that is the direction to move. Just don't ban VTIMEZONE as email (iTIP) 
may be the only way to get appointment information on Internet 
restricted systems.

In the spirit of John Postel and his comments on accept and generate 
grammar:

How about text for a new tzdist compliant CUA generating an iCal object 
(I still think last-modified == version info):

         (A) When generating an iCal object, the CUA MUST include TZID + 
TZURL + version info, MAY include VTIMEZONE.

And accept for new CUAs:

        (B)  If TZURL + version info is included then it MUST BE used 
and the CUA MUST ignore any included VTIMEZONE.
               If the TZURL turns out not to be a tzdist server, then 
the included VTIMEZONE MUST BE used.

        (C)  If TZURL is not included, and when VTIMEZONE is included 
the CUA MUST use the enclosed VTIMEZONE.

This gives some transitional time for currently working CUAs and I don't 
see how it would break anything.

     (A) and (B) Would move things to tzdist and allow pre-tzdist CUAs 
that care, to read the VTIMEZONE.

     (B) Would cause the included VTIMEZONE to become obsolete and allow 
new tzdist compliant CUAs to read pre-tzdist objects.
         I know the first few versions of Apple iCal included both TZURL 
and VTIMEZONE, I have not checked it for years.

     (C) Would allow compatibility with existing compliant pre-tzdist CUAs.

--

---

From: Lester Caine <[email-masked]>
Date: Sat, 13 Dec 2014 09:20:56 +0000

> may be the only way to get appointment information on Internet

Strip section in some strange language I do not understand ;)

I think that this perhaps is a fundamental part of the problem? Since
the charter is predicated on iCalendar in an attempt to 'make things
easy for the first pass', all of the baggage of that is weighing down
the discussion. My own starting point bypasses that baggage simply
because it did not provide a usable base. VTIMEZONE came into existence
because of the unreliability of a users TZ local data? Fix that problem
and the need goes away? So using it to fix the problem is part of what
is wrong here? I'm looking for tzdist to provide a SINGLE reliable view
of TZ that gets around all of the problems currently CREATED by the
likes of iCalendar and the other consumers of TZ using different views.
iCalendar is only one user and not one that many embedded systems would
use for example, which is why 'expand' being a separate tzdist function
IS necessary.

This is why for me avoiding contaminating the TZ rule set with
conversions to VTIMEZONE's different format IS important. I have no
problem with a third party server producing VTIMEZONE converted TZ data
in much the same way as tzurl already seems to do? The FIRST requirement
of tzdist should be to ensure that the core data used for that matches
every body else's ... a step that the charter totally scraps. But as
long as one can access *A* clean and reliable source AND be able to
identify the version of it that was used for a particular job, then I
will be happy. It's the fact that currently when I look at data from
200x the TZ data is an unknown even if the publisher claims a particular
TZ version.

Once one has a reliable way of getting a stable set of data out in the
field, for which version is a key element, then everything else should
dovetail in on top. Much of the side discussion on the EXISTING race
hazard is in how the current process works and 'not breaking' something
that is fundamentally already broken? If event data is published with a
simple tag to the used tzdist version, the whole job becomes simple and
if people WANT to take advantage of the security provided by being able
to see that the current tzdist version is different then they can do ...
or they continue to simply ignore the information.

---

From: Cyrus Daboo <[email-masked]>
Date: Sat, 13 Dec 2014 09:44:40 -0500

Hi Lester,

> 200x the TZ data is an unknown even if the publisher claims a particular

Well I have made several proposals on how to address this. Let's try 
again...

Paul has suggested that "changedsince" should actually represent as version 
identifier (if I understood his comments correctly). Is it sufficient to do 
this:

1) Replace the "dtstamp" member of the list action response with a 
"version" member that represents the version details for the entire set of 
time zone data returned in the list action? Note that the default /zones 
action may aggregate data from multiple publishers in which case the 
version would be specific to the aggregator not to the publisher. However, 
if the /publisher/versions uri were used then the "version" value would be 
specific to the referenced publisher.

2) The new "version" member value could then be used as the value for 
"changedsince" for clients to get back just those time zones that have 
changed since the specified version.

3) Open issue: does the version value belong in the time zone data returned 
by the server. My big concern about this is that I would prefer it if 
clients did not have to download an entire time zone just to find out that 
the only difference is a change in the version. That is what would happen 
if the client only used ETag and If-None-Match with HTTP. This impacts one 
of the use cases that you championed - namely a simple, fixed, device such 
as a building wall clock that only ever cares about one time zone. Do we 
want all such devices to download the (potentially large) time zone data 
when the only thing that has changed is the version?

What I would like to do is get some consensus on (1) and (2) right now, and 
then come up with ways of addressing (3) that satisfy both your concerns 
about version integrity and my concerns about protocol efficiency (which 
ultimately translate into scaleability concerns if one is going to have a 
service supporting possibly millions of devices).

-- 
Cyrus Daboo

---

From: Paul Eggert <[email-masked]>
Date: Sat, 13 Dec 2014 10:28:05 -0800

Cyrus Daboo wrote:
> 1) Replace the "dtstamp" member of the list action response with a "version"
> member...
>
> "changedsince"...

I think something like this could work, but I confess that I'm not following all 
the details.  In particular I'm not understanding the relationship between 
"version" and ETag (mentioned in (3), below).  Why doesn't ETag subsume 
"version"?  Shouldn't we have just one versioning mechanism, not two or three? 
For example, if the ETag doesn't change for America/Los_Angeles between tz2014a 
and tz2014b, why does an America/Los_Angeles-using client need to even worry 
about versions?  It can just say "give me the latest ETag for 
America/Los_Angeles", verify that the ETag hasn't changed, and be done.

> I would prefer it if clients did
> not have to download an entire time zone just to find out that the only
> difference is a change in the version. That is what would happen if the client
> only used ETag and If-None-Match with HTTP....

Why prefer that?  Does it really cost that much to download an entire time zone? 
  I just did a back-of-the-envelope calculation, with tz binary data, and the 
cost is so low that I'm puzzled as to why we are bothering to try to optimize this.

Here's my quick estimate.  Let's assume a 32-bit operating system (which is what 
small devices would use).  For example, the 32-bit tz-format binary data used by 
Android represents all of America/Los_Angeles in 1058 bytes.  Updates to the tz 
database occur, on average, about ten times per year.  And an HTTP header might 
be 600 bytes (I just now checked Google's, and it's 785 bytes).  Say the tzdist 
protocol request size is 200 bytes, so that a request+response pair is ((600 + 
200) + (600 + 1100)) bytes, or 2500 bytes.  10 times a year makes this 25000 
bytes of network traffic per client per year, which is 0.0063 b/s (that's 6.3 
millibits per second) per client.  The numbers grow a bit for 64-bit clients, 
where America/Los_Angeles is 2819 bytes (this works out to 0.011 b/s), but 
they're still quite small.

Are we really going to bother about shrinking 0.0063 b/s to an even smaller 
number?  We can't shrink the headers, so even with an ideal protocol that 
compresses changes to zero bytes, we could at best shrink a request+response 
pair to ((600 + 200) + (600 + 0)) = 1400 bytes, or 0.0035 b/s.  And the tradeoff 
is a more-complicated client that consumes more energy and development time. 
It's not at all clear that this tiny benefit is worth the cost.

---

From: Lester Caine <[email-masked]>
Date: Sat, 13 Dec 2014 19:55:42 +0000

On 13/12/14 14:44, Cyrus Daboo wrote:

1 and 2 ... spot on ...

> 3) Open issue: does the version value belong in the time zone data
> returned by the server. My big concern about this is that I would prefer
> it if clients did not have to download an entire time zone just to find
> out that the only difference is a change in the version. That is what
> would happen if the client only used ETag and If-None-Match with HTTP.
> This impacts one of the use cases that you championed - namely a simple,
> fixed, device such as a building wall clock that only ever cares about
> one time zone. Do we want all such devices to download the (potentially
> large) time zone data when the only thing that has changed is the version?

I would not expect to be using anything other than 'expand' for an
embedded device with only perhaps a one year look ahead so just perhaps
at most two changes of offset. This just requires that IF there is an
update changing one or more of these forthcoming events then IDEALLY the
ETag for the expand package would change, but as Paul indicates, the
packet may well be a smaller footprint than the ETag label if the range
of the expand perhaps changes daily.

Moving to larger packets. I think I understand part of the
miss-conception on the version model of accessing TZ data. Once a user
has an active sync with the live data, then calling 'list' with  a
'changedsince' of an earlier version will only return a set of TZid's?
The ones which have changed since the earlier version. If none of these
relate to TZid's that the client is currently using there is no need to
download anything, but I would expect the client to bring their local
cached copy up to date. This is where *I* would prefer to be using the
base TZ data since many TZid's these days are using the one master TZ
rule set so as you say 'There is no need to download a lot of TZ data
that has not changed'.

One only needs to read ALL the TZid's of a version if one does not have
a local copy already. Otherwise one reads only the elements needed to
fill the gaps in the version history. I mentioned in a previous post
that probably 90% of TZid data will never change, so every single
version will be the same. Using Paul's point about ETag, every single
version should have the same ETag as there is no point re-downloading.
The problem comes when a single change to say 'central european' generic
rule changes dozens of related TZid's. I would prefer that just the
generic rule changed, but for other users it may be better to flag each
TZid as changed?

Like Paul I also don't see the actual need for ETag since the version
mechanism only requires a single read of each new block of data even if
they are duplicate updates for several TZid's ... It's only the
'changedsince' check of list that needs to be repeated and that only
returns any content if there is a change?

One little sideline here. The race condition where the version changes
AFTER doing a list read but before actually reading the changed data is
covered since the data is read for the version being processed. 'latest'
would not be appropriate against a TZid when processing updates and only
really applies to the list operation.

---

From: Lester Caine <[email-masked]>
Date: Sun, 14 Dec 2014 11:20:21 +0000

> really applies to the list operation.

Having now slept on this, a thought comes to mind that if 'changedsince'
IS using a version tag then a logical extension to the download strategy
is that doing a 'get' for a TZid data set and  applying a 'changedsince'
would allow the return to be simply a diff package. Then if the time
frame of interest at the client is outside the modified time frame again
the update can be ignored. Or more exactly one only looks for a change
to displayed data if the modification is in scope. That way, a change to
historic events can be ignored if the client is only interested in
post-2010 and a change to the next offset transition can be flagged with
the smallest of data packet. ( Not sure this is ETag friendly? ;) )

I would consider this a back burner improvement and one that may not be
needed if transmitting the full dataset for each TZid does not prove to
be a bottleneck.

However it flags a problem on how to handle an update covering several
versions. While the current view only needs the final result of several
version changes, the locally cached data may need to use one of the
intermediate versions if that is what is used by a new diary. EVEN over
the course of a current year, some published mater may still be a couple
a of version behind the current one and there is no need to ask for a
new 'changedsince' list each time one changes required version if that
information is already cached locally.

One way of handing this initially may be for the 'changedsince' list to
return only the next version's changes, which the client then processes
and does a further 'changedsince' list using the next version. This
would normally only happen if a client has been switched off for some
time, so I don't think there is ANY need to be able to handle more than
one version update at a time? Except that creating a new local cache
would potentially need the whole version history from day one ... that
may benefit from an extended data get providing all the previous
versions ...

CURRENT application software will have problems with this process since
none of the existing TZ systems allow viewing an older version than
THEIR latest (which ever version that is) and I think it is that hurdle
which may prevent this process being adopted? If the existing code
continues to simply use 'latest' then all of the existing problems
remain and I don't think it breaks anything? Then over time countries
who will benefit more from the security provided by proper version
control can start to leaver it knowing that the underlying tzdist DOES
have access to all the relevant data and that it is being reliably
updated. Material published with an identified version can always be
reliably handled.

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 14 Dec 2014 11:04:23 -0500

Hi Paul,

--On December 13, 2014 at 10:28:05 AM -0800 Paul Eggert 
<[email-masked]> wrote:

>> "version" member...
> I think something like this could work, but I confess that I'm not
> following all the details.  In particular I'm not understanding the
> relationship between "version" and ETag (mentioned in (3), below).  Why
> doesn't ETag subsume "version"?  Shouldn't we have just one versioning
> mechanism, not two or three? For example, if the ETag doesn't change for
> America/Los_Angeles between tz2014a and tz2014b, why does an
> America/Los_Angeles-using client need to even worry about versions?  It
> can just say "give me the latest ETag for America/Los_Angeles", verify
> that the ETag hasn't changed, and be done.

Well ETag is something very specific to the HTTP protocol and is used in 
several ways - in particular it is important for caching by intermediaries 
etc. Plus there are reasons when you might want to refresh the client data 
in the absence of a publisher version upgrade (e.g., the tzdist server is 
using a more efficient, compact representation that it previously had). So 
tying the ETag to the publisher version is probably not the right approach. 
It is worth noting, however, that HTTP has the concept of "strong" and 
"weak" ETags (see <http://tools.ietf.org/html/rfc7232#section-2.1>). It may 
be possible to make use of "weak" ETags for the version id.

>> difference is a change in the version. That is what would happen if the
>> client only used ETag and If-None-Match with HTTP....
>

> Are we really going to bother about shrinking 0.0063 b/s to an even
> smaller number?  We can't shrink the headers, so even with an ideal
> protocol that compresses changes to zero bytes, we could at best shrink a
> request+response pair to ((600 + 200) + (600 + 0)) = 1400 bytes, or
> 0.0035 b/s.  And the tradeoff is a more-complicated client that consumes
> more energy and development time. It's not at all clear that this tiny
> benefit is worth the cost.

So when we first developed CalDAV we were not that concerned with 
scaleability issues - or key goal was to just get any acceptable, 
interoperable calendaring and scheduling client/server protocol into use. 
Since then CalDAV's use has spread to services hosting potentially 10's to 
100's of millions of users. In that kind of environment, yes, every byte 
does count, because the sum of all those extra bytes across the full set of 
users/devices is huge. For CalDAV we have incrementally added new protocol 
features to improve scaleability (e.g., the WebDAV sync report). I don't 
want to ignore scaleability this time around, since the ultimate goal of 
tzdist is to have every device on the planet that needs to know local time, 
make use of it. It may well be that we have large service providers 
supporting it and they will care about how often and how much data clients 
will download. So I really don't want a change to a time zone version id, 
with nothing else being different, to invalidate a client cache and force a 
download.

As far as more complicated goes - simply making use of existing HTTP 
features such as ETags and conditional responses is not an issue because 
nearly every device we are likely to care about has an HTTP protocol stack 
that copes with those aspects of the protocol.

That said, at this point, it may be simply better to take the CalDAV 
approach and go with something that may be less efficient in the first 
place, but has the necessary extensibility to be scaled to huge deployments 
when the demand arises.

-- 
Cyrus Daboo

---

From: Doug Royer <[email-masked]>
Date: Sun, 14 Dec 2014 11:19:10 -0700

On 12/13/2014 07:44 AM, Cyrus Daboo wrote:
> Paul has suggested that "changedsince" should actually represent as 
> version identifier (if I understood his comments correctly). Is it 
> sufficient to do this:
>
> "version" member that represents the version details for the entire 
> set of time zone data returned in the list action? Note that the 
> default /zones action may aggregate data from multiple publishers in 
> which case the version would be specific to the aggregator not to the 
> publisher. However, if the /publisher/versions uri were used then the 
> "version" value would be specific to the referenced publisher.

*QUESTION 1:* Given:

The DTSTAMP property gets updated each time a VTIMEZONE is 
*instantiated*, not updated.

  RFC 5545 section 3.8.7.2: (DTSTAMP)

    ...Property Name:  DTSTAMP ...Purpose:  In the case of an iCalendar 
object that specifies a
       "METHOD" property, this property specifies the date and time that
       the instance of the iCalendar object was created. ...

RFC 5545 section 3.8.7.3.  Last Modified

     ...Purpose:  This property specifies the date and time that the
       information associated with the calendar component was last
       revised in the calendar store....

DTSTAMP tells you when the object instance was created, not when it was 
LAST-MODIFIED. DTSTAMP is used to disambiguate multiple similar objects 
to determine the order of processing (mostly in iTIP). The oldest 
DTSTAMP is the oldest one sent in a series.

It also allows me to tell when I stored TZID:foo in my store by looking 
at its DTSTAMP time (which better be a  newer or the same timestamp 
value as the LAST-MODIFIED timestamp value).

Question 1 issue:  (A) I  am confused (DTSTAMP vs LAST-MODIFIED) and 
VERSION:

     draft-tzdist says in 5.4 "expand" Action

         ... *changedsince*  OPTIONAL, but MUST occur only once.  If 
present, its
          value MUST be taken from the "dtstamp" result of a previous
          expand result.  If the targeted time zone has not changed over
          the expansion range queried in the request, then the server
          MUST return a 304 HTTP status response.

So, if your going to use it for version, then each time I 'expand' 
query, I get a new version, because each new VTIMEZONE reply has a new 
instantiation time and would have an updated DTSTAMP?

And draft-tzdist says in 5.2

     Description:  This action lists all time zone identifiers in summary
       format, with aliases and optional localized data.  In addition, it
       returns a timestamp which is the current server last modification
       value.  If the "changedsince" URI query parameter is present, its
       value MUST correspond to a previously returned timestamp value.
       When "changedsince" timestamp is used, the server MUST return only
       those time zones that have changed since the specified timestamp.

If I do a 'list', I get LAST-MODIFIED as the version, if I do a 
'expand", I get a DTSTAMP which will often give me different versions?

*QUESTION 2:*

And how at event generation time would VERSION be put into a VEVENT so 
that it can be sent to a CUA to understand (as in update ABNF to 5545?)? 
The ultimate goal is provide the CUA with sufficient information to get 
any missing TZID or TZID/version and verify it is the same TZID used by 
the organizer.

Are you saying the VERSION would be placed in the events  also so that 
CUAs can check their cache for having the same version? If not in the 
organizers sent event, then how does it help me determine if I have the 
latest version?

--

---

From: Paul Eggert <[email-masked]>
Date: Sun, 14 Dec 2014 10:28:10 -0800

Cyrus Daboo wrote:
> ETag is something very specific to the HTTP protocol and is used in several
> ways - in particular it is important for caching by intermediaries etc.

Yes, and that's an argument for our using ETag instead of an 
application-specific versioning system.  Why reinvent a caching system when we 
can use HTTP's?

 > Plus
> there are reasons when you might want to refresh the client data in the absence
> of a publisher version upgrade (e.g., the tzdist server is using a more
> efficient, compact representation that it previously had).

That sounds fairly unusual, but if it happens, then the server could append 
something to the ETag, e.g., "tz2014j-format2" instead of plain "tz-2014j". 
That is, the ETag needn't be just the publisher version; it could be the 
publisher version plus the server format if the latter matters.

> HTTP has the concept of "strong" and "weak" ETags (see
> <http://tools.ietf.org/html/rfc7232#section-2.1>). It may be possible to make
> use of "weak" ETags for the version id.

Sure, and if people want to use weak ETags then they wouldn't need to mess with 
"-format2" suffixes.

---

From: Lester Caine <[email-masked]>
Date: Sun, 14 Dec 2014 19:59:37 +0000

> organizers sent event, then how does it help me determine if I have the

That material is currently published without any reference to the source
of the TZ data used is the main problem with archived data even current
event diaries. In order to be able to TAG which source is used, we need
a clean set of published data ... with a clean means of identifying the
version of data used. So adding a tag to VEVENT and any other published
diary that identifies the exact source at least allows one to know what
data was used, and then check if a later version results in a potential
change to the published data.

Question 1 is just picking up the fact that the current draft is not
consistent since it has not fully embraced the 'version' tagging
process. LAST_MODIFIED IS a different name for version, and DTSTAMP is
not a replacement for a version tag, but if 'changedsince' uses the
version tag, then DTSTAMP is replaced by TZVERSION. Both list and expand
would return the latest version, unless 'changedsince' is used to
identify an earlier base to work from. If there is agreement now on how
the version process protects against the various hazards, then I'm quite
happy to re-write some element of the current draft, but it IS how this
would then dovetail into the iCalendar method of working that I am weak on.

---

From: Paul Eggert <[email-masked]>
Date: Sun, 14 Dec 2014 12:12:50 -0800

Cyrus Daboo wrote:
> In that kind of environment, yes, every byte does count, because the sum of all
> those extra bytes across the full set of users/devices is huge.

In that case I'm puzzled why VTIMEZONE was chosen as the format for data, as 
it's pretty fat.  For America/Los_Angeles, vzic generates a VTIMEZONE format 
containing 4004 bytes, compared to the 1058 bytes that Android uses for tz 
binary format.  If the problem is that VTIMEZONE is bloated, perhaps a better 
solution would be to recommend a more-compact data format.

That being said, I'm still puzzled as to why shrinking the byte count is so 
important that we should complicate the protocol to support it.  As a 
back-of-the-envelope calculation, suppose a VTIMEZONE zone is 4000 bytes and we 
make the same assumptions as before for everything else (32-bit native tz binary 
data is 1100 bytes, the tzdist request is 200 bytes, the protocol overhead is 
600 bytes for both request and response, and there are 10 updates per year). 
Then we see the following per-client costs over time:

0.0137 b/s VTIMEZONE full copy each request
0.0063 b/s 32-bit tz binary full copy each request
0.0035 b/s tzdist version-style no-change handshake
0.0030 b/s? ETag-style no-change handshake? (not clear what's involved here)

To give us some perspective on this, if we assume a single reasonably-cheap 
server can process 300 Mb/s sustained, then a single tzdist server could support 
21 billion tzdist clients each using the least-efficient protocol mentioned 
above.  That's pretty scalable: one cheapish server should be able to handle the 
entire planet and then some.  So why are we worrying so much about making every 
byte count?  Shouldn't it be more important to keep things simple?

---

From: Lester Caine <[email-masked]>
Date: Sun, 14 Dec 2014 20:27:11 +0000

On 14/12/14 20:12, Paul Eggert wrote:
> In that case I'm puzzled why VTIMEZONE was chosen as the format for
> data, as it's pretty fat.  For America/Los_Angeles, vzic generates a
> VTIMEZONE format containing 4004 bytes, compared to the 1058 bytes that
> Android uses for tz binary format.  If the problem is that VTIMEZONE is
> bloated, perhaps a better solution would be to recommend a more-compact
> data format.

Paul ... The problem was the original drafting of the charter for the
tzdist working group was probably too biased towards iCalendar. So
reliance on iCalendar processes is what was chartered. ADDITIONAL
processes are covered and permitted, so I'm not actually bothered that
the first incarnation will document using VTIMEZONE. I'm planning to
retain TZ format since that can be pumped directly into the PHP library
and other TZ based tools, something that is made too complicated by the
reworking of the rules into VTIMEZONE format and then trying to restore
them back to TZ data. It's the 'version' update process that I'm
convinced is the only way to make this work, and we potentially even
have incremental diffs we can use directly from the TZ process! Using
sub-versions on the basic version mechanism.

---

From: Doug Royer <[email-masked]>
Date: Sun, 14 Dec 2014 13:56:34 -0700

On 12/14/2014 01:12 PM, Paul Eggert wrote:
> VTIMEZONE format containing 4004 bytes, compared to the 1058 bytes 
> that Android uses for tz binary format.  If the problem is that 
> VTIMEZONE is bloated, perhaps a better solution would be to recommend 
> a more-compact data format.
>
> That being said, I'm still puzzled as to why shrinking the byte count 
> is so important that we should complicate the protocol to support it.  
> As a back-of-the-envelope calculation, suppose a VTIMEZONE zone is 
> 4000 bytes and we make the same assumptions as before for everything 
> else (32-bit native tz binary data is 1100 bytes, the tzdist request 
> is 200 bytes, the protocol overhead is 600 bytes for both request and 
> response, and there are 10 updates per year). Then we see the 
> following per-client costs over time:
>
> To give us some perspective on this, if we assume a single 
> reasonably-cheap server can process 300 Mb/s sustained, then a single 
> tzdist server could support 21 billion tzdist clients each using the 
> least-efficient protocol mentioned above.  That's pretty scalable: one 
> cheapish server should be able to handle the entire planet and then 
> some.  So why are we worrying so much about making every byte count?  
> Shouldn't it be more important to keep things simple?

The update might change 10 times a year. I would think the CUAs will be 
checking a lot more often just to see if today is one of the days an 
important TZ changed.

--

---

From: Doug Royer <[email-masked]>
Date: Sun, 14 Dec 2014 14:28:48 -0700

> change to the published data.
YES. And I am just hopping for a way that has minimal impact on existing 
code.

The existing definition for tzparam (RFC5545) includes the provision for 
prefixing the TZID with a '/' to specify it is from a registry.

By specifying a format for a registered tzidparm, then TZID + source + 
version could be included in existing events with minimal impact on 
exiting code. Perhaps specifying that tz-dist compliant CUAs mandate the 
tzidprefix (now optional):

     From:
              tzidparam  = "TZID" "=" [tzidprefix] paramtext

              tzidprefix = "/"

     To:
              tzidparam  = "TZID" "=" tzidprefix tzparamtext

              tzidprefix = "/"

             tzparamtext = tzsourceurl '?' tzversionused

             tzsourceurl = <any valid url: http: file: caldav, or 
tz-dist server ....>

             tzversionused = last-mod-value        ; The date-time form 
the LAST-MODIFIED from the original VTIMEZONE on the tz-dist server.

--

---

From: Paul Eggert <[email-masked]>
Date: Sun, 14 Dec 2014 16:29:50 -0800

Doug Royer wrote:
> The update might change 10 times a year. I would think the CUAs will be checking
> a lot more often

If so, the overall cost of checking-for-no-change will completely swamp the 
overall cost of transferring data (on the rare occasions where an update is 
actually needed).  So why expend design and implementation effort optimizing a 
case that doesn't much matter?

I'd feel better about this if there were some analysis justifying why the 
protocol has relatively-complicated features to support incremental updates. 
The back-of-the-envelope calculations I've done make it look like they're more 
trouble than they're worth.

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 15 Dec 2014 14:00:01 -0500

Hi Doug,

> *instantiated*, not updated.

Well that's an incorrect assumption given that RFC5545 does not in fact 
define DTSTAMP as a required or optional property for use in VTIMEZONE.

> So, if your going to use it for version, then each time I 'expand' query,
> I get a new version, because each new VTIMEZONE reply has a new

changedsince has been removed from the expand action (latest draft) since 
ETag is actually sufficient for detecting expand changes. changedsince is 
only present in the list action.

> organizers sent event, then how does it help me determine if I have the

That is a question for iCalendar, not tzdist (though we may decide it is 
better to define any new property in tzdist as we did with the TZUNTIL and 
TZID-ALIAS-OF). Note that since we are currently not proposing to change 
iCalendar/iTIP to deal with this issue it is moot. We are proposing 
changing CalDAV - but there everything is simple because the requirement is 
that the CalDAV server advertises what tzdist server is in use (with the 
assumption being that is readily accessible to all clients using the CalDAV 
server) - so the clients are all "sync'd" in regard to what the time zones 
should be - namely the latest ones provided by the referenced tzdist server 
(which is what the CalDAV server will be using). If/when we decide to 
change iTIP to allow skipping of the VTIMEZONE component, then we can add 
what is necessary to ensure both sides of the exchange have what they need 
to detect mismatches.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 15 Dec 2014 14:15:35 -0500

Hi Paul,

--On December 14, 2014 at 12:12:50 PM -0800 Paul Eggert 
<[email-masked]> wrote:

>> In that kind of environment, yes, every byte does count, because the sum
>> of all those extra bytes across the full set of users/devices is huge.
>
> In that case I'm puzzled why VTIMEZONE was chosen as the format for data,
> as it's pretty fat.  For America/Los_Angeles, vzic generates a VTIMEZONE
> format containing 4004 bytes, compared to the 1058 bytes that Android
> uses for tz binary format.  If the problem is that VTIMEZONE is bloated,
> perhaps a better solution would be to recommend a more-compact data
> format.

VTIMEZONE is used because the first consumers of this protocol are 
iCalendar-based products (and as has been stated before it was those 
products that started this whole thing off by first implementing their own 
tzdist-like behavior in a proprietary manner, with the subsequent 
realization that an interoperable protocol would be better for all). Yes it 
could be considered bloated compared to some binary representation, however 
it is an interoperable standard format that exists today.

> That being said, I'm still puzzled as to why shrinking the byte count is
> so important that we should complicate the protocol to support it.

I think the protocol as it stands is far from complicated. It relies on 
ETags (which are a fundamental element of HTTP that nearly all devices 
support in their HTTP stacks), and the dtstamp/changedsince behavior, that 
I am sure all the other implementors of the current protocol will claim, 
like me, is trivial to implement and provides an incremental update 
mechanism that is more efficient at very little added cost.

The complexity is all being added by the apparent need for versioning. I 
still contend that is unnecessary for tzdist. It is a problem for the 
consumers of the tzdist data and tzdist itself provides sufficient 
meta-data for them to deal with that.

> As a
> back-of-the-envelope calculation, suppose a VTIMEZONE zone is 4000 bytes
> and we make the same assumptions as before for everything else (32-bit
> native tz binary data is 1100 bytes, the tzdist request is 200 bytes, the
> protocol overhead is 600 bytes for both request and response, and there
> are 10 updates per year). Then we see the following per-client costs over
> time:
>
> cheapish server should be able to handle the entire planet and then some.
> So why are we worrying so much about making every byte count?  Shouldn't
> it be more important to keep things simple?

Scaleability involves a lot more than just the server by itself, there are 
many other issues related to intermediaries, to efficiency on the client 
side (where battery drain on devices can be a show stopper), security 
requirements, etc. The systems I have built have not had to scale much 
beyond 100K's of users, but I work with others who do build out to 10's 
millions or more. Simple back of the envelope calculations don't suffice - 
there are far too many variables involved - but the efficiency of the base 
protocol itself is one major aspect that we do have control over when we 
design it. Even in the systems I have built, protocol efficiency has been 
one of the biggest pieces to deal with in terms of scaleability.

I think, as described earlier, what we have right now is a pretty lean 
protocol that provides a reasonable path to scaleable implementations, and 
we have extensibility to go further if what we have done to date is not 
sufficient for the very large providers.

-- 
Cyrus Daboo

---

From: Paul Eggert <[email-masked]>
Date: Mon, 15 Dec 2014 12:01:56 -0800

On 12/15/2014 11:15 AM, Cyrus Daboo wrote:
> Simple back of the envelope calculations don't suffice

Sure, but it's not unreasonable to do them, as a sanity check.  I didn't 
pull that 300 Mb/s figure out of the air: I took it from Squid 
benchmarks, and it's a good basis for what a caching tzdist server 
should be able to do.  If our back-of-the-envelope calculations indicate 
that the protocol is overdesigned, the response should not be to ignore 
the calculations: it should be to do better analysis to see whether the 
problem is real.

> there are many other issues related to intermediaries, to efficiency 
> on the client side (where battery drain on devices can be a show 
> stopper), security requirements, etc.

Of course, but those issues also argue against having a changesince 
feature, as its use would complicate clients and this complication would 
drain their batteries (not to mention making security considerations 
more difficult to analyze and to get wrong).  And if our 
back-of-the-envelope calculations indicate that changesince doesn't help 
overall performance much, why are we bothering with it?

> the protocol as it stands is far from complicated

It's pretty simple as protocols go, yes, but the 
versioning/changedsince/ETag part is confusing (just read the email 
messages in this thread :-) and this part should be simplified. Perhaps 
if we had just ETag, that would be simpler and easier for everyone to 
understand, and we could build versioning and changesince and etc. on 
top of that later, as needed, and as we gain experience on what's 
actually useful.

---

From: Lester Caine <[email-masked]>
Date: Mon, 15 Dec 2014 20:33:22 +0000

> meta-data for them to deal with that.

There are race conditions that can only be identified by the correct use
of versions. These are currently totally unmanaged and if tzdist is
released retaining these problems, that fact has to be documented
properly. Something that the current draft simply ignores.

I don't see how a proper management of the publication of data makes
things complicated. It is simply that some users will never ever see the
race condition that it protects against, and it is the attitude that
potential errors are acceptable I find uncomfortable.

I ask again ... how does your current draft handle the race conditions
created when users do NOT have the same version of data and may well be
fed data that is out of sync with the publish3d event data? This thread
is still under the 'managing historic data' item, and being able to read
the matching version of data is the only way to achieve that, something
that could perhaps be classed as 'optional', but the distribution race
problems are VERY MUCH to do with current data and something that can be
totally eliminated using exactly the same process. If you are happy to
distribute unreliable data then that fact should be documented so that
people understand the holes.

Of cause if tzdist is 'only' to feed iCalendar services then perhaps we
need something different to distribute TZ data to services that do not
want iCalendar formats such as all of the core OS systems?

---

From: Doug Royer <[email-masked]>
Date: Mon, 15 Dec 2014 15:06:35 -0700

On 12/15/2014 12:00 PM, Cyrus Daboo wrote:
> Well that's an incorrect assumption given that RFC5545 does not in 
> fact define DTSTAMP as a required or optional property for use in 
> VTIMEZONE.

Okay, so PLEASE change the name to not be confused with the other 
'dtstamp' ?

Are the other exactly the same name objects as RFC5545 the same? Or not?

--

---

From: Daniel Migault <[email-masked]>
Date: Tue, 16 Dec 2014 20:12:40 +0100

Dear Friends and Colleagues,

I would like to close this thread to move on. There was not a strong
consensus that Lester's historical data use case was not to be considered.
So my current position is that we should consider it. It seems to me we
have reached consensus that using a version number solves this issue.
Discussion are on whether this number should be optional or mandatory. I
understand that optional does not solve this issue, and there is no strong
argument against making it mandatory.

I am thus inclined to have a mandatory version number. Although there are
alternative ways to do/think. I would like to see if there are any strong
opposition to this. Please send you opinion. I wish we find consensus
by/during the phone call.

Best Regards,
Daniel

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 16 Dec 2014 14:58:39 -0500

Hi Daniel,

> by/during the phone call.

As I proposed before I want to see the version number be part of the 
meta-data associated with the time zone data - i.e., part of the list 
action response. I don't want to see it in the time zone data due to the 
fact that, with the current manner in which IANA tz data is "tagged" with a 
version, the entire set of time zones would be marked as changed.

I believe providing that information as meta-data is the right thing to do 
and gives clients/applications/protocols all the information they need to 
manage version discrepancies if that is something that is an issue for 
them. That does not preclude the possibility of using version ids that the 
publisher attaches to time zone data in the time zone data itself, if the 
publisher chooses to do that.

As for how the version information is stored and used - that should be left 
solely up to the applications using the time zone data.

-- 
Cyrus Daboo

---

From: Doug Royer <[email-masked]>
Date: Tue, 16 Dec 2014 13:20:15 -0700

On 12/16/2014 12:58 PM, Cyrus Daboo wrote:
> ...
> action response. I don't want to see it in the time zone data due to 
> the fact that, with the current manner in which IANA tz data is 
> "tagged" with a version, the entire set of time zones would be marked 
> as changed.

That point is valid, however as soon as the data is saved on the 
tzdist-server, and the first fetch is performed, is it not just as 
likely that it would trigger a everything is out of date when they 
compare it to 'nothing authoritative known about this TZID yet - so get 
it' ?

> I believe providing that information as meta-data is the right thing 
> to do and gives clients/applications/protocols all the information 
> they need to manage version discrepancies if that is something that is 
> an issue for them. ...

I don't care if is meta data or in VTIMEZONE. I see the same mass fetch 
everything outcome on the first try.

--

---

From: Paul Eggert <[email-masked]>
Date: Tue, 16 Dec 2014 12:25:46 -0800

On 12/16/2014 11:12 AM, Daniel Migault wrote:
> I am thus inclined to have a mandatory version number.

I also think a version number of some sort is important, and that it 
should be metadata.  The devil is in the details, though, and the 
current proposal seems a bit complicated.  It should be OK, for example, 
for a server to report the version number in the ETag, without having to 
repeat the version number in some other mandatory location.

---

From: Lester Caine <[email-masked]>
Date: Tue, 16 Dec 2014 20:27:59 +0000

On 16/12/14 19:58, Cyrus Daboo wrote:
> fact that, with the current manner in which IANA tz data is "tagged"
> with a version, the entire set of time zones would be marked as changed.

The same problem applies to the current TZ distribution process. One
gets a complete set of data every time and then have to work back
through to see if ANY of your live data needs looking at and perhaps
updating. That is the main reason I'm looking for a process that DOES
only return the changeset rather than a complete dataset.

If one has nothing cached, then one needs a full set of data. Once one
has a local cache running doing a list/changedsince for a later version
only has to return what has changed. This is no different to what would
have to happen anyway in the server. It would have to build the
changeset between the old and new version in order to provide the sub-set.

> I believe providing that information as meta-data is the right thing to
> do and gives clients/applications/protocols all the information they
> need to manage version discrepancies if that is something that is an
> issue for them. That does not preclude the possibility of using version
> ids that the publisher attaches to time zone data in the time zone data
> itself, if the publisher chooses to do that.

The whole point here is to provide some consistency and while there may
be more than one data provider, consistent use of the base TZ data set
is another charter point?

> As for how the version information is stored and used - that should be
> left solely up to the applications using the time zone data.

That is a given. The only problem with the statement is that there are
several consumers other than iCalendar apps that rely on the current
data format how ever it stored. Looking at a single data version is
relatively easy, it's the ongoing process of calculating the change set
each time new data is available. I can see that randomly adding changes
as they become available may seem an easy way of publishing updates, but
it's that very process that makes the management of historic material
almost impossible.

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 16 Dec 2014 16:33:57 -0500

Hi Doug,

--On December 16, 2014 at 1:20:15 PM -0700 Doug Royer 
<[email-masked]> wrote:

> tzdist-server, and the first fetch is performed, is it not just as likely
> that it would trigger a everything is out of date when they compare it to
> 'nothing authoritative known about this TZID yet - so get it' ?

OK, so when a list action changedsince is done we need to make sure of the 
following:

1) The server only reports a time zone as changed if the underlying data or 
meta-data (other than version id) has changed.

2) The server always reports the latest versions now in use for each 
publisher.

So #1 allows clients to fetch the updated tzdata and update any meta-data 
associated with a "changed" time zone.

#2 allows the client to update the version id meta-data for any other time 
zone it has cached, but was not reported in #1.

Note that, if a publisher chooses to include a version number in time zones 
they publish, and they ensure those only change when the data changes, then 
having the version embedded in the time zone data works fine as part of #1.

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Tue, 16 Dec 2014 22:30:33 +0000

On 16/12/14 21:33, Cyrus Daboo wrote:
> OK, so when a list action changedsince is done we need to make sure of
> the following:
> 
> 1) The server only reports a time zone as changed if the underlying data
> or meta-data (other than version id) has changed.
The version is not part of the data. And if one is serving a 'modern'
truncated data set then and changes to TZ outside scope of the period
are ignored as well.

> 2) The server always reports the latest versions now in use for each
This is the bit that bothers me.
And it relates to your incremental changes, so there may be several
'versions' uncovered by a list/changedsince. So the server has to
produce a concatenated list based on all the versions between the old
and new version. Not a particular problem with the right caching store,
but another race hazard.

> So #1 allows clients to fetch the updated tzdata and update any
> meta-data associated with a "changed" time zone.
As indicated previously ...  one does not need to provide the whole TZid
period. Only the diff needs to be transmitted and this simplifies the
checking of material which may be using the updated period. Transmitting
the whole data set does then require the client to establish what the
change was and then assess the knock on effects. Discussion on package
size may be mute if processing a change set becomes the bottleneck?

> #2 allows the client to update the version id meta-data for any other
> time zone it has cached, but was not reported in #1.
While the implementation is a matter for the server software, I envision
a system which would have a database of TZid records which is read and
as a TZid record is updated it is added keyed by it's version number.
When one asks for a 'changedsince', the database returns all of the
versions up until latest. At any time one can read a version as a
complete set of TZid's or the diff's for a version number. I envisage
the same server working as the client cache so all that ever needs to be
transferred is the update diff. The problem arises where there are two
changes to a TZid in the window. Operationally one may only need the
second, but later the first one may be important for archival reasons.

> Note that, if a publisher chooses to include a version number in time
> zones they publish, and they ensure those only change when the data
> changes, then having the version embedded in the time zone data works
> fine as part of #1.
I think the problem people are having is that some caching processes do
treat the version as a variable. Code management systems where the
version number is in every file are a right pain when only the odd file
actually changed. As long as the version is only used as a tag to
identify a particular view then the caching system need only transfer
those 'files' which actually have changes. ETag can safely contain a
version since the list/changedsince mechanism will only list TZid's that
have changed. Users can still access any TZid at any version ...
bypassing the list/changedsince mechanism but if the data is unchanged
from the currently stored version then it can be thrown away.

I think it is probably worth pointing out that probably 90% of the
initial data can be installed as currently happens as part of the OS
installation. Then only updates from that base version need to be
supplied by tzdist. The installed version is a base version ID and
tzdist simply starts up with a list/changedsince from that base. But in
future that base install would also include the historic versions from
the start point that tzdist becomes active. The extra historic versions
are unlikely to be more than a percentage of the total data.

---

From: Doug Royer <[email-masked]>
Date: Tue, 16 Dec 2014 17:21:58 -0700

On 12/16/2014 02:33 PM, Cyrus Daboo wrote:
>> likely
> 1) The server only reports a time zone as changed if the underlying 
> data or meta-data (other than version id) has changed.

Trying to make sure I understand this:

The very first time a tzdist aware application or OS asks a 
tzdist-service, everything will appear to have changed - correct?
So a changedsince makes no sense as the very first query to a tzdist, 
for each unique client - correct?

> 2) The server always reports the latest versions now in use for each 

Great, for my purposes thats all I need.

--

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 16 Dec 2014 20:40:19 -0500

Hi Doug,

--On December 16, 2014 at 5:21:58 PM -0700 Doug Royer 
<[email-masked]> wrote:

> So a changedsince makes no sense as the very first query to a tzdist, for
> each unique client - correct?

Yes. The client will request a "full" list from the server the very first 
time. Note that the client does not then have to download every time zone 
in the list response - it can pick and choose the ones it cares about 
(i.e., an iCalendar client only needs the data for those time zones 
actually referenced in events in its calendar store).

-- 
Cyrus Daboo

---

From: Lester Caine <[email-masked]>
Date: Wed, 17 Dec 2014 08:39:01 +0000

> for each unique client - correct?

Unless a particular version is already pre-loaded ... in which case
tzdist just picks up from that version. Once we have a reliable archive
of versions.

> Great, for my purposes thats all I need.

For a large majority of people once they have a set of data for their
area's of interest nothing will ever change. It is only a small number
of TZid's that will change, although Russia's recent revision is an
example of what can happen and other countries make short term political
decisions. The race problem that THOSE users need to be aware of is one
where they have 'latest', but the diary they are looking at has yet to
be updated to that version! Yes eventually things will catch up, but it
MAY be that 'the organisers' are not even aware there has been a change
over night and now need to make a decision. A message from SOMEONE that
there is a potential problem could assist if the organisers are
themselves travelling between locations at the time? This is the very
black hole that tzdist should be chartered to secure. We do have a few
examples of predicted DST transitions which simply did not happen, which
is something that even tzdist would have a problem with, but at least
tzdist would allow prompt correction of that problem, something which is
not currently practical. The need for tzdist is to publish the changes
... the base data is available anyway.

---

From: "tzdist issue tracker" <[email-masked]>
Date: Thu, 18 Dec 2014 16:45:22 -0000

#32: managing historical data

Comment (by [email-masked]):

 Proposal made on December 17 by Cyrus that handles use cases discussed on
 list.
 Next steps: some examples to list.
 Examples may be included in an appendix.
 Chairs will seek agreement on proposal by end of first week of January.
 This does not handle the historical data case.  That will be handled in
 the tz/version request.
 Cyrus may put out a new draft to show how this will read in context.

-- 
--------------------------------+-------------------------------
 Reporter:  [email-masked]  |       Owner:  [email-masked]
     Type:  defect              |      Status:  new
 Priority:  major               |   Milestone:
Component:  service             |     Version:
 Severity:  Active WG Document  |  Resolution:
 Keywords:                      |
--------------------------------+-------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/tzdist/trac/ticket/32#comment:2>
tzdist <http://tools.ietf.org/tzdist/>

---

From: Daniel Migault <[email-masked]>
Date: Tue, 23 Dec 2014 10:37:02 +0100

Hi,

This notification issue seems to me important for users. However, if I
understand it correctly, it should be handled by the client and looks to me
more an implementation issue. The client should regularly check the latest
version by listing available versions on the server(s). Then it is up to it
to take appropriated actions.

BR,
Daniel

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58