Subject: Re: [Tzdist] Ticket #3: scalability and expected volume of data

From: Martin Burnicki <[email-masked]>
Date: Mon, 01 Sep 2014 10:03:41 +0200

Tobias Conradi wrote:
> On Sun, Aug 31, 2014 at 4:46 PM, Daniel Migault <[email-masked]> wrote:
>
>> I am wondering what volume of data is
>> expected to be transfer for each end user
>
> To analyze this I would split transfers into different cases:
>
> 1) first time setup
> 2) updating data set for one tzid
> 2.1) for tzids where the data often changes
> 2.2) for tzids where it rarely changes
> 2.3) for tzids where almost never changes (e.g. no DST used ever)
> 3) updating all data sets
>
> World population ~ 7 billion:
> ~ 1 billion in China use UTC+08:00, no DST
> ~ 1 billion in India use UTC+05:30, no DST

I agree. I think the folks who will mostly benefit from a tzdist 
protocol are those where the TZ data often changes, e.g. Israel, Egypt, 
Morocco. They have the most problems if those changes are only 
determined announced very late and they have to deploy the updates.

However, even if TZ data rarely or almost never changes polling for TZ 
updates makes computers and embedded systems future-proof with regard to 
cases where a government suddenly decides to change the "usual" TZ data.

> Simple clocks that rely on tzdist would only need to check data for
> one zone and only would need to check whether the current and future
> offsets changed.

Simple clocks (e.g. in embedded devices) might not even be interested in 
the complete historic data for "their" tzid, which can be quite a lot 
for zones where the rules are changed year by year. So there should 
possibly be a way to retrieve only "current" data for a particular tzid.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: [email-masked]
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 01 Sep 2014 09:33:54 -0400

Hi Martin,

--On September 1, 2014 at 10:03:41 AM +0200 Martin Burnicki 
<[email-masked]> wrote:

> I agree. I think the folks who will mostly benefit from a tzdist protocol
> are those where the TZ data often changes, e.g. Israel, Egypt, Morocco.
> They have the most problems if those changes are only determined
> announced very late and they have to deploy the updates.
>

Not only those people, but others who interact with them (remotely) via 
scheduled events. That potentially covers a lot more people than just the 
populations of those countries.

> cases where a government suddenly decides to change the "usual" TZ data.

Yes, absolutely. There are many devices in use today using old versions of 
OS's that are no longer officially supported by the manufacturer, and those 
not getting automatic tz updates. If they used tzdist, they could stay up 
to date, in spite of the OS itself not being updated.

> the complete historic data for "their" tzid, which can be quite a lot for
> zones where the rules are changed year by year. So there should possibly
> be a way to retrieve only "current" data for a particular tzid.

tzdist currently provides two options for that: the truncation approach (so 
the device can just get data for 2014 onwards), and the action=expand which 
allows for a fixed range of observances to be returned. That later is 
probably better since servers are not required to support truncation at 
arbitrary years.

-- 
Cyrus Daboo

---

From: Tobias Conradi <[email-masked]>
Date: Mon, 1 Sep 2014 15:57:46 +0200

On Sun, Aug 31, 2014 at 11:04 PM, Lester Caine <[email-masked]> wrote:
> On 31/08/14 16:28, Tobias Conradi wrote:
>> Data transfer is affected by the "substantive"-concept, which I
>> proposed to be replaced:
>>
>> http://www.ietf.org/mail-archive/web/tzdist/current/msg00237.html
>
> Simple clocks only need to use the 'expand' version of a timezone

Which means more data delivery than under my proposal.

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Tobias Conradi <[email-masked]>
Date: Mon, 1 Sep 2014 16:00:05 +0200

On Mon, Sep 1, 2014 at 10:03 AM, Martin Burnicki
<[email-masked]> wrote:
> Simple clocks (e.g. in embedded devices) might not even be interested in the
> complete historic data for "their" tzid, which can be quite a lot for zones
> where the rules are changed year by year. So there should possibly be a way
> to retrieve only "current" data for a particular tzid.
"Current data" can include data for past offsets.

That's why I wrote "current and future offsets"

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Tobias Conradi <[email-masked]>
Date: Mon, 1 Sep 2014 16:07:50 +0200

On Mon, Sep 1, 2014 at 3:33 PM, Cyrus Daboo <[email-masked]> wrote:

> the device can just get data for 2014 onwards),
But does it allow to see whether the rules for offsets from a
specified point in time onward did change? And only then
- retrieve the data for /current/ and future offsets, or
- retrieve a diff of the data for offsets from that point in time
?

> and the action=expand which
> allows for a fixed range of observances to be returned.
Which might mean more data delivery.

> That later is
Than it is only better because the protocol is limited in its capabilities.

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Lester Caine <[email-masked]>
Date: Mon, 01 Sep 2014 15:50:52 +0100

> Which means more data delivery than under my proposal.
? A simple clock will only pull the transitions for the period it is
working in, and only needs an update if the next transition changes.

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

---

From: Martin Burnicki <[email-masked]>
Date: Mon, 01 Sep 2014 17:12:00 +0200

Hi Cyrus,

Cyrus Daboo wrote:
> scheduled events. That potentially covers a lot more people than just
> the populations of those countries.

I agree concerning the scheduled personal events.

I don't know if you know that the company I'm working for is 
manufacturing NTP time servers and lots of other stuff useful for time 
synchronization.

Every year when countries like those I've mentioned before publish new 
rules for their time zone we get a number of requests from industries 
(telecommunication, etc.) from those countries asking for an NTP server 
which could provide them automatically with the "right" time according 
to the latest rules.

Of course it's not the task of NTP to care about this, and every year we 
need to explain to those folks why this wouldn't work as they expect, 
and that instead they had to update the TZ databases on all their 
machines. It requires quite some effort to deploy those rules to a huge 
number of computer, with different operating systems, if it's possible 
at all in a short time frame until the new rules come into effect.

So I think a huge number of people may not be affected directly but 
indirectly, e.g. if the telecom infrastructure in a specific country 
doesn't work properly just because time zone rules could not be updated 
consistently in time.

That's why I proposed back in 2013 to introduce a protocol which can do 
this automatically, without the requirement of a software or firmware 
update:
http://mm.icann.org/pipermail/tz/2013-August/019521.html

So I'm very happy this is on it's way now.

> Yes, absolutely. There are many devices in use today using old versions
> of OS's that are no longer officially supported by the manufacturer, and
> those not getting automatic tz updates. If they used tzdist, they could
> stay up to date, in spite of the OS itself not being updated.

Indeed.

> tzdist currently provides two options for that: the truncation approach
> (so the device can just get data for 2014 onwards), and the
> action=expand which allows for a fixed range of observances to be
> returned. That later is probably better since servers are not required
> to support truncation at arbitrary years.

Sounds good, and is very much appreciated.

Martin
-- 
Martin Burnicki

Senior Software Engineer

---

From: Tobias Conradi <[email-masked]>
Date: Mon, 1 Sep 2014 19:10:11 +0200

On Mon, Sep 1, 2014 at 4:50 PM, Lester Caine <[email-masked]> wrote:
> working in,
what is your definition of such a "period"? Widely defined terms that
come to my mind are "geologic period" and "millisecond".

> and only needs an update if the next transition changes.
That would fail. Let me give an example using years only.

2013 - synchronization, no future transition recorded

2014 - one transition for 2015, and one for 2016 is added
there was no next, but maybe you could have meant, the next was in
indefinite future, now it moved to 2015. You sync 2015. Get no data
for 2016.

2016 - clock is wrong

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com

---

From: Eliot Lear <[email-masked]>
Date: Mon, 01 Sep 2014 19:18:40 +0200

Hi Tobias,

> come to my mind are "geologic period" and "millisecond".

For purposes of our effort, perhaps we should apply an engineering
constraint and assume that for most applications between 1 an 2 years
should be sufficient.  Not saying you can't use it for longer, but let's
also apply a constraint and say that at least a day is the most often a
change could be made ;-)

Eliot

---

From: Daniel Migault <[email-masked]>
Date: Tue, 2 Sep 2014 11:48:19 +0200

Hi,

Thank for all the feed backs. It looks to me, the current spec do not
suffer from scalability issues anymore. In fact scalability was mostly
raised by the TLS requirements which is currently not anymore required
[Issue #25]. On the other hand, the scalability issue raised by this thread
concerns data truncation which remains an open thread in issue #15.

In order to keep discussion in the appropriated thread, I propose to close
this ticket on issue #3 related to scalability.

Pleas elet me know if you think this thread should remain open.

BR,
Daniel

On Mon, Sep 1, 2014 at 7:18 PM, Eliot Lear <[email-masked]> wrote:

> _______________________________________________
> Tzdist mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tzdist
>
>

-- 
Daniel Migault
Orange Labs -- Security
+33 6 70 72 69 58

---

From: Lester Caine <[email-masked]>
Date: Tue, 02 Sep 2014 11:24:49 +0100

> Pleas elet me know if you think this thread should remain open.

Things are getting messy on trac as most of the early tickets are just
copied from the list and loose some context? The ticket may be out of
date, but there are a few new thoughts which are being tagged as
'future', and certainly "In future version I would like to elaborate a
bit more how this system is scalable." is something that deserves proper
documentation at some point? So need a 'todo' list entry?

I seem to recall some trac setups having a 'milestone' for future
tickets rather than simply closing them, but been a while since projects
I work on dropped trac ;)

---

From: Tobias Conradi <[email-masked]>
Date: Tue, 2 Sep 2014 15:54:35 +0200

On Tue, Sep 2, 2014 at 11:48 AM, Daniel Migault <[email-masked]> wrote:
> Thank for all the feed backs. It looks to me, the current spec do not suffer
> from scalability issues anymore.
Not to me, since the issues in this email thread are not solved yet.

> In fact scalability was mostly raised by
> the TLS requirements which is currently not anymore required [Issue #25].
http://trac.tools.ietf.org/wg/tzdist/trac/ticket/3
is named "Section 4.1.3: Scalability"

This section also contains the concept of "substantive" which was
proposed for removal - also affecting scalability.

> On
> the other hand, the scalability issue raised by this thread concerns data
> truncation which remains an open thread in issue #15.
Then this is a sub-item of #3 or a duplicate. If items are merged,
then I propose to do what many systems do, keep the older one, i.e.
the one with the lower number.

If #15 is a sub-item, then #3 should be open until all sub-items of #3
are solved.

> this ticket on issue #3 related to scalability.
That can be done by referring to the other thread, if it is a sub-item.

If it is a duplicate, then close the other.

-- 
Tobias Conradi
Rheinsberger Str. 18
10115 Berlin
Germany

http://tobiasconradi.com