Subject: RE: TSA Response serialNumber Field

From: "Manger, James H" <[email-masked]>
Date: Thu, 1 Jun 2000 19:20:07 +1000

Ed,
 
The TSTInfo ASN.1 type has a field labelled "accuracy" that "represents the
time deviation around the UTC time contained in GeneralizedTime" (quoting
from the draft) [a misleading label & poorly worded sentence].  Its function
is to indicate how close the stated time is likely to be to UTC.  The field
is meant to be useful to a relying party when they compare the time in the
timestamp to any other time (e.g. a deadline, a revocation date in a CRL,
validity dates in a certificate, another timestamp).  I think the field
should be labelled "uncertainty" and I suspect you would label it
"reliability".
 
A TSA would determine the value of the field from historical measurements of
how much adjustment was required when synchronising with other clocks,
perhaps factoring in the clock drift specification from the manufacture
(e.g. +/-3 milliseconds/day) plus a safety margin (to err on the
conservative side).
 
Now, given that the "accuracy" field actually conveys uncertainty, do I
really need to know the accuracy as well?  How would a comparison work if
you knew the uncertainty and accuracy?
 
-----Original Message-----
From: Ed Gerck [  <mailto:[email-masked]> mailto:[email-masked]]
Sent: Wednesday, 31 May 2000 13:42

..
Now, if by "accuracy" above you mean uncertainty then I would have to
ask about their accuracies, since the time values may have very low
uncertainty
and still be as far apart as apples and speedboats.

So, to the original question: "How do I do it [compare the stamps]?"
The answer must be (for what was provided): insufficient data.

---

From: Ed Gerck <[email-masked]>
Date: Thu, 01 Jun 2000 12:34:57 -0700

> should be labelled "uncertainty" and I suspect you would label it

I would label it "uncertainty" -- so we are in agreement on this change, but
would rewrite the quote also. I initially suggest:

 "uncertainty" -- represents the time deviation around the UTC time 
  contained in GeneralizedTime, with high reliability.

which, btw, explains why I would NOT name it "reliability" [1].  The draft 
must also define "high reliability" and the allowed statistics (see [1] 
and my comments below). 

Using this definition, we may say that with a reliability of 95% the 
UTC time lies between [To - 1.2] and [To + 1.2] s -- when we are given
an uncertainty of 2.4 s and consider "high reliability" defined as 95%
for a gaussian distribution. With gaussian distributions, this directly 
means that the standard deviation is 1.2 s [2].

Clearly, just saying "uncertainty = 2.4 s" is not enough, without the 
reliability.  If the reliability would be 99% then the standard deviation
for the *same* uncertainty of 2.4 s would be 0.8 s [2].

About "high reliability" in my suggested definition change -- I know that defining
high reliability may be tricky (and soon, outdated) -- but this can be done well by
reference. The draft could define a parameter called "reliability level" (for example)
which the TSA associates to a statistic distribution and sets as high as it can put its 
money to it ;-) -- for  example, 99,9997% [2].  This limit will also provide a useful
parameter  to estimate the quality of service, to calculate risks, to estimate 
insurance costs, etc. Note that the TSA bears a risk not only in setting the 
reliability level but also in declaring that its data follows a statistic distribution
-- for example, if we use gaussian statistics to model the stock market there is a 
tail that significantly exceeds that expected from Gaussian statistics, where the 1929
and 1987 stock market crashes were each about a 10 sigma event (ie, 10 times than the
standard deviation), which would only happen once in the lifetime of the planet
under Gaussian statistics [2].

It is also interesting to note that even if "high reliability" is 70% it might 
still be useful if the uncertainty is low enough -- and the price. It is a
one sigma event [2].

> you knew the uncertainty and accuracy?

;-) let's make clear which is which, first.

We have four quantitites:

1. accuracy: that rather arbitrary fine precision that the TSA sets for 
   its timestamp ordering purposes.  

2. uncertainty: set and guaranteed by the TSA, as given above. 

3. reliability: set and guaranteed by the TSA, as given above.

4. statistics: set and guaranteed by the TSA, as given above.

Thus, just having uncertainty and even accuracy does not allow you to 
compare time between two different TSAs for which you know nothing
about their reliability nor about their statistics (as was the case in 
that example [3])

So, how would a comparison work for two timestamps T1 and T2, each one 
from a different TSA -- with different accuracies, uncertainties, 
reliabilities and statistics?

For each timestamp, you have the uncertainty AND the reliability 
as given by each TSA. You would use a statistical model that both TSAs
agree with (eg, gaussian) with each value in respective full accuracy
and compute the probability that T1 >= T2. If this probability is larger 
than 50%, then T1 >- T2, otherwise it is T1 < T2.

Let me build an example using data from Michaels':

T1. Stamp from TSA_1: 20000529053138.34509878Z. 
accuracy = 8 digits
uncertainty = 1 s
reliability = 95%
statistics: gauss

T2. Stamp from TSA_2: 20000529053138.876Z. 
accuracy = 3 digits
uncertainty = 1.5 s
reliability = 99%
statistics: gauss

How do I do it [compare the stamps]?

Even though the uncertainties are different, the sigmas are the same 
for each case, as we see from [2]. The sigma is 0.5 s and we see
that T2 - T1 > sigma. Thus we may directly compare T1 with T2 and say
that T2 > T1. Otherwise, we would need to calculate using the appropriate
gaussian statistics for each.

Now, if we had (with an added "9" in the timestamp for ordering purposes):

T1'. Stamp from TSA_1: 20000529053138.345098789Z. 
accuracy = 8 digits
uncertainty = 1 s
reliability = 95%
statistics: gauss

then we could NOT say that T1' > T1 even if we take into account that they 
came from the same TSA. We could only say that the TSA issued T1' AFTER 
issuing T1.

Further, if we had:

T2'. Stamp from TSA_2: 20000529053138.345098789Z. 
accuracy = 3 digits
uncertainty = 1.5 s
reliability = 99%
statistics: gauss

then we could say nothing about T2' being issued before or after T1,
much less whether T1' > T1 or not.

Cheers,

Ed Gerck

[1] When estimating the mean of a normal distribution, the sample average
provides a point estimate or best guess about the value of the mean. However, 
this estimate is almost surely not exactly correct. A confidence interval 
provides a range of values around that estimate to show how precise the 
estimate is. The reliability level (also called confidence level) associated 
with the interval, usually 90%, 95%, or 99%, is the percentage of times in 
repeated sampling that the intervals will contain the true value of the 
unknown parameter. See [2].

[2] A gaussian distribution with a standard deviation value "sigma" has,
approximately: 69% of all events within one sigma, 95% of all events 
within two sigma, 99% of all events within three sigma. Nearly 100% 
(exactly: 99,9997%) of all events are covered in the range of  six sigma
-- in other words: out of an amount of 1 Million events only 3 events
are outside the range of six sigma.

[3]

---

From: Paul Koning <[email-masked]>
Date: Thu, 1 Jun 2000 15:50:10 -0400 (EDT)

Whenever someone creates a discussion like this where "reliability"
and "accuracy" (or, similarly, "accuracy" and "tolerance" or
"accuracy" and "precision") are used with the intent of describing
quite *different* things, alarms go off.  I find that such discussions
tend to be difficult if not impossible to understand, and it isn't at
all clear whether the conclusions are meaningful.

Take the case discussed here.  Since we're dealing with variables
having some statistical distribution (though very likely *not* one of
the mathematically tractable ones statisticians like to work with) it
is clear that any bounds specified have some probability associated
with them.

If I specify bounds such that the probability of being outside those
bounds is high, I suppose one might speak of this as "low
reliability".  I'd rather not, because to me reliability is a term
that describes the lack of brokenness.  (Or do you want to say that
specifying overly optimistic bounds means you're broken?  I guess I
could go for that...)

If we had a well defined distribution of the random variable, we could
talk about mean and standard deviation.  But I don't believe we do; in
particular I do know that it is a mistake to think that the
distribution is Normal.

Also, consider the intended usage.  One major reason we want to have
TSAs is so we can timestamp information with legally convincing
timestamps.  In other words, we want to be able to answer the question
"did T1 occur before T2" well enough that we can stand up in court and
give that answer.  Not just here on this list... :-)

This gives me a criterion for chosing the bounds I would advertise in
a timestamp: they are chosen reasonably tightly, but such that I am
willing to stand up in court and testify to the effect that an event
timestamped as [t - a .. t + a] did indeed occur within a of UTC t.
In other words, I pick "a" large enough that I am not worried about
the likelihood of being shown wrong when I make that testimony.

Does that mean that I'm implicitly setting "reliability" as Ed uses
the term, to the value "high enough for legal work"?

    paul

---

From: Ed Gerck <[email-masked]>
Date: Thu, 01 Jun 2000 13:49:33 -0700

Paul Koning wrote:

> quite *different* things, alarms go off.

;-) My alarms too -- especially when such terms are used to mean quite
*different* things than usage in the corresponding technical field.

> particular I do know that it is a mistake to think that the

That is up to each TSA to specify, according to my suggestion.  And
I recall that it always makes sense to talk about "mean" and "standard
deviation" for *any* distribution of like things (apples with apples)
 -- even if the distribution is not Normal (Gauss).

> give that answer.  Not just here on this list... :-)

That is why we need: (i) to call a dodo a dodo,  (ii) to make sure that we are
comparing dodos to dodos and not to platipus.

Of course, this WG is free to create all sorts of names and concepts de
novo, as well as to rename the entire corpus of engineering and law.
I suggest this is not in our best interest, though.

BTW, just taking a look at the new IETF "glossary" should convince us
that we may have already gone too far in our solipsistic phantasies as we
start to rename that which we ourselves named ;-)

> Does that mean that I'm implicitly setting "reliability" as Ed

and engineering textbooks

> uses

well, that means you *wish* to do so -- and you may not suceed if
the other side can show with *equal* reliability that you are wrong.
Perhaps, even less than equal would suffice -- reasonable doubt is
enough.

To exemplify why the concept of reliability is important, and quite
disticnt from uncertainty and accuracy, pls read my former email
when I wrote:

 Clearly, just saying "uncertainty = 2.4 s" is not enough, without the
 reliability.  If the reliability would be 99% then the standard deviation
 for the *same* uncertainty of 2.4 s would be 0.8 s [2]. (and not 1.2 s)

Cheers,

Ed Gerck

---

From: Paul Koning <[email-masked]>
Date: Thu, 1 Jun 2000 17:15:48 -0400 (EDT)

>>>>> "Ed" == Ed Gerck <[email-masked]> writes:

 Ed> Paul Koning wrote:

 >> If we had a well defined distribution of the random variable, we
 >> could talk about mean and standard deviation.  But I don't believe
 >> we do; in particular I do know that it is a mistake to think that
 >> the distribution is Normal.

 Ed> That is up to each TSA to specify, according to my suggestion.
 Ed> And I recall that it always makes sense to talk about "mean" and
 Ed> "standard deviation" for *any* distribution of like things
 Ed> (apples with apples) -- even if the distribution is not Normal
 Ed> (Gauss).

Well, I never got very far in that area of Math.

Here's why I brought up the issue:

Suppose we define that the timestamp will contain the mean and
standard deviation.  What will that actually tell you?

If you know the distribution (e.g., Normal) you can answer the
question "with 0.999999 confidence, what are the bounds".  With an
unknown distribution, you cannot do that.  So clearly standard
deviation is not useful.

That leaves two possibilities:

a. What I suggested before: mean plus bound chosen so the confidence
is "very high".

b. Mean plus bound plus confidence value -- preferably with the
recommendation that the confidence value should be a number quite
close to 1.  (I don't think I want to do business with a TSA that
reports intervals with "confidence 0.68"...)

	paul

---

From: Ed Gerck <[email-masked]>
Date: Thu, 01 Jun 2000 15:17:11 -0700

> unknown distribution, you cannot do that.  So clearly standard

Let me recall that you can indeed compare reliability values by using the
standard  deviation for statistics other than gaussian (normal). This shows
why it is useful to include the *four* types of information I suggested for the
TSA (and not just that "accuracy"  -- which is a meaningless number by
itself, and misleadingly named as it was):

For example ([2] in my posting before),  in the case of  gaussian statistics,
95% reliability corresponds to an uncertainty which is twice the standard
deviation.

On these matters, to save cyber trees, I looked for some references on the web
that could help provide some examples for each concept. I found the ones below,
which can be helpful (I am sure there are many others):

http://bjmath.com/bjmath/Stats/sd.htm
http://bjmath.com/bjmath/Stats/sd.htm
http://bjmath.com/bjmath/Stats/confide.htm

> a. What I suggested before: mean plus bound chosen so the confidence

these numbers cannot be taken out of the blue yonder, but must be
based on a sound and objective basis -- mean, bound and confidence
cannot be arbitrary.  Further, "very high" confidence (ie, reliability) may
indeed mean 60% in some cases and this may be as good as 99% in
other cases -- as I exemplified in my previous posting with 95% and 99%.

> reports intervals with "confidence 0.68"...)

Would it surprise you to know that a confidence (reliability) level of
68% for an uncertainty of 0.2 s is much better than 99% for an
uncertainty of 1 s, in terms of gauss distributions?

Further, without specifying to which statistical distribution the value
(mean), the uncertainty (bound) and the reliability (confidence) apply to,
the TSA is essentially selling snake oil.  Let's keep this in mind --
statistics is quite useless unless the terms are well defined, as politics
exemplifies all the time ;-)

As another example, I wish to recall the case of that guy who drowned in
a lake that had one inch depth on average -- he fell in a 40 ft deep hole.

Cheers,

Ed Gerck

---

From: "William Lattin" <[email-masked]>
Date: Thu, 1 Jun 2000 17:03:23 -0700

Another place to look for the treatment of time accuracy and deviation is
RFC 1305 - NTP has attempted to deal with these very issues (albeit for NTP
purposes :-)).  NTP introduces the concept of dispersion (see Appendix H)
which, I believe, addresses the issues being discussed.

Bill

> From: Ed Gerck [mailto:[email-masked]]
> Sent: Thursday, June 01, 2000 15:17
> To: Paul Koning
> Cc: [email-masked]; [email-masked]
> Subject: Re: TSA Response serialNumber Field
>
>
>
>
> Paul Koning wrote:
>
> >

Cheers,

Ed Gerck

---

From: Ed Gerck <[email-masked]>
Date: Thu, 01 Jun 2000 23:51:28 -0700

> which, I believe, addresses the issues being discussed.

No. Appendix H has to do with corrections for delays and offsets in
 NTP communication paths -- roundtrip delay, synch dispersion, etc.
The TSAs timestamps we are talking about have nothing to do with
this model  -- the timestamps here represent static numbers for which
communication delay and synch dispersion is irrelevant after they are
issued by the TSA (and before they are issued we as r-ps do not care --
it is the TSA's business).

So, not only the issues are different but also the consequences. To our
case, what is relevant is a simple question. What does a TSA need to inform
together with a timestamp to a relying-party so that the r-p can rely on that
timestamp for:

(i) a value by itself;
(ii) comparison between timestamps from that TSA;  and
(iii) comparison between timestamps from different TSAs.

My suggestion was that the TSDA needs to inform four quantities besides
the timestamp itself to the r-p, at least.  Otherwise the information may not
allow (i), or (ii, or (iii).

Cheers,

Ed Gerck