Subject: Summary, was Re: Every time ..., was Re: General formula

From: Ed Gerck <[email-masked]>
Date: Fri, 04 Jun 1999 09:53:41 -0700

Alan Lloyd wrote:

>  For my three bobs worth, the fundamental issue is that a complex
> problem like the lifetime of a lump of data cannot be solved by a simple
> (or lightweight) solution or formula

Alan:

I feel that a summary is in order -- too many subjects and topics
on this issue ... we need to see the forest in spite of the trees.
I also include some comments to other msgs and will try to
synthezise list arguments so far.

1. never say never -- I mean, we don't have a dogma here, do we?

2. The more you say it is impossible, the happier I feel ;-)

3. As I commented before, I am thankful for your doubts -- they
have helped me bring to the forefront issues such as Poisson and Erlang
statistics. Even though I may not have been 100% successfull (it seems)
to convince everyone, if my private mailbox can be proof then I have
been at least partially successfull in pointing out that modelling is much,
much better than guessing ;-)

4. But, I also do not want to impose upon this list a course on poisson
statistics modelling  -- just using the results is usually fine for all
engineering purposes. Further, I take that current doubts from Tom and
others can be solved by resorting to  textbooks, so I am trying to wait
and see what happens in time on this regard.  Again, the equation does
not have to be deduced before it is used ;-) It is already there and 
those that may feel more confortable with examples will be able to verify
that it works; again, in their own due time and there is no pressure.

5. If you don't trust the equation, then I suggest you *first* do the only
thing available today -- wild guess the result. Then, I suggest you do
calculate it. Then, I suggest you use whatever you feel is better  -- the
"wild guess" or the "wild calculation" ;-) (as you may call it). Then, I
further suggest you wait out and check what happens in the practice.
With all due respect to theory, this is IMO a far more significat test
than pages and pages of equations -- paper is too flexible and,
bytes too easily changed, as we all know ;-)

6. If you do want to "wild calculate" then please do read the inlined
reference [1] in my original posting -- it explains the modelling phase that
*must* precede any lifetime calculation using that equation. As it was very
timely remarked here yesterday by Andrew Probert:

"However for this, or any capacity planning exercise, real-world data and
assumptions are input to the model."

Thus, slightly paraphrasing Andrew's following text, I may say:

"Gerck's certificate lifetime equation requires the user to input their
assumptions on each presented attribute validity lifetime (an average)
of a *given* certificate, when a single exponential decay is assumed for
each attribute validity.  When this is not true for an attribute (eg, in the
case of a square-function validty) but the attribute can be described
by a piecewise exponential decay (as it generally can, at least
approximately) then the user must define and assume enough delayed
sub-attributes to that attribute so that the each sub-attribute validity
function is given by a single exponential decay in its time slot *and*
the user must also consider the resulting piecewise application of the
certificate lifetime equation -- so that in each time slot, all attributes
and sub-attributes are given by single exponential decays.
Alternatively, the user can input what certificate lifetime is desired,
for a given set of operational conditions, to derive a maximum number
of attributes of equal lifetime or a mix of attributes of different lifetimes.
The user does not need to make any assumption about attribute
dependency besides the trivial case of not using fully redundant
(whatever this may mean in the operational context) attributes; such
as repeatingly using the same attribute lifetime in the equation if the
attribute happens to repeated verbatim in the certificate -- since the
attribute is essentially "one" if simply repeated."

Hope it is clearer.

Cheers,

Ed Gerck

---

From: Tony Bartoletti <[email-masked]>
Date: Fri, 04 Jun 1999 11:54:16 -0700

At 09:53 AM 6/4/99 -0700, Ed Gerck wrote:

[snip]

>Thus, slightly paraphrasing Andrew's following text, I may say:

[snip]

>attribute is essentially "one" if simply repeated."

But is this "trivial case" always obvious?  This issue is rarely one
where we disagree axiomatically with statements like "Under these
premises, this formula holds true".  Rather, the disagreements arise
in assuming particular case data satisfy the formula's premises.

And "verbatum repetition" need not characterize "perfect redundancy".
My silly example:

We certify that person X is named "fred", person Y is named "john".

Fred's cert has one attribute:

1   Person's name = fred

John's cert has 5 attributes:

1   Person's name = fred
2   First letter of person's name  = j  (or first letter of attribute 1)
3   second letter of person's name = o
4   Third letter of person's name  = h
5   Fourth letter of person's name = n

I would love to see example lifetimes assigned to these attributes,
and how the formula might be applied.  Should the second certificate
have a shorter lifetime?  Can we "hide" redundancy even further?

Yes, almost anyone can see the strict dependency of attribute 1 with
attributes 2 .. 5 in the second cert.  But it seems intuitive that
in cases where attributes only "tend toward" perfect dependency, the
formula must yield a value that "tends toward" the value of any single
attribute.

I wish I could construct an example for (say) 100 "almost dependent"
attributes, each with expected lifetime t.  As "almost dependent" tends
toward perfection, intuition says that the collection must have an
expected lifetime that approaches t. I fail to see how the formula
can behave so.

Comments?

___tony___

Tony Bartoletti                                             LL
IOWA Center                                              LL LL
Lawrence Livermore National Laboratory                LL LL LL
PO Box 808, L - 303                                   LL LL LL
Livermore, CA 94551-9900                              LL LL LLLLLLLL
phone: 925-422-3881   fax: 925-423-8002               LL LLLLLLLL
email: [email-masked]                                   LLLLLLLL

---

From: Ed Gerck <[email-masked]>
Date: Fri, 04 Jun 1999 16:18:18 -0700

Tony Bartoletti wrote:

> But is this "trivial case" always obvious?

Tony:

Good question. The answer is indeed a NO (as you imply) because of
the caveat above "(whatever this may mean in the operational context)".
It is trivial (a word often used in science to mean that the math is easy)
since you have nothing to calculate ... but deciding it may not be obvious.

For example, some postings ago you presented the following interesting
case:

T> Consider certifying my following attributes:
T>
T> 1.  First Name = "Anthony"
T>
T> 2.  First Name Writ Backward = "ynohtnA"
T>
T> 3.  First Name all in caps = "ANTHONY"

which you thought (in your operational context) to be 100% redundant.
However, I provided a different operational context (perverted?) where
you could have reached a different conclusion:

E> For example, if those 3 attributes were to be interpreted as three different
E> things as viewed from your computer (as the observer):

E> 1. "Anthony" is your login
E> 2. "ynohtnA" is your initial password
E> 3. "ANTHONY" is your email address at the maihost.
E>
E> Now, of course, each one of them has a different meaning and denotes
E> a different entity to the observer (the computer) -- and each one is also
E> quite independent in their lifetimes.

As I hinted then, we need to recognize that 100% redundant terms carry no
information (in IT terms). Thus, there is no sense in distinguishing among
them if they are indeed 100% redundant. So, they should NOT be
included in the equation  -- this is the simple answer.

Why? Because the equation 1/T = 1/T1 + .. +1/TN was derived *for
each* attribute that can be *recognized*. In other words, if there is no
information (surprise) for an attribute, it must not be counted -- since it
was already counted.

However, and here lies that subtle (so, certainly also not obvious) point,
the measurement of information (hence, redundancy) depends on the
observer -- as I showed then using your own example (and, copied
above).

> This issue is rarely one

Yes, and this must be very clear to the user, as I commented in my
sixth point in this thread (previous msg):

sub-attributes ..."

In all, the "equation user" is central. And, most surely *different* users will
disagree! But, how can that be possible? Or, useful? Because we are
dealing here with an intersubjective evaluation -- much like the medical
diagnosis of a patient, which, acceptably, is not unique. See "intersubjective"
in http://www.mcg.org.br/trustdef.htm for definition and examples.

Mind you, the same applies to the calculation of entropy in Physics.
Without wishing to open another can of worms (not this time), it
is entirely possible that two competent physicists justifiedly disagree
on an entropy calculation. And yet, Thernodynamics is a very useful
science.

> John's cert has 5 attributes:

simple mistype:

1   Person's name = john

> 5   Fourth letter of person's name = n

Your example is not silly -- it is actually much more difficult than several
very complicated  examples I have seen.

> have a shorter lifetime?

If both have the same operational context, I can safely say that
they should have the same a priori lifetime.

> Can we "hide" redundancy even further?

;-) Well, there are many things that you have "hidden" already in
that example -- just look at the whole backward trust chain you
are taking at face value even for that declaration:

person  X is named "fred"

not to mention the validity of the CA root key, for example
(which is NOT part of the cert's *visible* attributes).
See also my previous comments to Bob Jueneman on this:

E> The a priori decay of trust with time is what makes
E> the attribute validity decay for the a priori determination
E> of its lifetime. This also shows that a certificate has many
E> unseen attributes, if you recall the list I used to discuss
E> the topics, which will nonetheless reduce its lifetime. One
E> example, of several discussed in
E> http://www.mcg.org.br/cert.htm , is the validity of the CA
E> signing certificate -- which is not mentioned in the
E> subscriber certificate as an attribute but can render it
E> invalid if the CA cert expires or is revoked before the
E> subscriber's certificate lifetime expires.

and, as I commented to Peter Williams on this:

E> As exemplified above, the *same* certificate could afford
E> increased lifetimes in the progression of the different
E> contexts presented -- even legally. And, in all cases,
E> the same lifetime equation would apply, though with
E> a different number of attribute lifetimes and respective
E> values.

> attributes 2 .. 5 in the second cert.

Not everyone will see it in the same way, though. For example,
if that cert was created *that* way, then it may be verifying the
correct decoding of r-p's UNICODE platform when (for some
lamguages) there may be more than one coding for a single word,
though each single letter codes uniquely.

Again, it depends (as we both already agreed in this thread) on the
operational context.

>  But it seems intuitive that

Yes, for sure -- for that *same* operational context though

> expected lifetime that approaches t. I fail to see how the formula

There are two different issues here (see my sixth point,before, at
the end):

1. Direct use: The equation does not calculate attribute lifetime -- it
calculates the average certificate lifetime *given* all the attribute
lifetimes which the user has decided upon. Then, simply, the
certificate lifetime will change (increasing *or* decreasing) as
you increase the inter-attribute dependency from "almost
dependent" to "100% dependent".

2. Inverse use (your case, above): You have a given certificate
lifetime you want and you investigate how many "almost dependent"
attributes you need in order to obtain  lifetime To -- as a function
of what you mean by "almost dependent". As you write the formula
starting with attribute 1, the certificate lifetime must decrease with
each term you add for that specific value of redundancy -- so you
must then stop adding attributes when you are in a neighborhood
of the desired lifetime To. Now, as you do various tests (note, it is
you that inputs the attribute lifetimes) and  increase inter-attribute
redundancy, each individual attribute lifetime  will change and this
will allow the inverse of the sum of their inverses (ie, certificate
lifetime T) to also change until To is reached when you have
only that *one* attribute (100% redundancy) and 1/T = 1/To,
so that T = To.

Comments?

Cheers,

Ed Gerck

---

From: Tony Bartoletti <[email-masked]>
Date: Fri, 04 Jun 1999 19:13:32 -0700

Ed,

Apologies for being somewhat imprecise (and perhaps a little dense)
but I still fail to see how T can approach To.

I know that terms like "almost perfectly redundant or dependent"
are vague, so let me try to define "X almost redundant to Y" to
mean that the probability that X changes (becomes invalid) while
Y does not change can be considered very small.  Indeed, smaller
by orders of magnitude when compared to the size of the set of
variables to which the phrase "almost redundant" applies to every
pair.

Now if I consider 100 such variables, and independently assess
each to have an expected lifetime of 1 year, then I get

   To = T1 = T2 = ... = T99 = 1 (year)
and
   1/T = 1/To + 1/T1 + ... + 1/T99
so
   1/T = 100, T = 1/100  (not at all close to To = 1)

But perhaps the problem is that I cannot find REAL variables that
behave this way.  As hard as I tried, I could only come up with
examples that support your formula.  To wit:

   Consider a very old stick of dynamite, so unstable that it
   may self-detonate at any time, with an expected life of 1 year.

   Now consider a box of 100 such sticks.  IN ISOLATION, each
   stick has expected lifetime 1 year.  But given 100 such sticks,
   the likelyhood that at least one stick will detonate within a
   weeks time is rather large.  So, packaged together in the box,
   the collection of dynamite sticks will likely not last a week.
   (When one of them goes boom, it is almost a certainty that the
   others will follow!)

But this is a strange kind of "strong dependency", and not the kind
I intend by "almost redundant".  In the dynamite example, the
"linkage" of lifetimes is one of causality.  If one stick goes,
then the others follow as an effect.  And if, at time t, one of the
sticks is still intact, it is almost certainly because none of the
other sticks have detonated.

Perhaps what I intend by "almost redundant" is not a reality?
Informationally, a bit is either redundant, or not (boolean).

Comments?

___tony___

Tony Bartoletti                                             LL
IOWA Center                                              LL LL

---

From: Ed Gerck <[email-masked]>
Date: Sat, 05 Jun 1999 00:32:08 -0700

Tony Bartoletti wrote:

> examples that support your formula.

that is not bad ... to agree is also valid ;-)

> To wit:

I think this is a very illuminating example ;-)

Indeed, a certificate behaves this way -- after all, any attribute that is
in the cert....  is there because it is needed and the cert does "explode"
if any attribute "explodes". At least, it will "explode" as a whole, though
it may survive in "pieces".

Your other question (attribute redundancy) depends on the assumed
redudancy model. In the model  I suggested in my former reply to you,
the number of *effective* attributes is reduced as the inter-attribute
redundancy is increased -- so that when inter-attribute redundancy
is 100%, the number of attributes is only one and that is why 1/T =
1/To.

This redundancy model has some real-world significance. It models
a group that gradually loses individual identities by "quantum jumps"
into a collective identity, which is the final collective attribute. In other
words, as you increase redundancy, the number of individuals will
steadly decrease until only the collective is left. Thus, we need to take
into account also *that* attribute lifetime, the "collective attribute
lifetime". Which can have, interestingly enough, the same lifetime,
a lower lifetime or even a higher lifetime than each individual
attribute. Thus, the first formula is

  To = collective attribute lifetime = x years
  T1 = T2 = ... = T100 = individual lifetime = 1 year

and, to begin, suppose there is no collective yet:

   1/T = 1/T1 + ... + 1/T100 => T = 1/100 year

but, as redundancy is increased to the next notch, n individuals
are lost to the collective. Thus, the second formula has less attributes
to sum up and we obtain the partial expression:

  1/T = 1/To + 1/T1 + ... + 1/T(100 - n) => T =  x/((100-n)*x +1) years

and, as redundancy is further increased, there is a time when all
individuals are lost to the collective and n = 100, so that we only
have the collective to take into account:

  1/T = 1/To = x years

which is *also* the result we obtain by making n =100 in the partial
expression for n above:

 T =  x/((100-n)*x +1) years => x/((100-100)*x +1) = x years

Note thus that there is a progressive change from "1 year" to "x years",
going over  x/((100-n)*x +1) years, as redundancy is varied from 0%
to 100%. However, as I wrote above, one may have x >1, x < 1 or
x =1 -- as a function of the operational  context.

Is this what you had in mind?

Cheers,

Ed Gerck

---

From: Tony Bartoletti <[email-masked]>
Date: Mon, 07 Jun 1999 16:28:59 -0700

At 12:32 AM 6/5/99 -0700, you wrote:

>Thus, the first formula is
>

The given formula "works" (assuming all component attributes have identical
lifetimes.)

I wanted to explore the more general case, where the component lifetimes
are unequal, and to help define "degrees of redundancy".

A simple (?) example:

  Consider a certificate C over two attributes A and B with lifetimes given
  by TA and TB, respectively.

  Assuming no redundancy, we allow 1/TC = 1/TA + 1/TB.

  Suppose, upon closer inspection, we agree that attributes A and B are
  themselves formed of components.  Specifically:

	A = a1 and a2 and a3  (respective lifetimes Ta1, Ta2, Ta3)
	B = b1 and b2 and b3  (respective lifetimes Tb1, Tb2, Tb3)

  Note:  Attribute A is valid IFF all of a1, a2, a3 are valid, etc.

  Continuing under assumptions of no redundancy, we still find

	1/TC = 1/TA + 1/TB
	     = 1/Ta1 + 1/Ta2 + 1/Ta3 + 1/Tb1 + 1/Tb2 + 1/Tb3

  Suppose a "final investigation" reveals that the components a1 and a2 of A
  are precisely the components b1 and b2 of B.  Thus the certificate C
  certifies exactly the components a1, a2, a3, b3.

  Comparing the result to the one that assumed no redundancy, I summarize

	1/TC = 1/TA + 1/TB - SUM(1/Tci) for ci in INTERSECT(A,B)

	     = 1/TA + 1/TB - [ 1/Tb1 + 1/Tb2 ]

	     < 1/TA + 1/TB

  Thus   TC > 1/( 1/TA + 1/TB )

Now this result is not really different than the one you give above, where
increasing redundancy gradually reduces the population of elements.  But it
demonstrates the difficulty in practice.

Since the individual components may have differing lifetimes, it may matter
greatly, exactly which components are removed from the intersection.  Saying
"two out of six components were redundant, so use only four" would not suffice.

And, most importantly, in those (many) cases where no one has the time or
know-how to correctly identify the component attributes of A and B, but
rather applies historical "lifetimes" exhibited by A-like and B-like
attributes, there is no opportunity to adjust proactively the formula

	1/TC = 1/TA + 1/TB

Instead, one would just apply it, and soon discover that observed TC
does not agree with its predicted value (and thus surmise that there
must be redundancy at work somewhere.)

Comments?

___tony___

Tony Bartoletti                                             LL
IOWA Center                                              LL LL

---

From: Ed Gerck <[email-masked]>
Date: Tue, 08 Jun 1999 01:17:01 -0700

Tony:

I can't understand the "works" -- do you find a mistake in the results?

>   Assuming no redundancy, we allow 1/TC = 1/TA + 1/TB.

You mean, there is no 100% redundancy? Because redundancy is NOT
a problem in the equation -- as I showed in my e-mail anterior and
you recognized when you wrote above "works" ;-))))

Of course, you cannot just *repeat* the *same* attribute two times
and expect that you have two attributes, in fact you would just one
attribute, no?

>              = 1/Ta1 + 1/Ta2 + 1/Ta3 + 1/Tb1 + 1/Tb2 + 1/Tb3

You "forgot" to impose the conditions, derived directly by your
hypothesis:

1/TA = 1/Ta1 + 1/Ta2 + 1/Ta3
1/TB = 1/Tb1 + 1/Tb2 + 1/Tb3

>   certifies exactly the components a1, a2, a3, b3.

Your affirmation does not make mathematical sense, Tony. What do you mean by

"the components a1 and a2 of A are precisely the components b1 and b2 of B."

Do you mean they are equal? So what? You affirmed before that:

 "Assuming no redundancy, we allow 1/TC = 1/TA + 1/TB"

so we know we are not dealing here with the *same* attributes that were
just repeated, like:

Attribute1 = 'tony'
Attribute2 = 'tony'

So,  your :"results" do not apply. But, it is a simple confusion. Calculate
again, taking into account that:

1/TA = 1/Ta1 + 1/Ta2 + 1/Ta3
1/TB = 1/Tb1 + 1/Tb2 + 1/Tb3

and you will see that it "works".

BTW, the case you want to deal with above, by extending my example
of your suggestion to the case where the lifetimes are different can be
dealt with by regrouping a different number of terms of like lifetimes in
the equation and then simply assigning them to a different set of attributes
of unlike lifetimes. The results will remain the same.

Cheers,

Ed Gerck

---

From: Tony Bartoletti <[email-masked]>
Date: Tue, 08 Jun 1999 12:01:18 -0700

>and expect that you have two attributes, in fact you would just one

At this point, we cannot tell from inspection whether A and B have any
components which are themselves 100% redundant.  In any case, A and B
are not 100% redundant.

>1/TB = 1/Tb1 + 1/Tb2 + 1/Tb3

If I did not impose these conditions, how could I have made the 
direct substitution in the equation for 1/TC above?

> "Assuming no redundancy, we allow 1/TC = 1/TA + 1/TB"

And I believe you maintain, in essense, that unless A and B are themselves
exactly redundant, "redundancy is not a problem in the equation."

Hence, I force A and B to be "partially redundant", and intentionally
drop the "Assuming no redundancy" to illustrate.
 
I mean that a1 is exactly redundant to b1, and a2 to b2.  They ARE the
same attributes, just repeated.  Although this does not make A and B
completely redundant, it DOES violate the original assumption, which
was exactly the point.  The formula 1/TC = 1/TA + 1/TB will not provide
the right answer, even though A and B are "different" attributes.

Unless A and B reveal their structure to us, so that we may identify
and eliminate duplicated components, we cannot apply such a formula.
And if the duplicated components [ai,bi] for some of the i, all exhibit
different expected lifetimes, there is no opportunity for the "grouping"
you mentioned, which says nothing more than SUM[C] to n terms = n*C for
a constant C.  They will be "groups of 1", and we are reduced to the
equation I gave:

  1/TC  =  1/TA + 1/TB - ( SUM[1/TCi] for ci in INTERSECT(A,B) )

The problem is in identifying this intersection, and not simply treating
attributes A and B as atoms.

___tony___

Tony Bartoletti                                             LL
IOWA Center                                              LL LL

---

From: David Chia <[email-masked]>
Date: Wed, 9 Jun 1999 12:41:31 +1000 (EST)

> attributes A and B as atoms.

In practice the relationship between A and B might not be simple nor
deterministic. The first order life equation for totally independent
attributes looks simple and straight forward. However, if there are
observable correlations among the attributes, second (or higher) order
life equations might be required and there is also the question of
which type of higher order life equation to be used. Furthermore, since
the relationship (if any) between the attributes most probably will not
be known deterministic, the coeffs of the first order term in the
higher order life equations most probably will not be ones.

If historical data (from the environment and in the context where the
cert is to be used) is available and the distributions of the attribute
lifetimes are roughly statistically normal, the expected cert life
equation might be able to be determined through stepwise regression
which will only include significant first or higher order terms.
Example of a simple second order life equation

   1/T = SUM a(i)/T(i) + SUM SUM b(i,j)/T(i)/T(j)
          i               i   j

where a(i) and b(i,j) are coeffs determined from the statistical regression.
b(i,j) is a measure of the 'intersection' you mentioned.

If historical data is not available, life equation determined from other
'similar' situation could be used for rough approximation but due care
should be taken for possible errors.

David Chia

---

From: David Chia <[email-masked]>
Date: Wed, 9 Jun 1999 13:02:37 +1000 (EST)

Sorry, forgot the constant term. Should be
 
     1/T = SUM a(i)/T(i) + SUM SUM b(i,j)/T(i)/T(j) + c
            i               i   j
 
where j>=i, a(i), b(i,j) and c are determined through statistical regression.
If environmental factors other than cert hold attributes are important,
they can also be included in the life equation.

David Chia

---

From: Stephen Kent <[email-masked]>
Date: Wed, 9 Jun 1999 09:51:30 -0400

Folks,

This really has gotten out of hand.  I admire the dedication of folks who
have been applying serious intellectual effort to creating a general
formula for cert lifetime as a function of the number of attributes, but
...

As the creator of "Steve's Rule of Revocation" I have to admit that I
generated the simple inverse square formula just to make a point, i.e.,
that, in general, adding attributes to a cert will shorten it's effective
lifetime and thus is generally a bad idea.

As some have pointed out, a general formula is hard, since one can cite
examples whwre added attributes are so closely linked to existing
attributes that the addition has no real effect on expected lifetime.
Also, contributors to this thread pointed out early on that it is the
attribute with the shortest expected lifetime that governs the lifetime of
the cert.  So, trying to express the lifetime in terms of a pure attribute
count seems futile.

Now, if I had to justify my original formula, I might try the following
analogy:

1. Adding attributes to a cert is a generally bad idea.  In vernacular
terms, it "sucks."

2. Looking to physics for an analogy, we note that, in the vernacular,
gravity "sucks."

3. The inverse square law applies to gravitational attraction between bodies.

4. Therefore, the effective lifetime of a certificate is inversely
propotional to the inverse square of the number of attributes :-)

Steve