Subject: [extensibility 2] How do we make Kerberos extensible?

From: Sam Hartman <[email-masked]>
Date: Wed, 5 Dec 2001 00:51:12 -0500 (EST)

Until recently, the Kerberos revisions work has focused on individual
extensions rather than examining extensibility as a general problem.
So far, our track record has not been very good.

Before the first draft of RFC 1510bis was released, people were
already proposing extensions: At IETF38, there was initial discussion
of adding checksums to Kerberos errors.  That discussion went forward
under the mistaken assumption that fields can be added to the end of
an ASN.1 sequence with no thought given to backward compatibility.
The proposal was integrated into the revisions draft, but recently
removed from section 5 by Tom, even though the feature is still
important .

Later at IETF39, the idea of ticket extensions was introduced.
Especially because tickets are stored by implementations for use in
later protocol exchanges, this extension mechanism has been very
problematic.  It was added again under the assumption that fields can
be added to ASN.1 sequences.  Later people believed that some
implementations might drop the ticket extensions; it was felt this
might be acceptable.  However still later we realized that
implementations would likely reject the ticket completely and so
ticket extensions have been removed from the ASN.1 pending a better
solution.

At several points, facets of the problem of authenticated cleartext
have been raised.  The krb-error checksums are one instance of this
problem.  Another early instance of this problem is the proposal to
include individual checksums of ticket extensions in the authorization
data associated with a ticket in order to integrity protect these
extensions.  Recently, WG members working with Packet Cable have
indicated interest in a checksum to protect the AS reply.  However
this issue has not been considered as a general problem.  People
looked at protecting the small part of the protocol that they were
concerned about.  There was not even a consistent method used to
protect all the protocol elements.

we also attempted to clarify how authorization data was used.  This
was first discussed at IETF39.  While much of this text is in the
current draft, few if any implementations comply with the text.  In
particular, both Heimdal and the MIT implementation seem to ignore
authorization data they do not understand rather than rejecting
authorization.  There is no negotiation mechanism provided to
distinguish between implementations that will reject ticket
authorization elements they do not understand from those who will
accept them.  While the discussion at IETF39 claimed interoperability
with old code is important, I do not believe that the clarification of
authorization data has been handled in a manner that is likely to lead
to both useful interoperability and availability of the new features.  

There are several general problems that seem to be common to many of
these attempts at extensibility.  One is that we have traditionally
not had a good understanding of ASN.1 and so we have failed to
understand how to extend it.  Another problem is that we have actively
tried to avoid negotiation of capabilities.  Unfortunately we have
paid a price for avoiding this negotiation: we want several features
that depend on knowing whether the other side of an interaction
supports the feature.  Without negotiation, implementers will have an
unpleasant choice when they consider using features like ticket
extensions.  They can use the feature, in which case their packet will
likely be discarded by old implementations, or they can maintain
interoperability and not use the new feature.

So since we value interoperability, we must consider how clients will
perform capability negotiation to inform the KDC of their capabilities
and learn the capabilities of the target service.  In addition if
there are reasonable ways we can achieve our goals while using ASN.1
correctly, we should do so.  We must at least use ASN.1 well enough
that we can maintain interoperability and have well-defined semantics.

Recently discussion in the working group has acknowledge that some
negotiation is necessary for some of the extensions being discussed.
However the trend has been to continue to treat each extensibility
feature as a distinct extension and to separate their specification
and design.  For example the most recent UTF8 discussion has focused
exclusively on UTF8 and the proposals to find out whether the KDC
supported UTF8 have been specific to that problem.  We believe this is
not the direction the working group should take for two reasons.
First, it has not worked well in the past.  But more importantly,
Kerberos's design does not lend itself to haphazard capability
negotiation.  Kerberos involves three parties: the client, the KDC and
the service.  For features like ticket extensions or UTF8 to be
useful, the client needs to discover the capabilities of the service
from the KDC.  Keeping state about the service's capabilities up to
date on the KDC is difficult.  We might be able to keep one or two
bits of state, but we do not want to record a generalized set of
service capabilities in the KDC entry.

			 Outline of Solution

So how can we do better with a general solution?  As we discussed in a
previous message, we have two primary goals.  We want to make sure
that there is sufficient extensibility for current proposals to be
implemented and enough extensibility for the IETF to make future
changes.  We also have a secondary goal of solving the authenticated
cleartext problem.

As we discussed in London we believe we can achieve these goals by
taking advantage of ASN.1 choices and constraints.  We can replace the
existing types with choices that wrap the types with appropriate
constraints.  The choice will have two arms, one for backward
compatibility and one for the new type.  The new types will include
appropriate typed holes for ticket extensions and for other PDUs that
can use this form of extensibility.  Since the choice is wrapping the
same common type, the old type will also have these fields, but the
ASN.1 constraint will make sure they are not included.  This should
meet our first goal.

In addition, ASN.1 extensibility markers are added to many types.  We
specify that new implementations must ignore elements added to
sequences while preserving them if the PDU is reencoded or included in
another PDU.  These extension markers can only be used by IETF action
because we need to carefully control assignment of tag numbers and
ordering within the sequences in order to maintain interoperability.
This simple change provides a powerful solution to the second goal.

It turns out that the choice types are made a bit more complex by the
desire to solve the authenticated cleartext problem.  We are fairly
certain that this is the simplest reasonable solution to that problem
but it does add complexity.  If the working group decides that the
authenticated cleartext problem does not need to be solved,this
complexity can be removed.  Unfortunately, if we decide to add a
solution to authenticated cleartext at any later point, it becomes
more complex than the current proposal.

Of course, so far I've only managed to reduce extensibility to the
previously unsolved problem of negotiation.  If we knew when we could
use the new types, we could get all the extensibility benefits.  The
hardest parte of negotiation is the initial AS-REQ message.  The
client needs to have a way to send a message that will be interpreted
by a new KDC as a new message potentially using extensibility like
UTF8 but is still readable by an old KDC.  I will describe that part
of the proposal in a later message along with the detailed mechanism
for handling authenticated cleartext; for the moment let us assume
that an acceptable solution exists for  the initial AS-REQ message.
In order to  produce an answer, the KDC needs to know if the service
supports new or old format messages; there is no way around having
some mechanism to get this one bit of state.  

By looking at the AS-REP, the client knows the properties of the KDC.
If the client received a new style AS-REP,  then the KDC supports the
new messages.  Note that the client should get a new AS-REP from a new
KDC whenever it sends a new AS-REQ, regardless of the capability of
the service.  However, whether  or not the ticket in the service  is
a new or old format ticket will depend on  the capabilities of the
service.  So, by reading the one AS-REP, the  client knows  the
capabilities both of the KDC and the  initial service.

So far, this sounds disturbingly like the overly complex Minneapolis
proposal.  However, rather than duplicating types as in that proposal,
we are using ASN.1 choices.  Certainly someone using off-the-shelf
ASN.1 tools could simply parse the choice and get whatever data is in
the type.  Since the inner types are the same except for ASN.1
constraints and authenticated cleartext handling, even implementations
that hand-code parsers can easily use a common parsing path to decode
both old and new types.  Also, gratuitous changes like changing the
krb-safe body were never part of this proposal.

The negotiation handling for other messages is simple.  You can use
new messages when talking to a new service as identified by the ticket
type.  The krb-error message can be new in response to a new message.

We believe this proposal provides a reasonable balance between solving
the extensibility problems the working group has been struggling with
since before the first draft of Kerberos revisions, while minimizing
unnecessary complexity.

---

From: "Douglas E. Engert" <[email-masked]>
Date: Wed, 05 Dec 2001 15:08:07 -0600

Sam, 
Are you proposing that these idea be incorporated into the revisions,
or are you proposing that they be in a separate document? 

The primary WG goal is to get the revisions out the door. But if 
there are too many open issues with the revisions and the use of ASN.1 
helps in this, and it could be added to the revisions in some short
time, then can you have something ready by the WG meeting on the changes
needed to the revisions?

-- 

 Douglas E. Engert  <[email-masked]>
 Argonne National Laboratory
 9700 South Cass Avenue
 Argonne, Illinois  60439 
 (630) 252-5444

---

From: Sam Hartman <[email-masked]>
Date: 05 Dec 2001 16:13:59 -0500

>>>>> "Douglas" == Douglas E Engert <[email-masked]> writes:

    Douglas> Sam, Are you proposing that these idea be incorporated
    Douglas> into the revisions, or are you proposing that they be in
    Douglas> a separate document?

We have been proposing adding something like this to revisions since
London.  I'm working on describing the details not included in message
two right now.  We already have a complete ASN.1 module ready but I'd
rather get the explanations to the list before raw ASN.1.

It's certainly our intent to have this ready by the WG meeting and to
have discussions with interested parties over the list or in
appropriate fora so that everyone understands the proposal by the WG
meeting.  We believe this will best facilitate judging WG consensus.

    Douglas> The primary WG goal is to get the revisions out the
    Douglas> door. But if there are too many open issues with the
    Douglas> revisions and the use of ASN.1 helps in this, and it
    Douglas> could be added to the revisions in some short time, then
    Douglas> can you have something ready by the WG meeting on the
    Douglas> changes needed to the revisions?

    Douglas> Sam Hartman wrote:
    >>  Until recently, the Kerberos revisions work has focused on
    >> individual extensions rather than examining extensibility as a
    >> general problem.  So far, our track record has not been very
    >> good.
    >> 
    >> already proposing extensions: At IETF38, there was initial
    >> discussion of adding checksums to Kerberos errors.  That
    >> discussion went forward under the mistaken assumption that
    >> fields can be added to the end of an ASN.1 sequence with no
    >> thought given to backward compatibility.  The proposal was
    >> integrated into the revisions draft, but recently removed from
    >> section 5 by Tom, even though the feature is still important .
    >> 
    >> Especially because tickets are stored by implementations for
    >> use in later protocol exchanges, this extension mechanism has
    >> been very problematic.  It was added again under the assumption
    >> that fields can be added to ASN.1 sequences.  Later people
    >> believed that some implementations might drop the ticket
    >> extensions; it was felt this might be acceptable.  However
    >> still later we realized that implementations would likely
    >> reject the ticket completely and so ticket extensions have been
    >> removed from the ASN.1 pending a better solution.
    >> 
    >> At several points, facets of the problem of authenticated
    >> cleartext have been raised.  The krb-error checksums are one
    >> instance of this problem.  Another early instance of this
    >> problem is the proposal to include individual checksums of
    >> ticket extensions in the authorization data associated with a
    >> ticket in order to integrity protect these extensions.
    >> Recently, WG members working with Packet Cable have indicated
    >> interest in a checksum to protect the AS reply.  However this
    >> issue has not been considered as a general problem.  People
    >> looked at protecting the small part of the protocol that they
    >> were concerned about.  There was not even a consistent method
    >> used to protect all the protocol elements.
    >> 
    >> we also attempted to clarify how authorization data was used.
    >> This was first discussed at IETF39.  While much of this text is
    >> in the current draft, few if any implementations comply with
    >> the text.  In particular, both Heimdal and the MIT
    >> implementation seem to ignore authorization data they do not
    >> understand rather than rejecting authorization.  There is no
    >> negotiation mechanism provided to distinguish between
    >> implementations that will reject ticket authorization elements
    >> they do not understand from those who will accept them.  While
    >> the discussion at IETF39 claimed interoperability with old code
    >> is important, I do not believe that the clarification of
    >> authorization data has been handled in a manner that is likely
    >> to lead to both useful interoperability and availability of the
    >> new features.
    >> 
    >> There are several general problems that seem to be common to
    >> many of these attempts at extensibility.  One is that we have
    >> traditionally not had a good understanding of ASN.1 and so we
    >> have failed to understand how to extend it.  Another problem is
    >> that we have actively tried to avoid negotiation of
    >> capabilities.  Unfortunately we have paid a price for avoiding
    >> this negotiation: we want several features that depend on
    >> knowing whether the other side of an interaction supports the
    >> feature.  Without negotiation, implementers will have an
    >> extensions.  They can use the feature, in which case their
    >> packet will likely be discarded by old implementations, or they
    >> can maintain interoperability and not use the new feature.
    >> 
    >> So since we value interoperability, we must consider how
    >> clients will perform capability negotiation to inform the KDC
    >> of their capabilities and learn the capabilities of the target
    >> service.  In addition if there are reasonable ways we can
    >> achieve our goals while using ASN.1 correctly, we should do so.
    >> We must at least use ASN.1 well enough that we can maintain
    >> interoperability and have well-defined semantics.
    >> 
    >> Recently discussion in the working group has acknowledge that
    >> some negotiation is necessary for some of the extensions being
    >> discussed.  However the trend has been to continue to treat
    >> each extensibility feature as a distinct extension and to
    >> separate their specification and design.  For example the most
    >> recent UTF8 discussion has focused exclusively on UTF8 and the
    >> proposals to find out whether the KDC supported UTF8 have been
    >> specific to that problem.  We believe this is not the direction
    >> the working group should take for two reasons.  First, it has
    >> not worked well in the past.  But more importantly, Kerberos's
    >> design does not lend itself to haphazard capability
    >> negotiation.  Kerberos involves three parties: the client, the
    >> KDC and the service.  For features like ticket extensions or
    >> UTF8 to be useful, the client needs to discover the
    >> capabilities of the service from the KDC.  Keeping state about
    >> the service's capabilities up to date on the KDC is difficult.
    >> We might be able to keep one or two bits of state, but we do
    >> not want to record a generalized set of service capabilities in
    >> the KDC entry.
    >> 
    >> Outline of Solution
    >> 
    >> So how can we do better with a general solution?  As we
    >> discussed in a previous message, we have two primary goals.  We
    >> want to make sure that there is sufficient extensibility for
    >> current proposals to be implemented and enough extensibility
    >> for the IETF to make future changes.  We also have a secondary
    >> goal of solving the authenticated cleartext problem.
    >> 
    >> As we discussed in London we believe we can achieve these goals
    >> by taking advantage of ASN.1 choices and constraints.  We can
    >> replace the existing types with choices that wrap the types
    >> with appropriate constraints.  The choice will have two arms,
    >> one for backward compatibility and one for the new type.  The
    >> new types will include appropriate typed holes for ticket
    >> extensions and for other PDUs that can use this form of
    >> extensibility.  Since the choice is wrapping the same common
    >> type, the old type will also have these fields, but the ASN.1
    >> constraint will make sure they are not included.  This should
    >> meet our first goal.
    >> 
    >> In addition, ASN.1 extensibility markers are added to many
    >> types.  We specify that new implementations must ignore
    >> elements added to sequences while preserving them if the PDU is
    >> reencoded or included in another PDU.  These extension markers
    >> can only be used by IETF action because we need to carefully
    >> control assignment of tag numbers and ordering within the
    >> sequences in order to maintain interoperability.  This simple
    >> change provides a powerful solution to the second goal.
    >> 
    >> It turns out that the choice types are made a bit more complex
    >> by the desire to solve the authenticated cleartext problem.  We
    >> are fairly certain that this is the simplest reasonable
    >> solution to that problem but it does add complexity.  If the
    >> working group decides that the authenticated cleartext problem
    >> does not need to be solved,this complexity can be removed.
    >> Unfortunately, if we decide to add a solution to authenticated
    >> cleartext at any later point, it becomes more complex than the
    >> current proposal.
    >> 
    >> Of course, so far I've only managed to reduce extensibility to
    >> the previously unsolved problem of negotiation.  If we knew
    >> when we could use the new types, we could get all the
    >> extensibility benefits.  The hardest parte of negotiation is
    >> the initial AS-REQ message.  The client needs to have a way to
    >> send a message that will be interpreted by a new KDC as a new
    >> message potentially using extensibility like UTF8 but is still
    >> readable by an old KDC.  I will describe that part of the
    >> proposal in a later message along with the detailed mechanism
    >> for handling authenticated cleartext; for the moment let us
    >> assume that an acceptable solution exists for the initial
    >> AS-REQ message.  In order to produce an answer, the KDC needs
    >> to know if the service supports new or old format messages;
    >> there is no way around having some mechanism to get this one
    >> bit of state.
    >> 
    >> By looking at the AS-REP, the client knows the properties of
    >> the KDC.  If the client received a new style AS-REP, then the
    >> KDC supports the new messages.  Note that the client should get
    >> a new AS-REP from a new KDC whenever it sends a new AS-REQ,
    >> regardless of the capability of the service.  However, whether
    >> or not the ticket in the service is a new or old format ticket
    >> will depend on the capabilities of the service.  So, by reading
    >> the one AS-REP, the client knows the capabilities both of the
    >> KDC and the initial service.
    >> 
    >> So far, this sounds disturbingly like the overly complex
    >> Minneapolis proposal.  However, rather than duplicating types
    >> as in that proposal, we are using ASN.1 choices.  Certainly
    >> someone using off-the-shelf ASN.1 tools could simply parse the
    >> choice and get whatever data is in the type.  Since the inner
    >> types are the same except for ASN.1 constraints and
    >> authenticated cleartext handling, even implementations that
    >> hand-code parsers can easily use a common parsing path to
    >> decode both old and new types.  Also, gratuitous changes like
    >> changing the krb-safe body were never part of this proposal.
    >> 
    >> The negotiation handling for other messages is simple.  You can
    >> use new messages when talking to a new service as identified by
    >> the ticket type.  The krb-error message can be new in response
    >> to a new message.
    >> 
    >> We believe this proposal provides a reasonable balance between
    >> solving the extensibility problems the working group has been
    >> struggling with since before the first draft of Kerberos
    >> revisions, while minimizing unnecessary complexity.

    Douglas> --

    Douglas>  Douglas E. Engert <[email-masked]> Argonne National
    Douglas> Laboratory 9700 South Cass Avenue Argonne, Illinois 60439
    Douglas> (630) 252-5444