Subject: Re: UTF-8, ASN.1, and our past sins

From: Clifford Neuman <[email-masked]>
Date: Mon, 19 Nov 2001 16:46:19 -0800 (PST)

I like the idea, but want to hear what Tom has to say.  One of the
issues with ASN.1 compatability is to simplify life for those that are
using ASN.1 development tools and this might not address that
problem.  But the points that you make are correct.  

Cliff

---

From: "Paul Leach" <[email-masked]>
Date: Mon, 19 Nov 2001 16:32:24 -0800

So, I'm going to propose something maybe a bit radical.

No one can tell us (an IETF WG) what metalanguage to use to describe our
protocol. They can legitimately gripe if we use an existing one
incorrectly ("perversion" comes to mind as an adjective).

Section 5.1 of 1510bis
(www.isi.edu/people/bn/krb-revisions/draft-kerberos-010302.html is what
I'm looking at) describes our perversions, in areas other than
GeneralString.

What if we change it to say something along the following lines: "The
Kerberos protocol uses the KASN.1 metalanguage to describes its PDUs.
This language has the same syntax as ASN.1, but the following sections
(5.1.1 to 5.1.7) describe where the semantics differs from ASN.1. These
differences are for historical reasons, not from any disagreement with
the ASN.1 standard, but they have become so entrenched that it would
present major interoperability and migration problems to root them out
at this point. They also do not pr"

Then change each of the sections 5.1.1 to 5.1.6 to reflect this point of
view, and add a section 5.1.7, which today does not exist, that explains
that "GeneralString" means UTF-8.

Let me be perfectly clear -- I do not claim that the above is the exact
correct wording, and I am sure I am not taking some problems into
account. I am trying to suggest a direction that I don't recall others
suggesting, which may permit us to live with our sins.

Then, we can go to full fledged pure ASN.1 in Kerberos V6, as it seems
that trying to do that in V5 is essentially a new protocol anyway (from
an interop and migration point of view).

Paul

---

From: Sam Hartman <[email-masked]>
Date: 19 Nov 2001 21:57:18 -0500

>>>>> "Paul" == Paul Leach <[email-masked]> writes:

    Paul> So, I'm going to propose something maybe a bit radical.  No
    Paul> one can tell us (an IETF WG) what metalanguage to use to
    Paul> describe our protocol. They can legitimately gripe if we use
    Paul> an existing one incorrectly ("perversion" comes to mind as
    Paul> an adjective).

So, when you are proposing something this radical it helps if you
explain what advantage your proposal has.  It's not clear to me what
we gain by adopting this proposal (other than saying general string is
UTF8) in simplifying getting Kerberos revisions to proposed standard.  I'd be much more likely to agree with such a change if I understand how it helps us.

Note that we discussed tagging UTF8 strings as general strings in
London.  There are several problems.  First, we apparently have a
customer base in the DCE community that needs a way to know whether
the KDC/clients support UTF8 or whether they use just-send-8.  Your
proposal does not provide a solution for that community and  they seem likely to block consensus if we do not meet their needs.

Second the IESG chair stood up at the London meeting and indicated
that misusing ASN.1 (in particular tagging UTF8 strings as general
strings) would create significant political friction between the ITU
and IETF.  I don't think he said we definitely couldn't do this
tagging, but he seemed unlikely to accept a solution that violated
ASN.1 while viable other solutions were on the table.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 19 Nov 2001 21:59:00 -0500

Heh. I thought that's what yall say was discussed in Pittsburg only to
have the IESG and the ITU (presumably) get up in arms about this and
yell at the WG.

Hey, it sure is easier to say: "for the purposes of Kerberos5,
GeneralString shall be taken to mean "an unescaped string of UTF-8
characters in network byte order"" than it is to do the CHOICE thing.

There is an advantage to go with incompatible protocol changes, so long
as old clients and old KDCs with with new KDCs and clients,
respectively, and so long as old/new clients work with new/old services
(yes, old software would be unable to talk to principals with UTF-8
names or realsm with UTF-8 names, save for vendors like MS, that
already lie about GeneralString). The advantage is that a bunch of
other useful changes that also break protocol compatibility could be
bundled in one go.

To me it's a coin toss -- I'm not a Kerberos implementor. I'm a user,
sysadmin, deployer. As such I think that an incompatible protocol
change is livable given the constraints above. Perhaps we should hear
more from my counterparts elsewhere about this.

So, if you tell the IESG and the ITU to shove it, will the sky fall?

Cheers,

Nico

--
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

Visit our website at http://www.ubswarburg.com

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 19 Nov 2001 22:18:23 -0500 (EST)

> So, if you tell the IESG and the ITU to shove it, will the sky fall?

Well, I don't know about the sky falling, but somehow I consider it
extraordinarily unlikely that Kerberos will progress along the standards
track without IESG support.  Not impossible, but nearly so.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA

---

From: Sam Hartman <[email-masked]>
Date: 19 Nov 2001 22:32:45 -0500

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> writes:

    Nicolas> Heh. I thought that's what yall say was discussed in
    Nicolas> Pittsburg only to have the IESG and the ITU (presumably)
    Nicolas> get up in arms about this and yell at the WG.

    Nicolas> Hey, it sure is easier to say: "for the purposes of
    Nicolas> Kerberos5, GeneralString shall be taken to mean "an
    Nicolas> unescaped string of UTF-8 characters in network byte
    Nicolas> order"" than it is to do the CHOICE thing.

    Nicolas> There is an advantage to go with incompatible protocol
    Nicolas> changes, so long as old clients and old KDCs with with

So, I didn't quite understand your constraints, but it seems clear
especially because of cross-realm concerns that all possible
combinations of old and new clients, KDCs and servers need to work
together.  Your idea of requiring all KDCs in a realm to behave the
same certainly seems like something worth considering.

ALso note that based on the reactions of the working group to Tom's
proposed new message formats it seems clear that the WG is very
unlikely to accept a protocol where the old and new messages cannot be
parsed by the same codepath.  That limits the kinds of incompatible
protocol changes you can make.

---

From: Tom Yu <[email-masked]>
Date: 20 Nov 2001 09:42:03 -0500

>>>>> "Paul" == Paul Leach <[email-masked]> writes:

Paul> So, I'm going to propose something maybe a bit radical.  No one
Paul> can tell us (an IETF WG) what metalanguage to use to describe
Paul> our protocol. They can legitimately gripe if we use an existing
Paul> one incorrectly ("perversion" comes to mind as an adjective).

Others have responded already with respect to IESG and ITU issues.

Paul> Section 5.1 of 1510bis
Paul> (www.isi.edu/people/bn/krb-revisions/draft-kerberos-010302.html
Paul> is what I'm looking at) describes our perversions, in areas
Paul> other than GeneralString.

I believe you're not looking at the most recent draft by using that
URL.  That was the version for Minneapolis.  For some reason Cliff
hasn't updated the link of the "combined draft".  Look either at the
individual sections or the ASCII version on the internet-draft
repository.

It doesn't describe our perversions.

Nowhere in sections 5.1.1 through 5.1.6 of the 010302 draft is there
*any* mention of our use of ASN.1 notation to mean something besides
what the ASN.1 standard itself specifies.  Section 5.1 mostly provides
*compatibility* notes for implementors.  *Implementations* of RFC 1510
have violated the ASN.1 specification, and we're changing the kerberos
protocol spec in well-defined ways to accomodate that.

In recent drafts, the only non-compliant things we do with ASN.1 are
minor violations of the extensibility model, such as adding ASN.1
constraints on types to reflect what implementations are actually
doing, and doing things such as replacing a type with a CHOICE
containing the original type as one of its alternatives.

I would like to know where you believe section 5.1 of the 010302
document describes our perversions of the ASN.1 spec, rather than
describing how implementations have violated the spec, or describing
how RFC 1510 has semantic ambiguities that can confuse implementors.

Paul> What if we change it to say something along the following lines:
Paul> "The Kerberos protocol uses the KASN.1 metalanguage to describes
Paul> its PDUs.

[...]

Paul> Then change each of the sections 5.1.1 to 5.1.6 to reflect this
Paul> point of view, and add a section 5.1.7, which today does not
Paul> exist, that explains that "GeneralString" means UTF-8.

As others have said, the "GeneralString means UTF-8" language was
proposed in Minneaopolis, and shot down.  It was shot down again in
London by the IESG.

Paul> Let me be perfectly clear -- I do not claim that the above is
Paul> the exact correct wording, and I am sure I am not taking some
Paul> problems into account. I am trying to suggest a direction that I
Paul> don't recall others suggesting, which may permit us to live with
Paul> our sins.

Paul> Then, we can go to full fledged pure ASN.1 in Kerberos V6, as it
Paul> seems that trying to do that in V5 is essentially a new protocol
Paul> anyway (from an interop and migration point of view).

Some of the features that people are demanding, such as ticket
extensions, are essentially a new protocol anyway.  It's just a
question of how obvious you're making it.  Of course, it's also
possible to view them as the same protocol, but with minor changes.

Even if there are seemingly minor changes, these changes require a
migration strategy, since not all implementations will do the right
things with them, due to a *lack* of specification in RFC 1510 with
respect to exception behavior of implementations.  My changes to
section 5 have been done largely in recognition of the compatibility
problems that result from feature changes that *other* *people* have
requested and that they consider to be non-negotiable.  These changes
to section 5 have been undertaken with an eye towards making it
possible to implement a migration strategy without forcing the use of
custom ASN.1 parsers.

Paul, since Microsoft is, to my knowledge, using a commercial ASN.1
compiler and tools, I would imagine that you of all people would be in
favor of a strategy that doesn't force the use of custom ASN.1 parsing
code.

---Tom

---

From: Tom Yu <[email-masked]>
Date: 20 Nov 2001 09:59:32 -0500

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> writes:

Nicolas> Changing Kerberos on the wire would strengthen the
Nicolas> cryptographic security of Kerberos though, by allowing nonces
Nicolas> larger than 32 bits.  This I think is a very important
Nicolas> consideration. I'd like to hear more about the ways in which
Nicolas> Kerberos becomes easier to cryptanalyze as time goes by
Nicolas> because of such limitations as are currently wired into it.

Where has this been discussed?  I do not recall hearing about changes
to nonces.

---Tom

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 20 Nov 2001 09:44:59 -0500

> track without IESG support.  Not impossible, but nearly so.

My guess is that the IESG would like to see the WG do its best to devise
solutions and analyze their pros and cons, only then might the IESG agree
to something it's already said it won't accept.

Personally, I find the CHOICE idea is very seductive and would rather
see that happen. But I also think that, if I have KDCs that support the
new KerberosString and old win2k and MIT clients that don't, that I
might end up having to upgrade the clients or hack my KDCs to support
UTF-8 and plain 8-bit chars in GeneralString. Which makes me wonder
just how necessary the CHOICE approach really is.

Changing Kerberos on the wire would strengthen the cryptographic
security of Kerberos though, by allowing nonces larger than 32 bits.
This I think is a very important consideration. I'd like to hear more
about the ways in which Kerberos becomes easier to cryptanalyze as time
goes by because of such limitations as are currently wired into it.

Cheers,

Nico

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 20 Nov 2001 10:18:22 -0500

I just searched and searched and found nothing. I made it up! I confess!

My bad,

Nico

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 20 Nov 2001 10:02:21 -0500

Somewhere on this list.

I thought that was mentioned.

If I made it up, I apologize.

Nico

---

From: Sam Hartman <[email-masked]>
Date: 20 Nov 2001 11:30:43 -0500

There has been a lot of discussion about what we're trying to do with
UTF8.  I want to see if we have a consensus that if we find a way of
achieving these compatibility guarantees  that we have met people's
needs with regard to compatibility.

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> wr
ites:

    Nicolas> Personally, I find the CHOICE idea is very seductive and
    Nicolas> would rather see that happen. But I also think that, if I
    Nicolas> have KDCs that support the new KerberosString and old
    Nicolas> win2k and MIT clients that don't, that I might end up
    Nicolas> having to upgrade the clients or hack my KDCs to support
    Nicolas> UTF-8 and plain 8-bit chars in GeneralString. Which makes
    Nicolas> me wonder just how necessary the CHOICE approach really
    Nicolas> is.

I'd like to define five classes of Kerberos entities.  An entity can
be a client, service or KDC.  The first two classes are
old-just-send-8 and old-utf8.  Microsoft is an example of old-utf8 and
MIT is currently an example of old-just-send-8.  MIT might not be an
example of old-just-send-8 if we adopt the suggestion of limiting
principal names for new principals.

The old signifies that the implementation is using the current
ambiguous protocol.

Ideally after some protocol change we want every Kerberos entity to be
in the class new.  The class new simply sends UTF8 strings tagged as
specified by the new protocol.  One way to do that would be to require
that new entities are incompatible with both old-just-send-8 entities
and old-utf8 entities.  I think this sort of flag day is absolutely
unacceptable to everyone involved, so we need something more complex.

Let's consider the entity class new-utf8.  This class can interoperate
both with new and with old-utf8.  Similarly there is a class
new-just-send-8 which can interoperate with new and old-just-send-8.
We would like it to be the case that when speaking the new protocol,
new, new-utf8 and new-just-send-8 are indistinguishable and work
together.

We propose to find a solution such that new-utf8 entities can exist
and can interoperate with old-utf8 entities.  Similarly we propose to
find a solution such that new-just-send-8 entities can exist and can
interoperate with old-just-send-8.

We have been assuming that just as old-utf8 and old-just-send-8 can
only interoperate for US ASCII, old-utf8 and new-just-send-8 will only
interoperate for US ASCII.  Similarly we have been assuming that
old-just-send-8 and new-utf8 will only interoperate for US ASCII.  We
do not believe can find a solution that solves this problem because we
believe it would involve knowing whether a string sent by an old client
was UTF8 or just-send-8.  The entire reason we are in this mess is
that we cannot solve that problem.

We believe that the choice or something like it is a critical part of
allowing the classes new-utf8 and new-just-send-8 to exist because it
allows them to distinguish  whether they are communicating with a new
entity or with an old entity.  If they find they are communicating
with  and old entity they can only assume it is the same type of old
entity that they support, but at least they know it is an old entity.
This is critically important for new-just-send-8 entities.  The choice
is seductive because  when an old entity sends to a new entity it
automatically tags itself as an old entity, while not requiring
significant changes to parsers.

Note however that the choice is only part of a solution.  New clients
need to know whether to send the old or new arm of the choice.  Some
negotiation is required.  We understand that.  Tom Ken and I have been
working on this within the context of the general extensibility work
we briefly presented at London and said we would flesh out for the
next IETF.  We have been making progress on that and should have a set
of alternatives to discuss for negotiation soon, possibly even today.

---

From: [email-masked] (Johan Danielsson)
Date: 20 Nov 2001 20:27:21 +0100

Tom Yu <[email-masked]> writes:

> compiler and tools, I would imagine that you of all people would be
> in favor of a strategy that doesn't force the use of custom ASN.1
> parsing code.

Isn't the whole issue about Microsoft (and others for that matter) not
using ASN.1 properly?

/Johan