Subject: RE: UTF-8, ASN.1, and our past sins

From: "Paul Leach" <[email-masked]>
Date: Tue, 20 Nov 2001 10:53:37 -0800

> -----Original Message-----
> From: Nicolas Williams [mailto:[email-masked]]
> Sent: Monday, November 19, 2001 6:59 PM
> To: Paul Leach; Jeffrey Hutzelman; Johan Danielsson
> Cc: [email-masked]
> Subject: Re: UTF-8, ASN.1, and our past sins
> 
> 
> Heh. I thought that's what yall say was discussed in
> Pittsburg only to have the IESG and the ITU (presumably) get 
> up in arms about this and yell at the WG.

I don't think we phrased it correctly in Pittsburg or London.

I think this is a language lawyer issue -- we're not telling them we're
ignoring or perverting ASN.1 -- we're phrasing it so that we don't even
claim that it is ASN.1. In fact, we will explicitly say that it is NOT
ASN.1 (just very similar).

If they gripe more, we can go thru the RFC and change every instance of
"CHOICE" to "ONEOF" and "SEQUENCE" to "LISTOF" and change the bracketing
convention, etc. etc. to create a totally new metalanguage. :-) And
define it semantics with reference to ASN.1 almost all the time. In such
a way that all you do is reverse the syntactic transformations and all
the commercial tools will work. We could even include the SED script in
the appendix :-) :-).

> 
> Hey, it sure is easier to say: "for the purposes of
> Kerberos5, GeneralString shall be taken to mean "an unescaped 
> string of UTF-8 characters in network byte order"" than it is 
> to do the CHOICE thing.
> 
> There is an advantage to go with incompatible protocol
> changes, so long as old clients and old KDCs with with new 
> KDCs and clients, respectively, and so long as old/new 
> clients work with new/old services (yes, old software would 
> be unable to talk to principals with UTF-8 names or realsm 
> with UTF-8 names, save for vendors like MS, that already lie 
> about GeneralString). The advantage is that a bunch of other 
> useful changes that also break protocol compatibility could 
> be bundled in one go.
> 
> To me it's a coin toss -- I'm not a Kerberos implementor. I'm
> a user, sysadmin, deployer. As such I think that an 
> incompatible protocol change is livable given the constraints 
> above. Perhaps we should hear more from my counterparts 
> elsewhere about this.
> 
> So, if you tell the IESG and the ITU to shove it, will the sky fall?

Yes, but that's isn't what I propose we do.

Paul

---

From: Ken Raeburn <[email-masked]>
Date: 21 Nov 2001 00:33:47 -0500

Honestly, I do not expect anyone -- at least, not any current vendors
-- to implement precisely the proposed revision with no extensions in
the string handling area.  If we put something like Tom's
"GeneralString (VisibleString)" in the spec, if that's the one that
limits it to ASCII, then we have a ASN.1 compliant specification which
describes the the protocol limited to the subset of names where we
should be interoperable now.  And by extending it in this area where
we've all effectively had incompatible extensions before, we can keep
compatibility with our existing deployments while we figure out how to
make the move to UTF8String.

I thought we had some agreement that this was a hard problem that
would likely take quite some time to resolve to everyone's
satisfaction, and therefore we'd put in the minimal necessary hooks
now and deal with the rest later.  Trying to force in one partial
solution like John's at this point, and one that (according to some
people) isn't even compliant with the specs and is different from what
other implementations are doing, means the non-Microsoft vendors have
to deal with the transition and backwards compatibility issues now,
precisely what we were hoping to avoid.

We're not trying to target Microsoft's implementation to break.  But
it seems likely we're going to have to break things a little for
*everyone* to get where we want to be.  The issue is how to best
arrange for a clean upgrade path for *everyone*, not just the poor
saps who didn't do it Microsoft's way.

Is it possible to construct a non-ASCII string -- maybe, a JIS X
character -- for which there is a DER GeneralString encoding that does
not use a UTF-8 escape, and also a DER GeneralString encoding that
does use UTF-8 (assuming for the moment that escape itself is
permitted)?  If so, we have the interesting problem of a string with
multiple "distinguished" encodings.  If both are allowed under DER,
pass phrases and principal names no longer have unique encodings, and
I believe the Kerberos spec -- and certainly much existing code --
assumes that they do.

Aside from GeneralString, I think all our "interesting" ASN.1
compliance issues come not from compliance with ASN.1 itself but with
variations from the DER, or poor but compliant use of OPTIONAL fields,
described as ways in which an implementation can "be liberal" without
being incompatible.  Is there anything besides the GeneralString
handling that indicates any non-compliance with ASN.1 itself and
drives this "KSN.1" idea?

Ken