Subject: RE: UTF-8, ASN.1, and our past sins

From: "Paul Leach" <[email-masked]>
Date: Tue, 20 Nov 2001 10:00:24 -0800

OK, back to new-old coexistence. This is not complete, but hopefully it
lays out a workable strategy.

I'm going to assume for a second that names in different encodings are
unlikely to conflict -- e.g., in any given realm "fred" in EBCDIC will
not be "mary" in ASCII where "mary" also has an entry in the KDC for
that realm. (In some cases, "fred" in one encoding will be "fred" in
another, because many encodings share code points for the base Latin
characters.)

OK, so now consider a new client for an EBCDIC system talking to an old
KDC, and its AS_REQ. The client asks for cname "fred" in UTF8 with
password "foo". Either the KDC is new, and understands UTF8, and this
works, or it is old and it works because the KDC understands ASCII and
"fred" is ASCII and UTF8, or it fails because the KDC is old and only
understands EBCDIC. In the last case, the client retries the AS_REQ in
EBCDIC, and all is golden (save for an auth failure audit event at the
KDC, perhaps...). The client remembers the encoding that worked for the
AS_REQ, and uses it in its TGS_REQs. 

Now for a client talking to a new KDC on an EBCDIC system. The new KDC
looks up the cname in the AS-REQ in UTF8; if there's a match and the
passwords check, all is cool. If not then it tries again after
converting to EBCDIC. This works for both old EBCDIC client and new UTF8
client.

Similar results should hold for TGS_REQs to servers in the same realm.
(The client will have to encode both cname and sname in the same way.) A
new KDC would have to mark each entry with the encoding used by these
server, and convert to that encoding in the tickets it issues.

One case that doesn't work is client A in realm B and server C in realm
D, where B and D are old and use different character encodings. But that
never worked and never will -- just like if A and B used different
encodings. However, if A is new, but B, C, and D are all old and the
same encoding, that would work.

There are more cases to check. I think they'll all go through, except
ones that are clearly impossible (unresolvable old-old conflicts in one
or more pairs of parties).

> -----Original Message-----
> From: Paul Leach 
> Sent: Monday, November 19, 2001 7:24 PM
> To: Sam Hartman
> Cc: Nicolas Williams; Jeffrey Hutzelman; Johan Danielsson; 
> [email-masked]
> Subject: RE: UTF-8, ASN.1, and our past sins
> 
> 
> The biggest advantage is that it gets us off the hook about misusing
> ASN.1 -- we would not be using it any more, we would be using 
> KSN.1. It
> appears there are lots of ways (see section 5) where we misuse ASN.1,
> not just the GeneralString problem, where UTF8 and "just-send-8" are
> probably equally evil.
> 
> I intended my comment in that regard to be independent of the solution
> to the character set problem -- that's why I said up front that I
> expected that not everything I said was correct.
> 
> With respect to the current state of the protocol, its pretty 
> clear that
> clients and servers that don't agree on the character set 
> encoding won't
> interoperate, and there is nothing we can do about that -- 
> the existing
> code is what it is. That means that no matter what we do, someone gets
> screwed -- the most we can do is try for the greatest good for the
> greatest number.
> 
> If people are going to change their code, then it seems to me like the
> cleanest solution is that everyone code as UTF8. That has the 
> advantage
> that all the people who currently use 7 bit ASCII don't have 
> to change.
> 
> Then there's the migration issue: how can new clients can coexist with
> old KDCs and vice versa coexist.
> That seems largely independent of what we choose as "the new 
> way". I'll
> address that in another message -- I'm late for a meeting right now.
> 
> > -----Original Message-----
> > From: Sam Hartman [mailto:[email-masked]] 
> > Sent: Monday, November 19, 2001 6:57 PM
> > To: Paul Leach
> > Cc: Nicolas Williams; Jeffrey Hutzelman; Johan Danielsson; 
> > [email-masked]
> > Subject: Re: UTF-8, ASN.1, and our past sins
> > 
> > 
> > >>>>> "Paul" == Paul Leach <[email-masked]> writes:
> > 
> >     Paul> So, I'm going to propose something maybe a bit 
> radical.  No
> >     Paul> one can tell us (an IETF WG) what metalanguage to use to
> >     Paul> describe our protocol. They can legitimately 
> gripe if we use
> >     Paul> an existing one incorrectly ("perversion" comes to mind as
> >     Paul> an adjective).
> > 
> > So, when you are proposing something this radical it helps if 
> > you explain what advantage your proposal has.  It's not clear 
> > to me what we gain by adopting this proposal (other than 
> > saying general string is
> > UTF8) in simplifying getting Kerberos revisions to proposed 
> > standard.  I'd be much more likely to agree with such a 
> > change if I understand how it helps us.
> > 
> > Note that we discussed tagging UTF8 strings as general 
> > strings in London.  There are several problems.  First, we 
> > apparently have a customer base in the DCE community that 
> > needs a way to know whether the KDC/clients support UTF8 or 
> > whether they use just-send-8.  Your proposal does not provide 
> > a solution for that community and  they seem likely to block 
> > consensus if we do not meet their needs.
> > 
> > Second the IESG chair stood up at the London meeting and 
> > indicated that misusing ASN.1 (in particular tagging UTF8 
> > strings as general
> > strings) would create significant political friction between 
> > the ITU and IETF.  I don't think he said we definitely 
> > couldn't do this tagging, but he seemed unlikely to accept a 
> > solution that violated ASN.1 while viable other solutions 
> > were on the table.
> > 
> > 
> > 
>

---

From: Sam Hartman <[email-masked]>
Date: 20 Nov 2001 13:11:51 -0500

>>>>> "Paul" == Paul Leach <[email-masked]> writes:

    Paul> This is a multi-part message in MIME format.
    Paul> ------=_NextPart_000_000F_01C171AA.4F21D1C0 Content-Type:
    Paul> text/plain; charset="us-ascii" Content-Transfer-Encoding:
    Paul> 7bit

    Paul> OK, back to new-old coexistence. This is not complete, but
    Paul> hopefully it lays out a workable strategy.

At this point I'm more interested in whether you agree with my
compatibility goals than a particular implementation to achieve them.
Are we on the same page with regard to what we want out of character
set compatibility?

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 20 Nov 2001 13:22:42 -0500

The crucial issue is: how do "new" clients detect whether the KDCs for a
given realm are also "new". A way to do this can be defined in a way
that does not break old wire compatibility, no doubt.

New clients need only probe KDC new-ness in AS exchanges and should
cache prior results.

New clients can tell whether a service is "new" by the KerberosString
CHOICE in their service tickets (if the tag is GeneralString, then the
service is "old").

New clients can tell whether other realm's KDC are "new" in the same
way, after all those KDCs are just services.

New services could handle new and old clients just as well -- the
KerberosString CHOICE in the clients' tickets identifies the clients'
(and/or KDCs') capabilities.

So far so good.

Old clients can only talk the old protocol, so all KDCs it might talk to
had better support the old protocol (with the old constraints too). Old
clients can reference principal names that are non-ASCII with the same
constaints as today (i.e., if the client/KDC/service are all using the
same encoding system). Old clients, therefore, may not be able to access
new realms with UTF-8 names, if those clients don't support
GeneralString-as-UTF-8. And so on...

Now, as for mixtures of old clients using different 8+-bit encodings for
GeneralString, they'd be left in the same lurch as today.

Old services could handle new and old clients just as well, provided
that the old services' realm's KDCs know that they are old services.

Now. I think we can summarize the requirements to make this work as
follows:

 - define client / KDC protocol new-ness probe mechanism (an "old"
   AS-REP from a "new" KDC must indicate somehow that the KDC supports
   "new" behaviour) [NOTE: the technical details of this have not been
   discussed yet]

 - require that realms' KDCs all be old or all be new

 - allow new clients to cache realms' KDC new-ness status

 - require that KDCs track services' new-ness status (pain-in-the-neck?)

Doable.

If the WG goes with KASN.1 then there should probably a discussion as
to how just-send-8 and utf-8 implementation should interoperate and how
a just-send-8-to-utf-8 transition should happen.

Cheers,

Nico

--
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

Visit our website at http://www.ubswarburg.com

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.

---

From: Sam Hartman <[email-masked]>
Date: 20 Nov 2001 14:11:27 -0500

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> writes:

    Nicolas> The crucial issue is: how do "new" clients detect whether
    Nicolas> the KDCs for a given realm are also "new". A way to do
    Nicolas> this can be defined in a way that does not break old wire
    Nicolas> compatibility, no doubt.

    Nicolas> New clients need only probe KDC new-ness in AS exchanges
    Nicolas> and should cache prior results.

Yes, and I don't think this list is a great forum to explore technical
proposals that haven't been fully thought through.  It is a great
forum (and is defined to be the only forum) for getting consensus on
such proposals and is also the right place to present proposals for
review.