Subject: Re: [idn] Re: 7 bits forever!

From: "John Stracke" <[email-masked]>
Date: Tue, 2 Apr 2002 09:54:32 -0500

D. J. Bernstein wrote:

>It is not reasonable to demand that this
>7-bit garbage be allowed to remain on the network forever, at the
>expense of all future implementors and users.

OK, you're obviously in a better position than I am to appreciate the cost 
to implementors.  But users? Suppose you could streamline qmail by 
dropping support for talking to 7-bit servers.  How do you convince a 
postmaster to install it? The bandwidth cost of having to use 7-bit 
encoding is real, but small; the cost of losing connectivity to sites is 
potentially high.  Worse, it's almost impossible to put an upper bound on 
the cost of lost connectivity.  For a company, it could mean losing 
messages (and sales) from important customers; businesses have failed over 
such things.

For commercial implementors, it's a no-brainer: dropping 7-bit support 
will lose customers, so don't do it.  For open-source implementors such as 
yourself, naturally, things are less clear-cut.

/==============================================================\
|John Stracke                    |Principal Engineer           |
[email-masked]   |Incentive Systems, Inc.      |
|http://www.incentivesystems.com |My opinions are my own.      |
|==============================================================|
|"This [OLE] isn't an API, this is a shaggy dog story in C!" --|
|Unknown                                                       |
\==============================================================/

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 09:56:44 -0500

I think the issue here is how we can plan a better transition than what we
have done for SMTP.
That is, if we start thinking about "8+ everywhere" instead of "ACE
everywhere" scenario, then how we can get there from here. That is the
question for the IDN group I believe.
Edmon

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 10:29:00 -0500

I am not saying its going to be non-backwards compatible.
Ok, I have mentioned a possible migration path before, let me try it again
(btw, it is funny that everytime I put this out, no one really debate the
merits of it... but then a spurt of discussion on UTF8 breaksout)

1. IDNA clients
    - the client does ACE and display it properly for the user
    - servers (including DNS and mail and http, etc.) does NOT upgrade for
ACE.  Even administration side is kept as is, allowing leakage
    - IDNA clients MUST also be able to decode IDN(UTF8)/EDNS responses

2. IDN(UTF8)/EDNS DNS Server upgrade
    - DNS servers will now accept both ACE (as usual) and IDN(UTF8)/EDNS
requests
    - ACE requests will yield Both an ACE response + an IDN(UTF8)/EDNS
response
    - IDNA clients continue to send in ACE but will start to see UDNA
responses

3. IDN(UTF8)/EDNS for clients & everywhere
    - IDNA clients begin to obsolete, new versions of application software
comes with IDN(UTF8)/EDNS built in.
    - In case DNS servers are not upgraded yet, clients will continue to use
the IDNA client, thereby also creating an incentive for server side to
upgrade
    - Other applications/servers upgrade to IDN(UTF8)/EDNS to accept UTF8
domains, administrators might continue to see ACE leakage if the client
still uses IDNA to send requests, but UDNA requests will be managed in UTF8

4. IDN(UTF8)/EDNS
    - Slowly but certainly, we will move towards full UTF8 because
administrators will likely want to deal with UTF8 than ACE, especially for
say those admins in China who are dealing with IDNs frequently.  Added to
the urge from the user community to upgrade the server as they upgrade their
clients to IDN(UTF8)/EDNS.

Thoughts anyone?

Edmon

----- Original Message -----
From: "John Stracke" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, April 02, 2002 10:08 AM
Subject: Re: [idn] Re: 7 bits forever!

> >everywhere" scenario, then how we can get there from here.
>
> Same question: if you have a non-backwards-compatible server, how do you
> get people to use it?
>
> /========================================================\
> [email-masked]   |Incentive Systems, Inc.|
> |http://www.incentivesystems.com |My opinions are my own.|
> |========================================================|
> |Speak softly and carry an Illudium Q-32 Explosive Space |
> |Disintegrator.                                          |
> \========================================================/
>
>

---

From: [email-masked]
Date: Tue, 02 Apr 2002 11:09:34 -0500

(trimming main IETF list off the cc:)

>     - IDNA clients MUST also be able to decode IDN(UTF8)/EDNS responses

This makes the rather rash assumption that the clients will be upgraded
to support a *future* feature.  You're also assuming that there will be
a way to test/debug IDN/EDNS responses when there are none actually being
seen "in the wild".

Re-work the model, assuming that (a) 50% of the people won't upgrade until
the new feature is *usable*, (b) 10% of the clients *wont* upgrade, and
(c) there will be at least 2 or 3 show-stopper bugs found in the EDNS
handling when it starts being widely used.

You'll never get widespread deployment of a feature that is both not usable
and untested (because there's no use of it).

>     - DNS servers will now accept both ACE (as usual) and IDN(UTF8)/EDNS

Are there any hidden gotchas here?  Are there any changes needed to BIND
to allow it to process a UTF8 zone file?  Are there any "bad news" characters
that might cause parsing problems?

Again - what happens if not everybody upgrades?

>     - ACE requests will yield Both an ACE response + an IDN(UTF8)/EDNS

Here there be dragons.  Do a 'dig aol.com mx', and ask yourself what
happens to performance if that doesn't fit into one DNS packet anymore,
and as a result a resolver has to drop back to TCP (instead of one packet
each way, you're now up to a 3-packet handshake, the data, and the FIN
packets) - and note that the TTL on the AOL.COM SOA is 3600....

What happens to an ACE-based client that receives an EDNS response that it's
not able to parse?

>     - IDNA clients continue to send in ACE but will start to see UDNA

This is a can of worms - how does the server know for sure that the client
is upgraded and is able to parse a UDNA response?  Remember - the client
is probably *NOT* asking directly.  If I've upgraded example.com's DNS
to be UDNA ready, how do I know it's safe to send a UDNA response to a query
from foo-bar.com (since it's quite possible that the *USER* has upgraded,
but the sysadmins have NOT upgraded the DNS that's recursing the query for
the user....

> comes with IDN(UTF8)/EDNS built in.

Which will be slower than you might think..

> the IDNA client, thereby also creating an incentive for server side to

OK.. How does my client tell that a DNS server in Zimbabwe is or is not
upgraded?  Remember - DNS is *distributed*, and just because YOUR DNS
server is upgraded doesn't mean that the one actually providing an
authoritative answer has been upgraded...

> still uses IDNA to send requests, but UDNA requests will be managed in UTF8

It's this "might continue to see ACE leakage" that's a major problem...

> clients to IDN(UTF8)/EDNS.

Just remember the pain felt by *both* 50%s when 50% of the users had
MIME-aware MUAs, and 50% didnt....

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Apr 2002 11:16:09 -0500

> question for the IDN group I believe.

I cannot imagine that the IDN group is competent to decide how email
(or any other specific application) should migrate.  

Keith

---

From: Keld Jørn Simonsen <[email-masked]>
Date: Tue, 2 Apr 2002 19:10:32 +0200

> question for the IDN group I believe.

Well, I think SMTP was very successful in getting to 8-bits everywhere.
Or almost everywhere. Today ESMTP is the predominant protocol for email
exchange. And this was really not in the cards when we designed the 
MIME stuff. The predominant policy at those times was  "use 7-bit
generally, and 8-bit locally", the so-called "enclave" model. 
I was amongst the most vocal proponents af a general 8-bit email
exchange protocol, the one that emerged as ESMTP. I remember being
at IETF in Santa Fé 1992-ish, where we discussed this, and I was the
only European around when it was asked if somebody, especially from
Europe, really wanted a general 8-bit exchange protocol. And I raised
my voice and insisted. So it was not really questioned anymore
and ESMTP was produced. But mostly as a courtesy to the Europeans.
Sometimes you can make a difference:-)

I think getting to >8bit is a non-starter, as per IESG policies
that promotes utf-8 as the primary encoding in protocols.
And I really do not see why we should open up to a myriad
of charsets (being the author of rfc 1345 I need to stress that:-)
We have one, we can deal with two, not many.

What we have now is a restricted ASCII for DNS, and we want to
go to a restricted utf-8. And nothing else. And we want the
backwards compatibility and operation between ascii and utf-8
so we can ensure a happy migration and coexistance.

Best regards
Keld

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 12:36:36 -0500

Its so great to at least have some response.  ;-)
I have said that there will be holes in my initial proposal, but this is a
good start.  Lets work on ironing out the pathway.

> a way to test/debug IDN/EDNS responses when there are none actually being

The assumption is no different than assuming clients will upgrade to ACE.
If we can assume that clients will be willing to upgrade to IDN then, there
is no reason to think that implementations would not include both ACE and
IDN(UTF8)/EDNS if IDN so specifies.

> that might cause parsing problems?

This is an application implementation issue, NOT a protocol issue.  If BIND
doesnt implement it well, there will be other software vendors that will
make it work as "specified".

> packets) - and note that the TTL on the AOL.COM SOA is 3600....

I dont understand.  If you do a dig with any non-upgraded clients, then you
will be only digging for English domains.  If you are using an upgraded
client then you will be able to parse the packet.  Also, for most cases I
would imagine it will still fit in one packet.

> What happens to an ACE-based client that receives an EDNS response that
it's

That, as specified in 1. is not possible.  ALL ACE implementations MUST also
implement IDN/EDNS.  That is why the discussion should start NOW! Finally
someone realizes my point on why we need to discuss this now and not after
we have done the ACE part :-)

> but the sysadmins have NOT upgraded the DNS that's recursing the query for

I see your point.  What if ACE requests will receive only ACE results, while
UDNA requests will get both the ACE and the UTF result.  This way, we know
for sure that the inquirer knows UDNA if they initiated it first.

> Which will be slower than you might think..

Not much slower than what you should expect if you go for "ACE everywhere"

> authoritative answer has been upgraded...

First of all, the EDNS migration path will be expected.  Then, if the
Zimbabwe DNS operator wishes to offer IDNs, they would upgrade to IDN/EDNS.
If they do not provide IDN resolution service, then IDNs wont work anyway.

> It's this "might continue to see ACE leakage" that's a major problem...

Clients that do not upgrade will eventually be obsoleted.  At least we know
that.  Also, if the admin is so desparate to know what the ACE is, s/he
could always use other conversion tools to view it.  But as the core app,
ACE need not be done.  That's the beauty of ACE, isnt it?

> MIME-aware MUAs, and 50% didnt....

Precisely why we should plan ahead better this time.

Edmon

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 12:43:22 -0500

Hi Keld,

----- Original Message -----
From: "Keld Jørn Simonsen" <[email-masked]>
> Sometimes you can make a difference:-)

Whole heartedly agree.  This time we have yourself and the Asian community
standing up saying we want IDN(UTF8), hopefully, this is unquestioned as
well.

> so we can ensure a happy migration and coexistance.

Makes good sense.

Edmon

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 02:07:12 +0800

> good start.  Lets work on ironing out the pathway.

It is nice to have a response. It would be better if you have a draft to
explain what you doing. It is also okay if you wish to start from UDNS, a
draft which I find it is reasonable altho incomplete. Talk to Oscarsson if
you wish but lets put it down in writing.

-James Seng

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 02:25:46 +0800

> standing up saying we want IDN(UTF8), hopefully, this is unquestioned as

I am aware that some chinese members in this group have stood up to ask for
TC/SC in IDN. I am also aware that Lee have asked for reordering.

But I am not aware of any "Asian community standing up saying we want
IDN(UTF8)". I honestly hope I havent missed anything important.

Or is this just another fragment of your imagation?

It is frustrating to hear you repeating "I want UTF-8, I want UTF-8 now!"
again and again. Lets just say "yes, we hear you say that and lets move on".

By moving on, I mean you write a draft to cover:
a) what is the advantage of IDN(UTF8) over the existing IDNA
b) how IDN(UTF8) is going to work
c) how you intend to get there

-James Seng

---

From: "David Leung \(Neteka Inc.\)" <[email-masked]>
Date: Tue, 2 Apr 2002 14:20:04 -0500

>and untested (because there's no use of it).

That's the point convincing all users to use a client is not easy to deploy,
so why IDNA with ACE??

>packets) - and note that the TTL on the AOL.COM SOA is 3600....

Hmm... talking about DNS packet that wont fit... so how about DNSSEC, their
packets are large as well...

>but the sysadmins have NOT upgraded the DNS that's recursing the query for
If any *entity* along the path that is not capable, I think the last current
capable *entity* will down version the protocol and not sending the new
thing.

>Which will be slower than you might think..

Is there any indication above to state how fast we think this maybe?! I
guess you must be good in mind reading in order to see that we are thinking
this deployment is fast!!

>It's this "might continue to see ACE leakage" that's a major problem...

bq--RACE.com is a IDN is not a problem... it is just some sort of alternate
representation of characters on the UI... I can now read bq--3cmzs3rp.com as
"hongkong".com : )

>MIME-aware MUAs, and 50% didnt....

So that's why servers need to be upgraded to be capable to handle the new
protocol first before the users need to know, or else the pain will come out
when they realized that IDN is not the real IDN and is just buying another
ASCII names, so what not have a system that just maps IDN  to existing
domains just like internet keywords?

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 14:24:43 -0500

James,

I think you missed the point completely.  The point, as Keld have mentioned
is that there are people, who believes that it is beneficial to have UTF8
option, I being one of them and Keld being one of them, and it is a fact
that cannot be ignored and should not be ignored.  From my interpretation of
your previous words, you also believe that it is beneficial to go beyond
ASCII.  That, is my point and please dont try to confuse the message with
the means!  Human language is not machine language and you have to
understand the context before you can get the message.  If you cant
understand this simple depth and intricacy of human interaction and
communication, I cant help you.

The exercise is to establish a meaningful support before I waste my time
working on a draft if no one wants to talk about it.
I have a draft of a draft inplace for UDNA, and I am updating it as we
continue to discuss some of the elements of it on the list.  I think it
makes more sense this way, than to throw something in here fresh.  Also I
think it would be beneficial to have a draft on IDNX (IDN Transition).  So
please stop distrupting the discussion of a good discussion here.

Edmon

----- Original Message -----
From: "James Seng" <[email-masked]>
To: "Edmon Chung" <[email-masked]>; "Keld Jørn Simonsen" <[email-masked]>
Cc: <[email-masked]>; "John Stracke" <[email-masked]>;
<[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, April 02, 2002 1:25 PM
Subject: Re: [idn] Re: 7 bits forever!

---

From: "Eric A. Hall" <[email-masked]>
Date: Tue, 02 Apr 2002 13:40:43 -0600

James Seng wrote:

> It is also okay if you wish to start from UDNS, a draft which I find
> it is reasonable altho incomplete. Talk to Oscarsson if you wish but
> lets put it down in writing.

Dan should certainly be encouraged to resubmit his UDNS proposal, but in
the meantime, I would like to point out that draft-hall-dm-idns-00.txt is
the dual-mode draft that is currently in the WG pool. Perhaps your
confusion stems from the I-D-N.net web site, which still lists UDNS as an
active document, and which does not list DM-IDNS.

[For the record, message <016c01c14c76$ea176c20$dd00a8c0@jamessonyvaio>
shows that UDNS was removed from the pool on Oct 4, 2001 while message
<023301c172ee$62f4fc50$0701000a@jamessonyvaio> shows that DM-IDNS was
accepted into the WG pool on Nov 22, 2001. Don't rely on the web site.]

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Keld Jørn Simonsen <[email-masked]>
Date: Tue, 2 Apr 2002 22:30:43 +0200

> communication, I cant help you.

Well, I have to state that I am not so sure we need an UTF-8 upgrade.
Maybe ACE is enough. Things can get complicated if eg the primary DNS
handles utf-8 while one of the secondaries do not. 
DNS is different from SMTP as all DNS clients need to *understand*
all DNS servers, each character needs to be understood. In SMTP the
MTAs are only exchanging messages between consenting adults, and
it is up to the sender and receiver to mutually understand eachother. 

Also the situation back in 1991 with SMTP was different from
todays DNS issues. Then we had a lot of email that was not
ASCII, but a myriad of other encodings, incl national 7-bit variants,
japanese, and iso-8859-?. Today we almost entirely still have
only (restricted) ascii in the DNS.

I frankly think that for interoperation, all DNS data needs to be
stored in ACE encoded ISO 10646, and on the wire it should be ACE
encoded queries and responses. It would then be up to the clients
to do the normalization and conversion from/to ACE.
That would mean no changes to existing DNS protocols!

Then you could have tools for maintenance of the zone data in
one or more charsets for convenience of the DNS zone maintainers.

A weak point of mine is that *iff* we go beyond ASCII, UTF-8 is
the only additional encoding we should consider for the wire,
and *iff* seemless interoperability can be proven with existing 
operations and some gains can be shown, I would not be against 
a new UTF-8 protocol for DNS.

Best regards
Keld

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 08:43:38 +0800

My apologises.

Let me correct myself: DM-IDNS or UDNS are both good starting point, one
which the UTF-8 supportor group should go aside and finish it.

-James Seng

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 08:41:44 +0800

And you miss the point completely too.

IETF move forward with draft. Repeatively say "i want utf-8, i want it now"
after the first time are just noise.

Write your draft. It is make sense, then support will come.

-James Seng

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 3 Apr 2002 12:26:41 +0200 (CEST)

> active document, and which does not list DM-IDNS.

And for those that are looking at other IDNA to UTF-8 in DNS transitional
schemes it would be useful to look at the open issue regarding DNSSEC
that I pointed out about draft-hall-dm-idns-00.txt a while back.

I suspect that all such transitional schemes will have the same issue
when DNSSEC is used to provide integrity of the RRsets: can't depends
on servers synthesing records in general and can't have caching resolvers
modify the queries and/or responses.

   Erik