Subject: RE: [idn] Presentation suggestion about "requirements" doc

From: John C Klensin <[email-masked]>
Date: Fri, 02 Mar 2001 10:11:52 -0800

--On Friday, March 02, 2001 10:20 AM -0500 "Hollenbeck, Scott"
<[email-masked]> wrote:

> What advantage is gained by moving the context and background
> information into a separate document that might be less
> controversial than the requirements?  Conversely, what is lost
> by keeping the document as-is?

My guess is that the WG is going to need to do some serious
backing up and revising of the "requirements" part, if for no
reason than that it lists some things as "MUST"-level
requirements that currently-proposed solutions don't meet.  Some
of us are likely to argue that those requirements should stand in
the document and that text should be added explaining why the WG
chose to give up on them (rather than just taking a "never mind"
attitude and quietly removing or downgrading them).  Whether that
view prevails or not, the debate and the fussing with the text
won't get the document finished quickly.

I deduce from the WG Last Call on the document that the co-chairs
are anxious to get the thing wrapped up and out.  I don't believe
that it would be wise (or proper) to publish it while some of its
provisions are controversial: If it contains strong-requirement
provisions that are not met by proposals under active
consideration, they are controversial, whether the WG wants to
explicitly face that or not.

By contrast, the context and background material probably is not
controversial.  So it makes sense to split it off (a fairly
trivial job, I think) and get it, at least, published and out of
the WG's queue.

     john

---

From: "James Seng/Personal" <[email-masked]>
Date: Sat, 3 Mar 2001 02:47:29 +0800

John,

1. Requirements I-D according to our milestone should be Last Call in
Aug 2000. We are now Mar 2001 and we have not receive any major request
for changes in wordings in it for many months. So the reason the
"co-chairs are anxious to get the thing wrapped up and out" is fairly
obvious.

2. If we can't nail down Requirements, it would be fairly difficult for
us to move forward with the next step, ie, comparison and protocol.
Neither of which is valid if we don't have any assumption to build upon.
(btw, according to Milestone, we should be doing Last Call on comparison
on Jan 2001).

3. If you have any specific request to downgrade MUST to SHOULD, or any
other changes in wordings for requirements, please send them in now.

4. Given that we keep slipping some of the schedule these few months, it
reflects either (a) poor scheduling of milestones when the WG is formed
or (b) poor management of the WG chair. I shall left it to the WG to
decide which is the reasons but moving forward, I proposed we readjust
the Goals and Milestone section of the Charter to accurately reflect
what we all think where we stand on now.

-James Seng

---

From: "D. J. Bernstein" <[email-masked]>
Date: 4 Mar 2001 18:41:58 -0000

I propose replacing ``Requirements'' with ``Considerations,'' and
rephrasing the document accordingly. For example, instead of saying

   The protocol MUST NOT require that the current DNS cache servers be
   modified to support IDN.

we can say

   Does this IDN solution require revision and redeployment of DNS
   caches? How difficult are the software changes? There are a huge
   number of caches on the Internet, perhaps millions. Upgrading them
   all will take time and effort, even if the software changes are easy.

and instead of saying

   It MUST make the minimum number of changes to existing protocols on
   all layers of the stack

we can say

   How many protocols are changed by this IDN solution? How extensive
   are the changes? For example, what changes are required in the SMTP
   protocol? There is a cost to changing a protocol, beyond the cost of
   changing implementations: namely, new protocol specifications must be
   written and distributed.

Readers should be able to look back at this document and understand what
we were thinking.

> us to move forward with the next step, ie, comparison and protocol.

It will be helpful to have a good framework for rational comparison of
the IDN proposals. But the current Requirements document doesn't seem to
have been written for this purpose. It appears to be an attempt to slip
a bunch of bad decisions past the WG. It's missing the justifications
that a legitimate document would have, and it imposes several premature
restrictions upon the directions that the WG can go.

---Dan

---

From: Rick H Wesson <[email-masked]>
Date: Sun, 4 Mar 2001 11:38:36 -0800 (PST)

Dan,

I whole heartedly disagree with your suggestions. The requirement
documents put constraints on the proposals REQUIRING them to conform to
prearranged constraints. your suggestion severely limited the working groups
ability to effectively evaluate proposals.

I therefore can't agree with your suggestion, after all working groups
often use requirements documents to avert such circumstanceses that would
occur should your proposal be implemented.

we MUST be able to effectively evaluate the proposals and move forward to
some conclusion.

-rick

---

From: "J. William Semich" <[email-masked]>
Date: Sun, 04 Mar 2001 17:44:06 -0500

Dan:

I strongly support your proposed approach (below).

Thanks,

Bill Semich

---

From: Marc Tamsky <[email-masked]>
Date: 08 Mar 2001 19:38:42 -0800

I concur with Dan's approach (below) as well.

Marc.

-- 
Marc Tamsky <[email-masked]>  --  dotTV, Inc. (http://www.tv/)

>>>>> On Sun, 04 Mar 2001 17:44:06 -0500, "J. William Semich" <[email-masked]> said:

   
   
   we can say

   and instead of saying

   we can say

   

   ---Dan

---

From: Marc Tamsky <[email-masked]>
Date: 08 Mar 2001 20:32:03 -0800

Is it outside the scope of this group's charter to start discussing
the formation of a detailed list of existing systems (whether
software, hardware, or protocols), and
  a) how they are affected by each of the proposals
  b) how difficult updates or upgrades are for each
  c) the time frame for announced upgrades
  d) the attrition rate for legacy systems
  e) the growth rate for that class of systems, and how soon newer
     systems will become the majority of used systems

My interpretation of discussions that happen on this list -- the
careful, methodical, as-scientific-as-possible review of the changes
proposed within this group have been discouraged at every possible
opportunity.

All the proposals involve significant changes to many systems.

The fact that no work has been done to quantify those changes should
raise alarm bells -- the work to quantify the number of affected
systems is large -- without any work to quantify it, perhaps very
large.

The changes needed by other systems is STRICTLY larger than the work
to come up with the estimates of work.

Comments?

Marc.

---

From: "Bruce Thomson" <[email-masked]>
Date: Fri, 9 Mar 2001 15:03:47 +0900

It is presumably within the group's charter to do this,
and it is probably a good idea to do such an investigation.
On the other hand, we should consider whether the
definitive results would have to be in before the standard
is created.

Every new protocol or platform inevitably requires bringing
people up to speed. I myself have spent more time than
I would like to admit making things multilingual that previously
were not, or worse, rewriting apps and drivers for the latest
version of Windows or the latest SCSI spec.

Everyone expects to have to do some work to take advantage
of the latest new technology. If I want to connect my digital
camera to my computer with USB, I have to get a computer
that was made in the last couple years with the right hardware,
and then I have to upgrade to at least Win98 OSR2. People
who are not in a hurry with this can wait. No big deal.

In the idn case, the biggest concern probably is whether we will
somehow melt down the entire DNS infrastructure by choosing
an inappropriate encoding. Here is my analysis on that:

There are basically 3 main classifications of DNS servers. I
would not like any of the 3 to break... RACE encodings of course
will not break the DNS, but what about UTF-8?

1) Authoritative servers.
These are set up by domain hosting services and have to have
records in local files for the domains they host. These servers
need to actually respond with data for their local database
when interrogated with UTF-8. If someone had an old version
of BIND, this might not work, but anyone who is actively
hosting idns would naturally upgrade. I am currently hosting
domains that can be accessed with UTF-8, using BIND 8.2.3.
Works with no problems, except the log entries are not readable.
There is a patch available for this, which I am not using.
Also, the .cc DNS is authoritative for .cc domains and answers
queries in UTF-8. They have been going fine for some time with
no apparent problems.

2) Root servers
If someone types an idn but mistypes .com, his request would
go to the root server full of UTF-8. Would this break something?
But again, .cc domains are in wide use, and UTF-8 requests
do get sent in large numbers to the root without any damage
that I have heard of.

3) Caching servers
Suppose I as a user want to resolve an idn, but my ISP's
caching server stands between me and the authoritative
and root servers. Will my ISP's BIND version cause me to
be unable to resolve the idn? Or will I crash his DNS?
This is by far the most important area for concern, because
there are many versions out there, and we can't just stop
the whole Internet and demand massive upgrades.
Y2K all over again! :P

But, in actual fact, BIND 8.2.3 works just fine here. Most
ISPs are using it already. How do I know this? Because
www.redhat.com and CERT, etc. have issued security
advisories urging everyone to immediate update. There
is a horrific security hole in previous versions that
allows hackers to get root access on the DNS, and/or
crash it. My site was actually a victim of this, and our
DNS crashed about twice a day for over a week until
we figured it out and upgraded.

Therefore, any ISP who is so out of it as not to have
upgraded has much more serious problems than whether
idns work, or perhaps crash him. In this case, let's just
distribute the word on those old, bad versions:
"ISC has discovered or has been notified of several bugs which can result in
vulnerabilities of varying levels of severity in BIND as distributed by ISC.
Upgrading to BIND version 9.1 is strongly recommended. If that is not
possible for your site, upgrading at least to BIND version 8.2.3 is
imperative."

Sendmail was also discussed as the source of possible problems.
This problem can be eliminated by suggesting that idns not
be used in e-mail communications except within the
intended country. (see my previous post.)

Bruce

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 09 Mar 2001 15:50:52 +0100

>intended country. (see my previous post.)

ouch.
a random selection of 974 self-declared Japanese that I just happen to have 
lying around gives:

- 784 .jp addresses
- 80 .com addresses
- 15 .net addresses
- 5 .org addresses
- 3 .mil addresses
- 2 .br adresses

the same set gives for the .jp addresses

- 784 in Japan
- 5 in the US
- 4 in Korea
- 3 in China
- 2 in Thailand

and so on.

what protocol extension were you proposing to make sure those Japanese 
domain names did not leave the country through Sendmail?

For better or worse, we only have one Internet.

--
Harald Tveit Alvestrand, [email-masked]
+47 41 44 29 94
Personal email: [email-masked]

---

From: "Bruce Thomson" <[email-masked]>
Date: Sat, 10 Mar 2001 19:10:02 +0900

> For better or worse, we only have one Internet.

I researched this issue a bit more, and found that RFC1342 allows
the "From:" field of an e-mail to be UTF-8, and that 8-bit-incapable
mail gateways can still be used by using base-64 encoding on
this field, the same as is currently used when the sender's name
is in an 8-bit codeset such as Japanese. So actually, the problem
is not sendmail at all, it is in the recipient's mail software only.
If he doesn't have the latest version, and has no idea who sent the
e-mail, he won't be able to reply.

It is important to consider this issue it its total context. It is difficult
to
work in a foreign language if you are not in the particular country, and
idns are just one small aspect of this.

To work in Japanese, I first need to have Japanese Windows. Yes,
I know you can install language packs, as the guy sitting next to me
does. But basically he is just playing around and can't really use his
system in the language.

Next, you need a Japanese printer. Japanese Office, Pagemaker,
Illustrator, Power Point, or whatever. Take the plunge, do it all
in that language, and suddenly it all comes together into something
quite nice.

I have also been in the position of trying to use Japanese while in
the U.S. An excruciating experience, I must say. I am missing just
too many pieces of what I need to make it work right. People
send me documents that have Japanese fonts I am missing. My
printer isn't really up to spec, etc.

So, my hat is off to those hardy Japanese ex-pats who run in
a Japanese environment overseas, because I realize what they
had to do to get that far, and what they have to put up with if
they have a half-way environment.

Either way, I think these guys can cope with idn e-mail without
any help from us. They will either choose not to use idns in e-mail,
or they will get the apps they need to make it work.

Please remember that although we all may be "one Internet", a
lot of "out-of-band" agreements are needed to inter-operate.
Lawyers commonly use WordPerfect, and that may work well
for their in-house communications, but if I can't read that file
format, they will be wasting time if they send me such a thing.
No protocol can prevent this, just common sense.

Bruce

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 10 Mar 2001 21:59:06 +0100

At 19.10 +0900 01-03-10, Bruce Thomson wrote:
>the "From:" field of an e-mail to be UTF-8

Not the whole From: field. Especially not the email address.

And, 1342 is obsoleted by 1522, which is obsoleted by 2045, 2046, 
2047, 2048, and 2049 which in turn are updated by 2184 and 2231.

All of this is found by just looking in the RFC Index file.

All of this is then updated by the new versions of RFC 821 and 822, 
which are in the RFC Editor queue.

    paf

---

From: "James Seng/Personal" <[email-masked]>
Date: Sun, 11 Mar 2001 06:43:12 +0800

Bruce,

Once again, you are bringing up issues which have been discussed before.
I would appreciate new comers go thru our archives before repeating
arguments which we have done.

The short answer to you on Base64 is that Base64 is good for data
transfer but not so good when comes to computer processing.

-James Seng

----- Original Message -----
From: "Bruce Thomson" <[email-masked]>
To: "Harald Alvestrand" <[email-masked]>; <[email-masked]>
Sent: Saturday, March 10, 2001 6:10 PM
Subject: Re: [idn] impacted systems investigation

> > a random selection of 974 self-declared Japanese that I just happen
to
> have

---

From: "D. J. Bernstein" <[email-masked]>
Date: 11 Mar 2001 23:37:16 -0000

Nobody has presented any evidence of DNS caches having trouble with
UTF-8. Nobody has presented any evidence of DNS servers having trouble
with UTF-8. The discussion of server upgrades is divorced from reality.

There are some widely distributed DNS _client_ libraries that, in
violation of RFC 2181, reject DNS names containing unusual characters.
In particular, the version 8 libresolv gethostbyname() implementation
does this; look at gethostans(), maybe_ok(), res_hnok(), and res_dnok().
Sysadmins can fix it by putting no-check-names into /etc/resolv.conf.

---Dan

---

From: [email-masked]
Date: Mon, 12 Mar 2001 11:39:29 +1100

> with UTF-8. The discussion of server upgrades is divorced from reality.

	I agree with you here.  UTF8 does not require a server upgrade
	unless you want the error messages to put out UTF8 rather than
	RFC1035.  Most will even allow entry of UTF8 in master files
	even though it is out of scope.  For those that don't the
	translation from UTF8 to RFC1035 is easy and reversable.

	The wire format has always supported UTF8 as it was designed to
	support 8 bit names.

> violation of RFC 2181, reject DNS names containing unusual characters.

	gethostbyname() and gethostbyaddr() are clients of the DNS
	and as such are not covered by RFC 2181.  Clients of the DNS
	are perfectly entitled to reject any answer they get back from
	the DNS.

--
Mark Andrews, Nominum Inc.
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "D. J. Bernstein" <[email-masked]>
Date: 12 Mar 2001 08:19:40 -0000

Mark Andrews, from the BIND company, writes:
> UTF8 does not require a server upgrade

Right. But Patrik and Paul claim the opposite. This claim is, in fact,
the centerpiece of the IDNA ``design philosophy.''

> and as such are not covered by RFC 2181.

RFC 2181 says that character-set restrictions are forbidden in all
``implementations of the DNS protocols.'' It doesn't matter whether the
implementations are clients, caches, or servers.

Are you saying that your client libraries are ``various applications
that make use of DNS data'' rather than ``implementations of the DNS
protocols''?

Anyway, why don't you make make no-check-names the default, or at least
make 8-bit characters exempt from the test? The specific bugs described
in CA-1996-04 were fixed a long time ago; bugs of this type can't occur
if you use \ddd for unusual ASCII characters in PTR results; your BIND 9
documentation already says that bugs of this type aren't BIND's problem.
Why are you prohibiting lookups of contourcname.cr.yp.to?

---Dan

---

From: [email-masked]
Date: Mon, 12 Mar 2001 20:46:19 +1100

> that make use of DNS data'' rather than ``implementations of the DNS

	The DNS is a database, like NIS or even /etc/hosts where hostnames
	can be stored and retrieved.  Once it has been retrieved from the
	DNS, or NIS database or /etc/hosts gethostby*() is free to reject
	names which fall outside of the spec, RFC 952, in this case.

	However this is not really appropriate for this list other than
	when RFC 952 is updated gethostby*(), getaddrinfo() etc. are still
	free to reject out of spec. names.

	Mark

---

From: [email-masked] (David C Lawrence)
Date: Mon, 12 Mar 2001 06:36:57 -0500 (EST)

> the centerpiece of the IDNA ``design philosophy.''

Not so.  We all know the servers can handle 8 bit domain names.  What
the servers can't tell, however, is whether some 8 bit string is UTF-8
or some local encoding, and that presents a security problem.  To use
UTF-8 at the server, the protocol would need to be updated so that a
client could affirmatively declare, "I'm IDN-aware, and thus my
request is using UTF-8, not some other local encoding."

---

From: [email-masked]
Date: Tue, 13 Mar 2001 09:13:08 +1100

> request is using UTF-8, not some other local encoding."
	And this is outside of the servers responsability.  It is the
	responsability the person entering the data to ensure that it
	is in valid and of the clients to ensure that answers are within
	the expected range.

	Anything else will break clients that expect existing behaviour
	like you can have a 0x00 as a octet in a label.

	BIND 8 had a tri state check-names fail/warn/ignore.
	A future server might want utf8-check fail/warn/ignore.  The default
	on the master should implementation dependent.  The slave
	and caching servers should default to warn/ignore.
	A future server may also want ace-convert.

	However all this should really be done outside of the server with
	tools that produce RFC 103[45] compatible zone files.

	Mark

---

From: "Allen Smith" <[email-masked]>
Date: Thu, 22 Mar 2001 23:11:52 -0500

> implementations are clients, caches, or servers.

Umm... "Clients of the DNS can impose whatever restrictions are
appropriate to their circumstances on the values they use as keys for
DNS lookup requests, and on the values returned by the DNS". Moreover, 
RFC 2181 specifically states that it is not considering security; I
therefore would argue that if one is to be concerned with security, if 
there is a contradition between RFC 2181 and security concerns,
security concerns override.

	-Allen

-- 
Allen Smith				[email-masked]

---

From: "Allen Smith" <[email-masked]>
Date: Thu, 22 Mar 2001 23:07:41 -0500

On Mar 12,  7:06am, David C Lawrence wrote:
> Not so.  We all know the servers can handle 8 bit domain names.

Incorrect, at least if they are using domain names for, for instance,
filename construction - which is currently the case for BIND 9 in
DNSSEC, for instance. I have submitted a patch to [email-masked]
to help solve this problem.

> What the servers can't tell, however, is whether some 8 bit string is UTF-8

Agreed. Unless a client affirmatively declares this, returning names
outside of RFC1123 standards should not be done, for many security
reasons.

	-Allen

---

From: "Allen Smith" <[email-masked]>
Date: Thu, 22 Mar 2001 23:32:21 -0500

On Mar 12,  6:00pm, [email-masked] wrote:
> 	And this is outside of the servers responsability.

This belief is a recipe for breaking old clients...

> It is the responsability the person entering the data to ensure that it

Clients should certainly do some checking of data, to ensure against
bad data being inserted by an untrustworthy master or slave
server. However, why should catching servers (which should be more
trustworthy) not do so also, or instead if they are considered
sufficiently trustworthy? Distributing code between all clients for
doing checking (particularly on machines without shared library
support, or ones in which the resolver code is incorporated into
shared libraries with other functions (e.g., libc)) is much less
efficient than doing it in one place, even leaving out the time
savings involved in checking then caching. (To some degree, this is
being done in BIND 9 via the lwres daemon, but that requires client
modifications so that the new "lightweight resolver protocol" is used
instead of the standard DNS protocol.)

> 	like you can have a 0x00 as a octet in a label.

??? Most clients that I am familiar with expect exactly the
opposite... hostnames and other DNS responses are generally treated as
_null_-terminated strings.

> 	BIND 8 had a tri state check-names fail/warn/ignore.

The lack of this in BIND 9 is one reason we aren't upgrading yet, yes.

> 	A future server may also want ace-convert.

Going with utf8-check instead of check-names (with the full RFC 1123
checks) will break things, introduce possible security problems,
etcetera... unless a client signals otherwise, only names passing RFC
1123 should be allowed (modulo length modifications - 255 (including
final null) is a reasonable maximum name length).

> 	tools that produce RFC 103[45] compatible zone files.

You're again assuming trustworthy (following safe standards) origin
servers; caching servers should certainly do this internally.

	-Allen

---

From: [email-masked]
Date: Fri, 23 Mar 2001 18:13:31 +1100

> to help solve this problem.

	It's not a server bug, those parts of the library are not called
	by the server.

> outside of RFC1123 standards should not be done, for many security

	It's not the servers job to enforce this.  This is not new.  As
	a problem it exists today.  IDN does nothing to change this other
	than (if we go with utf8) increase the number of names returned
	/ used that fall outside of RFC952/RFC1123.

	Part of the problem with the utf8 solution space is that those
	clients (OS's) that cared about security need to be updated to
	the new spec.  Those that didn't are just as vulnerable as they
	were before, just that they are more likely to recieve data
	they are not prepared for.

	Mark

---

From: "Allen Smith" <[email-masked]>
Date: Fri, 23 Mar 2001 03:11:51 -0500

> 	It's not a server bug, those parts of the library are not called

What about bin/named/tkeyconf.c?

> > 
> > > What the servers can't tell, however, is whether some 8 bit
> > > string is UTF-8 or some local encoding, and that presents a
> > > security problem.  To use UTF-8 at the server, the protocol
> > > would need to be updated so that a client could affirmatively
> > > declare, "I'm IDN-aware, and thus my request is using UTF-8, not
> > > some other local encoding."
> > 
> 	the new spec.

I'm not quite sure what you're meaning here. Clients that care about
security should indeed be updated to make sure they reject
non-RFC952/RFC1123 names (with escaping any characters outside that
set another possibility, although it admittedly does introduce the
escape character (which, normally being \, could cause problems on
machines using that as a directory separator)). Or were you meaning
that OSes that _did_ this should be updated to allow utf8 characters?
That "solves" the problem for the OSes, but not for applications...

> Those that didn't are just as vulnerable as they

We use a bind-8 caching-only server with check-names fail. As long as
requests are going only to this server or other, trusted servers, this 
does indeed protect the clients. Maintenance of this capability is of
importance (it's one reason we haven't moved to bind-9), and is of
significance for IDN.

	-Allen