Subject: Re: [I18ndir] Mixed Labels and RDAP

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 1 Feb 2019 12:12:46 +0000

> -----Original Message-----
> From: John C Klensin <[email-masked]>
> Sent: Thursday, January 31, 2019 5:36 PM
> To: Hollenbeck, Scott <[email-masked]>
> Cc: Marc Blanchet <[email-masked]>; Patrik Fältström
> <[email-masked]>; [email-masked]
> Subject: [EXTERNAL] Re: [I18ndir] Mixed Labels and RDAP
>
> Scott,
>
> While I agree with Patrik and Marc, your question raises two other concerns
> for me, concerns that I should probably identify now rather than later...
>
> (1) If I correctly understand your question, it seems to me that
> it is a business practices one, not a technical issue.   As

[snip]

I don't think so, John. The question came up because RFC 7482 doesn't clearly describe what implementers should do to develop interoperable implementations. It says "IDNs SHOULD NOT", but it fails to provide any guidance to guide behavior if/when the situation occurs. That's a protocol specification issue.

> (2) The creation of this list were evidence that there was very little energy in
> the IETF among those with significant expertise in i18n-related topics.  I hope
> we can keep the list sufficiently focused on topics in the IETF's critical path --
> especially documents that are long-overdue for consideration -- that we can
> effectively address those topics rather than getting distracted by discussions
> that are not in that critical path.
> If I'm in the rough about that, I hope someone will say so and explain.

My two cents: issues of specification ambiguity in IETF Proposed Standards (such as RFC 7482) are in scope and on the critical path - especially if we have an opportunity to influence implementation direction before these services go live (and we get stuck with them for years).

Scott

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 1 Feb 2019 12:37:49 +0000

> From: Marc Blanchet <[email-masked]>
> Sent: Thursday, January 31, 2019 12:30 PM
> To: Patrik Fältström <[email-masked]>
> Cc: Hollenbeck, Scott <[email-masked]>; [email-masked]
> On 31 Jan 2019, at 12:24, Patrik Fältström wrote:
>
> > On 31 Jan 2019, at 18:10, Hollenbeck, Scott wrote:
> >
> >> Should it reject the query as syntactically invalid, or should it try
> >> to be accommodating by doing something like converting U-labels to
> >> A-labels to produce an input domain name containing all A-labels?
> >> What would you folks suggest?
> >
> > Without thinking for too many seconds (I might change my mind) one
> > should be conservative with what one produce, but accept (if possible)
> > what others do wrong.
> >
> > As A-label / U-label relationship is 1:1, I do not see any problems
> > doing the transformation if needed for individual labels.
>
> right. I agree. My only concern is the difference in the query and the
> returned data. Query has one string of code points (mixed U and A), while
> the answer will have another string of code points (all A). Not a good design
> in general, and if the client is picky with verifying what it has sent and what it
> has received, then not good. (but honestly, those javascript web clients are
> pretty lazy and do not verify anything
> ;-)

The RDAP server could also return _both_ the ldhName and the unicodeName (and potentially variants) as described in Section 5.3 of RFC 7483. Neither will match what the client provided, but perhaps they'll provide some clue to better help the end user understand the response.

Scott

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 01 Feb 2019 07:59:49 -0500

On 1 Feb 2019, at 7:37, Hollenbeck, Scott wrote:

> The RDAP server could also return _both_ the ldhName and the 
> unicodeName (and potentially variants) as described in Section 5.3 of 
> RFC 7483. Neither will match what the client provided, but perhaps 
> they'll provide some clue to better help the end user understand the 
> response.

my take:
- the javascript client, if they ever verify the return data, might just 
do straight string comparison and won’t match. yeah they are 
« dumb » (that is why it is appropriate to have the servers answers 
both A-Labels only _and_ U-Labels only domain names queries.
- the likelyhood (from the end-user perspective) that a domain name 
might be typed or copy/paste as a mixed label to me is, in real 
scenarios, pretty near zero probability.
- typical end-user will have no clue whatsoever about if the two domains 
(what he typed and what is in the answer) are identical or not. He might 
even think the response (or most likely the presentation of the response 
to the user by the « client ») is rogue.

to me, we shall just suggest no, given that a) there is no good way to 
have the RDAP answer return the exact same domain in the query; b) the 
likelihood that an end-user (or even a software program) will actually 
use mix labels is near zero, therefore not worth the trouble.

Marc.

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 1 Feb 2019 13:25:34 +0000

From: Marc Blanchet <[email-masked]>
Sent: Friday, February 1, 2019 8:00 AM
To: Hollenbeck, Scott <[email-masked]>
Cc: [email-masked]
Subject: [EXTERNAL] Re: [I18ndir] Mixed Labels and RDAP

      -----Original Message-----
      From: Marc Blanchet <[email-masked]<mailto:[email-masked]>>
      Sent: Thursday, January 31, 2019 12:30 PM
      To: Patrik Fältström <[email-masked]<mailto:[email-masked]>>
      Cc: Hollenbeck, Scott <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
      Subject: [EXTERNAL] Re: [I18ndir] Mixed Labels and RDAP

   my take:
   - the javascript client, if they ever verify the return data, might just do straight string comparison and won’t match. yeah they are « dumb » (that is why it is appropriate to have the servers answers both A-Labels only and U-Labels only domain names queries.
   - the likelyhood (from the end-user perspective) that a domain name might be typed or copy/paste as a mixed label to me is, in real scenarios, pretty near zero probability.
   - typical end-user will have no clue whatsoever about if the two domains (what he typed and what is in the answer) are identical or not. He might even think the response (or most likely the presentation of the response to the user by the « client ») is rogue.

   to me, we shall just suggest no, given that a) there is no good way to have the RDAP answer return the exact same domain in the query; b) the likelihood that an end-user (or even a software program) will actually use mix labels is near zero, therefore not worth the trouble.

   Marc

   [SAH] Marc, so are you suggesting that an HTTP 400 “Bad Request” would be a better RDAP response?

   Scott

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 01 Feb 2019 08:36:59 -0500

On 1 Feb 2019, at 8:25, Hollenbeck, Scott wrote:

>       From: Marc Blanchet 
> <[email-masked]<mailto:[email-masked]>>
>       Cc: Hollenbeck, Scott 
> <[email-masked]<mailto:[email-masked]>>; 
> [email-masked]<mailto:[email-masked]>
>    - the javascript client, if they ever verify the return data, might 
> just do straight string comparison and won’t match. yeah they are « 
> dumb » (that is why it is appropriate to have the servers answers 
> both A-Labels only and U-Labels only domain names queries.
>    - typical end-user will have no clue whatsoever about if the two 
> domains (what he typed and what is in the answer) are identical or 
> not. He might even think the response (or most likely the presentation 
> of the response to the user by the « client ») is rogue.
>
>    to me, we shall just suggest no, given that a) there is no good way 
> to have the RDAP answer return the exact same domain in the query; b) 
> the likelihood that an end-user (or even a software program) will 
> actually use mix labels is near zero, therefore not worth the trouble.
>
>    Marc
>
>    [SAH] Marc, so are you suggesting that an HTTP 400 “Bad 
> Request” would be a better RDAP response?

I haven’t thought of the appropriate return code (need to relook at 
the semantics definitions), so some 4XX response seems appropriate to 
me.

my 2 cents.

Marc.

---

From: John C Klensin <[email-masked]>
Date: Fri, 01 Feb 2019 11:48:45 -0500

--On Friday, February 1, 2019 12:12 +0000 "Hollenbeck, Scott"
<[email-masked]> wrote:

>> Cc: Marc Blanchet <[email-masked]>; Patrik
>> Fältström <[email-masked]>; [email-masked]
>> While I agree with Patrik and Marc, your question raises two
>> other concerns for me, concerns that I should probably
>> identify now rather than later...
>> 
>> (1) If I correctly understand your question, it seems to me
>> that it is a business practices one, not a technical issue.
>> As
> 
> [snip]
> 
> I don't think so, John. The question came up because RFC 7482
> doesn't clearly describe what implementers should do to
> develop interoperable implementations. It says "IDNs SHOULD
> NOT", but it fails to provide any guidance to guide behavior
> if/when the situation occurs. That's a protocol specification
> issue.

Ok, Scott.  That is fair.  But let me pursue it a bit further.

First, and most important to this situation, we have a number of
IETF standards track documents that specify the conditions under
which a SHOULD or SHOULD NOT should reasonably be violated, what
should happen if the SHOULD (or SHOUDL NOT) applies to a sender
and the receiver gets things that violate it, and even what
SHOULD be done if a MUST is violated.    We also have many
specifications that don't go down that road and instead simply
take the position that something that violates those conditions
is outside the spec and that it is not the role of the document
to try to speculate on why the standard might be violated and
what should be done about it.  In the Apps/ART area, examples of
the latter as as diverse as SMTP (RFC 5321) and the URI spec
(RFC 3986).  The latter does not even invoke 2119, but contains
many statements about what must or must not occur but discusses
consequences in only a small subset of those cases.

In the case of SMTP, the decision to not specify what happens if
a SHOULD (or even a MUST) is violated was extremely explicit.
Indeed, one of the key origins of the robustness principle was
the desire to avoid having to try to specify all of the relevant
variant or edge cases and what should be done about them.

So, first of all, it is not clear whether having a SHOULD NOT
and no guidance is a problem with the protocol specification.
As you know, I didn't participate in the RDAP effort, so I don't
know whether the WG made an explicit decision to leave that
guidance out (a decision you are now trying to revisit) or
whether the fact that the information you are looking for isn't
there is "merely" the result of its absence slipping through the
cracks, presumably due to insufficient review.  In neither case
is the issue an I18N problem (See "motivation" below) -- it
seems to me to be a question of whether RDAP (and RFC 7482 in
particular) is inadequately or insufficiently specified, one
that should be discussed on the WEIRDS mailing list.  That view
is reinforced when I notice that your discussion with Marc after
the note to which this responded are very much about the details
of RDAP, error codes, etc.  A fast grep of 7482 shows three
instances of SHOULD (or SHOULD NOT) and four more of MUST (or
MUST NOT), none of which specify what should (sic) occur if the
constraints are violated, so this is not in any way an
IDN-specific issue.

Should the WG decide (even by default or accidental omission)
that these things should be left unspecified and to the
implementation, "business decision" clearly doesn't cover all of
the considerations that might go into what the implementation(s)
might do, but that doesn't make the answers any more of an IETF
problem.

If the mailing list on which RDAP was developed concludes that
7482 (and maybe other documents and the WG) should be reopened,
I think that would be entirely reasonable.   If a 7482bis is
produced, it would be reasonable for this directorate to review
it for i18n issues or fuzziness (I detected one problem in the
process of grepping for MUST and SHOULD and there are probably
others).  But, until the EPP/RDAP community reach a conclusion
about what it wants, I don't see it as an issue for this
group/list.

More below.

>> (2) The creation of this list were evidence that there was
>> very little energy in the IETF among those with significant
>> expertise in i18n-related topics.  I hope we can keep the
>> list sufficiently focused on topics in the IETF's critical
>> path -- especially documents that are long-overdue for
>> consideration -- that we can effectively address those topics
>> rather than getting distracted by discussions that are not in
>> that critical path.
> My two cents: issues of specification ambiguity in IETF
> Proposed Standards (such as RFC 7482) are in scope and on the
> critical path - especially if we have an opportunity to
> influence implementation direction before these services go
> live (and we get stuck with them for years).

In scope for the IETF, certainly.  In scope for WEIRDS (or the
mailing list left over from it), definitely.  In scope for this
directorate just because an i18n issue is involved... I hope
not, at least if we are going to be able to accomplish anything
other than very general discussions of perceived weaknesses in
existing, published, standards-track specs that happen to touch
on i18n issues.

Motivation: My apologies for jumping so strongly on your
message/inquiry.  Someone has to be unlucky to be the first one
and you apparently picked up the short straw.  My concern, which
I hope was obvious from the proposal Patrik and I wrote for the
July BOF, is that we've observed several things in the last
eight years or so that may call for questioning some of the
design decisions associated with IDNA2008 (and hence PRECIS,
etc.) and others in the last seven years that raise similar
questions about the SMTPUTF8 work.  The list isn't just the
non-decomposing character issue and the anxieties it caused.  It
includes the implications of introducing emoji and what
identifiers, if any, for which they might be appropriate;
questions of whether the EAI WG's specs are over-permissive,
unrealistic about expectations, and/or insufficiently specific
about what is not possible and why; whether the layering of
responsibilities that is implicit in IDNA2008 is explicit enough
and what, if anything, should be done when some of those layers
don't work; the appearance of efforts from the ICANN direction
to impose specifications on IDNs that are either supplemental to
the IDNA2008 specs or alternatives to them; whether assumptions
about normalization and its applications that pervade our i18n
documents are actually optimal; whether our strong assumption
that the IETF should not get into (and can successfully avoid)
code point by code point evaluation of Unicode is actually
feasible; some of the technical issues associated with
"confusability" and the mechanisms that might be used to
mitigate it while providing a good user experience (or
preserving it if one thinks we have that now); and so on.  We
also have the problem that, over eight years after the IDNA2008
specs were published, we've got more slightly (or more than
slightly) inconsistent implementations of things claiming to be
IDNA out there.  It isn't clear what the IETF can do about the
latter other than either explaining better why our
specifications are the ones that should be followed or dropping
ours in favor on one of the others and promoting it and I'm not
sure we could do the first or should do the second.  

My personal hypothesis is that we are actually in reasonably
good shape and there are no deficiencies in the fundamental IETF
i18n specifications that are so serious that we cannot patch
around some and explain others better.  But I could be wrong.
Even if I'm right, we are not making any progress on proposed
explanations or patches, including those that would be in drafts
that have not been written or posted because it was clear that
they would not be processed.  If I am wrong, then progressing
i18n-related documents may be building new (or revised)
protocols on quicksand, an activity that would seem to require
very careful consideration.  

So my hope is to keep the directorate very closely focused on
trying to figure how which changes to, or explanations of, those
fundamental protocols are needed and then either progressing
(and developing if necessary) the relevant documents or
explaining why they are not needed or not appropriate.   I think
we are also going to need to review non-i18n documents with i18n
implications and educate the community about the issues.   But I
fear that, if we get dragged into discussions of design choices
in non-fundamental protocols -- rather than encouraging those
discussions on the mailing lists relevant to those protocols and
them perhaps performing a sanity check on the resulting
documents -- we will end up running out of energy before we get
to the more fundamental issues.

If making and keep that distinction is hopeless, I predict that
the directorate will deteriorate into the level of involvement
and participation that characterized the waning days of the
PRECIS and EAI WGs and that we will need to find a drawing board
to go back to... or conclude that the IETF has run out of those
and should admit it cannot do this work.

best,
   john

---

From: "Pete Resnick" <[email-masked]>
Date: Fri, 01 Feb 2019 10:51:42 -0600

On 1 Feb 2019, at 6:12, Hollenbeck, Scott wrote:

> My two cents: issues of specification ambiguity in IETF Proposed 
> Standards (such as RFC 7482) are in scope and on the critical path - 
> especially if we have an opportunity to influence implementation 
> direction before these services go live (and we get stuck with them 
> for years).

My take would be that we are chartered to review and assist with IETF 
work products, which usually means documents. That said, I don't think 
it's problematic to ask general questions about i18n here (if for no 
other reason, it's probably helpful for pedagogical benefits), with the 
understanding that such questions always get put on the back burner if 
there's a review that we all have to discuss.

pr
-- 
Pete Resnick http://www.episteme.net/
All connections to the world are tenuous at best

---

From: Martin J. Dürst <[email-masked]>
Date: Sat, 2 Feb 2019 06:32:43 +0000

Marc Blanchet wrote:

>> dumb » (that is why it is appropriate to have the servers answers both 
>> A-Labels only and U-Labels only domain names queries.

I think this a good analysis, and this is where the I18N expertise of 
this group is helpful.

Even if this is about an existing RFC, depending on our advice, it may 
become an update.

> I haven’t thought of the appropriate return code (need to relook at the 
> semantics definitions), so some 4XX response seems appropriate to me.

I'm not sure about the exact code. I suggest the RDAP folks look at 
https://www.mnot.net/blog/2017/05/11/status_codes and 
https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis. If necessary, 
contact the httpbis WG or some other experts.

What I think is important is that the server sends explicit and 
understandable information back in the Error Response Body
(https://tools.ietf.org/html/rfc7483#section-6).

Regards,   Martin.

---

From: John C Klensin <[email-masked]>
Date: Thu, 07 Feb 2019 20:27:36 -0500

Coming back to this as my time earlier in the week went into
that long discussion of draft-faltstrom-unicode11-07 (which,
given the Last Call announcement and Pete's note, I hope people
will prioritize over this thread).

One caution, then on to the substance:  Let's try to avoid
calling these FQDNs with one or more A-labels, one or more
U-labels, and zero or more NR-LDH labels (all three terms as
defined in RFC 5890) "mixed labels" or even "mixed domain
names".  "Mixed script label" is very commonly used to refer to
a single label containing [coded forms of] code points from more
than one script.  I've seen that usage, although less
frequently, simply as "mixed label".  Because there are issues
about how to think about "Common" and "Inherited", even those
uses are not without ambiguity.  I've also seen "mixed domain
name" and "mixed label domain name" to refer to FQDNs which
contain labels that encode strings in different scripts, even if
each of those labels is internally homogeneous wrt the scripts
it uses.  There is an ambiguity or two there too: it is not
clear whether Latin counts as a script and hence whether, e.g.,
IDNlabel.TraditionalASCIITLD would be mixed or not and the
status of A-labels is unclear, i.e., if the IDN labels have to
be in U-label form to count.

We do no one any favors by creating a new usage in which an FQDN
with a mixture of A-labels and U-labels (and maybe some NR-LDH
labels), even if there is only one script (or one script in
addition to Latin) is also referred to as a "mixed label",
"mixed domain name", or anything else that is easily confused
with one of those two.

Now back to input and output from RDAP...

--On Friday, February 1, 2019 07:59 -0500 Marc Blanchet
<[email-masked]> wrote:

>...
>> unicodeName (and potentially variants) as described in
>> Section 5.3 of  RFC 7483. Neither will match what the client
>> provided, but perhaps  they'll provide some clue to better
>> help the end user understand the  response.

> my take:
> - the javascript client, if they ever verify the return data,
> might just do straight string comparison and won't match.
> yeah they are « dumb » (that is why it is appropriate to
> have the servers answers both A-Labels only _and_ U-Labels
> only domain names queries.

> - the likelyhood (from the end-user perspective) that a domain
> name might be typed or copy/paste as a mixed label to me is,
> in real scenarios, pretty near zero probability.

I've been away from registry and registrar operations and
databases for many years but, to the extent to which end user
queries to those databases still occur (despite the many recent
changes in what is likely to be returned) or other interactions
with end users are considered, I've seen those cases rather
often.  I expect they will disappear when we completely
eliminate situations in which end users see A-labels, but I
expect that will happen right after we successfully eliminate
phishing and other actions that depend on deliberate confusion
of names from the Internet.  The problem, to use the
traditional, stupid, mixed-script label case is that the only
ways a typical user can tell "paypal" from "pаypаl" are to
look at the A-label or to extract the code points (in U+0430
form, %-encoded-UTF8, or something equivalent to one of them).
AFAICT, the preferred solution among application-writers
involves A-labels.  From there, it is a relatively small step to
copying a partial domain name in A-label form, typing in one or
more putative U-labels, and then pasting the two together.  Of
course, any sort of DNS name completion with IDN labels being
used for completion would produce much the same effect unless a
lot of attention is paid to the UI.

Again, maybe this would never occur in RDAP contexts (I just
don't know) but, for the more general case, the likelihood is
considerably above zero.

> - typical end-user will have no clue whatsoever about if the
> two domains (what he typed and what is in the answer) are
> identical or not. He might even think the response (or most
> likely the presentation of the response to the user by the
> « client ») is rogue.

Yes, and that is one of my main concerns, not only about this
case, but about indiscriminate use of any sort of mapping.

> to me, we shall just suggest no, given that a) there is no
> good way to have the RDAP answer return the exact same domain
> in the query; b) the likelihood that an end-user (or even a
> software program) will actually use mix labels is near zero,
> therefore not worth the trouble.

Because of the above, I can't quite get there.  On the other
hand, while it would inconvenience some users, I can imagine the
UI for an RDAP application saying something that amounts to
"NR-LDH labels and U-labels only".  That is more or less the
rule for SMTPUTF8.  In both cases, it would be hard or
impossible to prevent the use of A-labels except by imposing
fairly arbitrary checks but, if the UI is presented that way,
one could rather easily take the position that, if someone
starts using A-labels, they are on their own if anything
confusing happens.

Martin's later comment...

--On Saturday, February 2, 2019 06:32 +0000 "Martin J. Dürst"
<[email-masked]> wrote:

>...
> What I think is important is that the server sends explicit
> and  understandable information back in the Error Response Body

Is something I agree with and is completely consistent with the
above, noting that accurate, explicit, and understandable
user-level information is going to be much easier to produce if
only U-labels and NR-LDH labels are nominally permitted in the
interface.

    best,
    john

   john