Subject: Re: [weirds] WEIRDS inactivity, and issues raised in London

From: Ernie Dainow <[email-masked]>
Date: Fri, 18 Jul 2014 11:08:41 -0400

On Sun, Jun 22, 2014 at 7:03 PM, Tom Harrison <[email-masked]> wrote:

> On Fri, Jun 13, 2014 at 06:17:32AM -0400, Ernie Dainow wrote:
> > The argument to avoid "describing a subset of the possible error
> conditions"
> > is contradicted by what is already in draft-ietf-weirds-using-http.
> Section
> > 5 contains several sections covering a subset of HTTP errors, including
> 400
> > and 404.
> >
> > Without further guidelines, there will be a lot of inconsistencies
> between
> > implementations. In the last inter-op test there were a number of cases
> > where XXXX was an invalid name (such as example...com) or an invalid
> UTF-8
> > representation. Some believed that 400 should be returned, others
> > interpreted 404 as the proper error.
> >
> > I suggest a statement to the effect that if the error is related to an
> > invalid value of the parameter, 404 should be returned; but if the error
> is
> > syntax or semantic (such as an invalid parameter), then 400 is returned.
> >
> > If 400/404 are used consistently, this provides useful feedback to help
> > diagnose the problem. 404 is due to the user input, whereas 400 is
> generally
> > a fault in the client implementation.
>
> I think using 400 for all client errors is the better approach. Most
> servers will return a description of the error in the body of the
> response (as per json-response), so the interactive use case is
> covered regardless of the error code, since the client program can
> display that error description to the user. However, in the case where
> the access is scripted/automated, or any other situation where the
> only thing from the response that can be depended on is the status
> code, it's more useful for 404 to be limited to the "valid query but
> there's no record" scenario.
>

I agree with this, perhaps my wording did not make it clear.

>
> For example, a user might have a scheduled script that fetches a
> series of domain names, some being non-IDNs and some being IDNs. If
> the script is not encoding the IDNs properly, then all of the IDN
> requests would yield "not found", which would be quite a different
> meaning in this case from any "not found" returned for a non-IDN. More
> insidious would be a change in how the server processes queries on the
> server side, such that a previously-valid request, returning a "not
> found 404" or a successful response, starts returning a "client error
> 404".
>

I don't think a server should be expected to validate the input before it
does a lookup. It is a lot simpler for a server just to do a lookup and
return the record or 404.

Here is modified suggested wording for Section 5.4 in
draft-ietf-weirds-using-http that may be more clear:

5.4.  Malformed Queries

If a server receives a query which it cannot interpret as an RDAP query, it
returns a 400 (Bad Request) response code.  Optionally, it MAY include
additional information regarding this negative answer in the Error Response
Body defined in draft-ietf-weirds-json-response.

A 400 response code is not returned to indicate an invalid value of a
parameter. In this case a 404 (Not Found) is returned. For example, if
domain/XXXX is a valid query, where XXXX is a domain name, 404 would be
returned for an invalid name such as example..com. If the server
distinguishes between an invalid name and record not found, it can indicate
this in the Error Response Body.

-Ernie

---

From: "Linlin Zhou" <[email-masked]>
Date: Tue, 22 Jul 2014 15:20:08 +0800

>I don't think a server should be expected to validate the input before it does a lookup. It is a lot simpler for a server just to do a lookup and return the record or 404. 

Agreed. Especially in the case of IDN domain validation check.
If a domain is a fake A-label, such as xn--7warambrecmdctfmhlrtrmbpyjdccaaahkrgl4chlip9bclcgddbb4ooioa.bnnhg (since the A-label corresponds to an invalid U-label). We don't check the domain which will be treated as a correct domain. The server response is 404 or 200 in JSON format.

>Here is modified suggested wording for Section 5.4 in draft-ietf-weirds-using-http that may be more clear:

>If a server receives a query which it cannot interpret as an RDAP query, it returns a 400 (Bad Request) response code.  Optionally, it MAY include additional information regarding this negative answer in the Error 
>Response Body defined in draft-ietf-weirds-json-response. 
   
>A 400 response code is not returned to indicate an invalid value of a parameter. In this case a 404 (Not Found) is returned. For example, if domain/XXXX is a valid query, where XXXX is a domain name, 404 would be 
>returned for an invalid name such as example..com. If the server distinguishes between an invalid name and record not found, it can indicate this in the Error Response Body. 

Regards,
Linlin

---

From: Tom Harrison <[email-masked]>
Date: Wed, 23 Jul 2014 02:46:52 -0400

> If a domain is a fake A-label, such as xn--7warambrecmdctfmhlrtrmbpyjdccaaahkrgl4chlip9bclcgddbb4ooioa.bnnhg (since the A-label corresponds to an invalid U-label). We don't check the domain which w…

It is certainly easier in the /domain case to just perform the lookup.
However, in the /ip and /autnum cases, it is necessary to parse the
argument to respond to the request, and it is consequently very easy
to return a 400 on receiving invalid input. This remains a useful
distinction to make, especially for HEAD requests and in situations
where access is scripted/automated.

I still don't think the semantics of 404 are appropriate for a
response to a request containing a syntactically-invalid domain name,
but I can see why avoiding the validation is a useful thing. Maybe
text along the following lines will suit:

    5.4 Malformed Queries

    If a server receives a query which it cannot interpret as an RDAP
    query, it returns a 400 (Bad Request) response code.  Optionally,
    it MAY include additional information regarding this negative
    answer in the Error Response Body defined in
    draft-ietf-weirds-json-response.

    For some query types, it is possible to respond to valid queries
    correctly without performing syntactic validation of the query
    parameters.  A server MAY omit to perform this validation for one
    or more of these query types.  For the types for which the server
    does not perform validation, it returns a 404 (Not Found) response
    code on receiving an invalid query.

-Tom

---

From: Maarten Bosteels <[email-masked]>
Date: Wed, 30 Jul 2014 15:51:34 +0000

On 23/07/14 02:46, "Tom Harrison" <[email-masked]> wrote:

>> >I don't think a server should be expected to validate the input before
>>it does a lookup. It is a lot simpler for a server just to do a lookup
>>and return the record or 404.
>> 
>> If a domain is a fake A-label, such as
>>xn--7warambrecmdctfmhlrtrmbpyjdccaaahkrgl4chlip9bclcgddbb4ooioa.bnnhg
>>(since the A-label corresponds to an invalid U-label). We don't check
>>the domain which will be treated as a correct domain. The server
>>response is 404 or 200 in JSON format.
>

I agree that 404 is not an appropriate response to a request containing a
syntactically invalid domain name.
And *if* we think skipping or avoiding validation is useful, why do we
specify that the server MUST convert to U-label format and do the lookup
by A-label ?

http://tools.ietf.org/html/draft-ietf-weirds-rdap-query-10#section-3.1.3

  If the client sends the server an IDN in U-label format, servers that
  support IDNs MUST convert the IDN into A-label format and perform
  IDNA processing as specified in RFC 5891
<http://tools.ietf.org/html/rfc5891> [RFC5891
<http://tools.ietf.org/html/rfc5891>].  The server
  should perform an exact match lookup using the A-label."

So, U-label input must be validated and converted to A-label before doing
the lookup but putative A-label input does not need validation before
doing the lookup ?

I know it was discussed before
 http://www.ietf.org/mail-archive/web/weirds/current/msg02357.html
and I do understand the "behave as if" argument but I don't understand why
we have to be so strict on how the server should implement the lookup of
putative U-labels and at the same time allow to skip validation of
putative A-labels.

One of the benefits of IDNA2008 is that you can safely convert from
U-label to A-label and back again, so you could as well do an exact match
on U-label (given that it is a valid U-label).
This would also better align with what is written in Section 4 about query
processing: 
"In general, storage of strings as Unicode is RECOMMENDED."

I would suggest 

A) to remove the sentence which says that the server should do an exact
lookup on A-label.
The lookup should be conform the IDNA2008 rules but in my opinion it's up
to the implementor to decide how to do it.

B) To respond with 400 instead of 404 in case of a syntactically invalid
domain name.

regards
Maarten

>_______________________________________________
>weirds mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/weirds

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 30 Jul 2014 14:22:44 -0400

> syntactically invalid domain name.

I worry that I'm not sure what "syntactically invalid domain name" is
in that sentence.

Keep in mind that "‘’„‼※⁙⁕" (without the ASCII quotes -- those are all
Unicode code points in the punctuation category, so they should all be
DISALLOWED under IDNA2008) is a perfectly legal DNS label.  It is
clearly not a conforming LDH-label, it is not a U-label because it
cannot be converted to an A-label, but it _can_ be represented in
IDNA2003 (the Punycode output is dvgcp0je5m1a, so the IDNA2003 label
is xn--dvgcp0je5m1a).  Which of these do you mean?

> specify that the server MUST convert to U-label format and do the lookup

This is because, if you're doing IDNA, you're publishing A-labels in
the DNS.  There was an early assumption that what you should be able
to look up in the RDAP is the name from the DNS, and if it's a U-label
is will _never_ be in that format in the DNS.   

My own view, for what it's worth, is that a server needs to use either
A-label or U-label form, and always use it.  There are a lot of
possible pitfalls in U-label forms, however, and several people said
that client-side mapping was not going to work.  In that case, you
might well get a U-label that is not the right one someone was looking
for, and it'd be good to be able to throw an error for "bad U-label"
as opposed to "no match".

> on U-label (given that it is a valid U-label).

That is true.  But just as true is that you can convert.

Best regards,

A
-- 
Andrew Sullivan
[email-masked]