Subject: [regext] Benjamin Kaduk's Discuss on draft-ietf-regext-rdap-partial-response-13: (with DISCUSS and COMMENT)

From: Benjamin Kaduk via Datatracker <[email-masked]>
Date: Wed, 09 Sep 2020 11:47:50 -0700

Benjamin Kaduk has entered the following ballot position for
draft-ietf-regext-rdap-partial-response-13: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-regext-rdap-partial-response/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

As was the case for Murray, I'm unconvinced that I have understood what
Section 3 intended to convey.  However, I am balloting Discuss because
my current best understanding is for a statement that seems inconsistent
with my understanding of how the partial response mechanism works.  In
particular, how would the topmost objects be returned according to
different field sets, if there's only a single query parameter and (I
assume) all topmost objects are the results of the same single query?

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks to the shepherd for noting that the examples have been through
JSONLint!

Abstract

   The Registration Data Access Protocol (RDAP) does not include
   capabilities to request partial responses.  Servers will only return
   full responses that include all of the information that a client is
   authorized to receive.  A partial response capability that limits the

[There is perhaps some stylistic question of whether to describe the
present state in the present tense vs. saying something like "prior to
the mechanism defined in this document" or "as specified in [RFC7483".]

Section 1

   Currently, RDAP does not provide a client with any way to request a
   partial response.  Servers can only provide the client with a full

nit: will this age well?

Section 2

   The path segment defined in this section is an OPTIONAL extension of
   search path segments defined in [RFC7482].  This document defines an

It's not entirely clear to me in what sense it's accurate to say that
this section "defines" a path segment.  (Perhaps RDAP path segments
diverge from normal HTTP and generic URI path segments in this sense?)
My current understanding is that we are defining an optional query
parameter in the purest URI sense, which does not require a new path
segment in the URI sense.

   full response (Figure 1).  The field sets supported by a server are
   usually described in out-of-band documents (e.g.  RDAP profile)

nit: comma (and only one space) after "e.g.".

Section 2.1

   responses about the available field sets.  Such information is
   collected in a new data structure named "subsetting_metadata"
   containing the following properties:

I assume this is a JSON datastructure, specifically?

   o  "availableFieldSets": "AvailableFieldSet[]" (OPTIONAL) an array of
      objects, with each element describing an available field set.
      Members are:

      *  "name": "String" (REQUIRED) the field set name;
      *  "default": "Boolean" (REQUIRED) whether the field set is
         applied by default;

Is it allowed to have more than one entry in the array set
"default":true?

      *  "links": "Link[]" (OPTIONAL) an array of links as described in
         [RFC8288] containing the query string that applies the field
         set.

I suggest giving an example or a bit more detail as to exactly what
structure/format is expected here; 8288 has a fair bit of meat to it.
Ah, but this is covered in Section 2.1.2, so maybe just a
forward-reference is enough.  It would be good (either here or there) to
give some indication as to which of "value"/"rel"/"href"/"title"/"type"
are required/optional, though.

Section 4

   o  "id": the server provides only the key field: "handle" for
      entities, "ldhName" for domains and nameservers.  If a returned
      domain or nameserver is an Internationalized Domain Name (IDN)
      [RFC5890], then the "unicodeName" field MUST be included in the

nit: I suggest "also" or "additionally be included".

   The "objectClassName" field is implicitly included in each of the
   above field sets.  RDAP providers SHOULD include a "self" link in
   each field set.  RDAP providers MAY also add any property providing
   service information.

Just to check my understanding: the "property" that might also be added
here is a "field" that would be included in a given field set and
corresponding query response?  (I don't know if "property" has
RDAP-specific connotations that make it a better word to use here.)

(nit?) I'm also not entirely sure that 'include a "self" link in each
field set' is pedantically correct, as opposed to 'include a "links"
field indicating the "self" link relationship'.

Section 8

I suppose it's always possible that by using a narrow field set a client
would not get back some important piece of information, e.g., a modifier
that restricts the scope of applicability of some information.  It seems
fairly obvious that the onus for knowing what fields are possible and
avoiding this scenario falls on the client, though perhaps it is worth
also giving some guidance to the server to take this into consideration
while defining field sets.

   Servers can also define different result limits according to the
   available field sets, so a more flexible truncation strategy can be

I'm not entirely sure I am understanding this correctly -- is it just
saying (roughly) "when using the 'id' field set it's safe to use much
larger page size than for the 'full' field set"?  (I didn't find a
specific definition of "result limits" in RFCs 7482 or 7483.)

   implemented.  The new query parameter presented in this document
   provides RDAP operators with a way to implement a server that reduces
   inefficiency risks.

Perhaps it's worth mentioning that in the face of unupdated clients
these potential gains are not actually realized.

Section 9.1

The one place we cite RFC 7481 does not seem to itself be a normative
usage (though I can imagine that one does need to read it in order to
build a complete system).

Appendix A

   Looking at the implementation experiences of partial response, two
   approaches are observed:

Looking at context from later in the suggestion suggests that this
survey was not restricted to RDAP partial response, which might be worth
mentioning.

   o  The request of some fields might not match the client's access and
      authorization levels.  Clients might request unauthorized fields
      and servers should define a strategy for responding, such as
      always returning an error response or returning a response that

nit: I think this is more of "servers have to" than "servers should"
(even though this strategy might just be "do whatever the existing code
happens to do").

Appendix A.1

   o  Relevant entity object information is included in a jCard, but
      such information cannot be easily selected because it is split
      into the items of a jagged array;

nit: I didn't find "jagged array" used in either RFC 7483 or RFC 7095;
is is relationship to jCard specified somewhere else?

   The latter approach seems to facilitate RDAP interoperability.
   Servers can define basic field sets which, if known to clients, can
   increase the probability of obtaining a valid response.  The usage of

nit: the "latter approach" seems to be an artifact of some text moves,
as there isn't a recent comparison of two approaches in this section.
("Latter approach" also appears in the following paragraph.)

---

From: Mario Loffredo <[email-masked]>
Date: Thu, 10 Sep 2020 19:00:06 +0200

Hi Benjamin,

thanks a lot for your review. Please find my comments inline.

> https://datatracker.ietf.org/doc/draft-ietf-regext-rdap-partial-response/
Perhaps I didn't make myself clear. Obviously, all the objects reutrned 
by a server in response to a search are provided according the same 
field set. What the Secion 3 aims to convey is that servers are free to 
represent relationships between the parent objects and child objects in 
the field sets as they want. For example, an RDAP profile could defne a  
"brief" field set for the response to a /domains search that don't 
include any information about the nameservers. Instead, in another RDAP 
profile the "brief" field set for the response to a /domains search 
could return the same information about the namservers as returned by 
the "brief" field set defined for the response to a /nameservers search.

Therefore, the list of fields of a short response to a /nameservers 
search couldn't be the same as the list of fields for each nameserver 
included in a short response to a /domains search.

Obviously, to achieve the largest interoperability, it's logical to 
expect that the policy about how to represent the information about 
relationships should be quite similar in the various RDAP profiles. But 
this is something out of the scope of the document.

Hope this make the rationale of Section 3 more clear.

Best,

Mario

-- 
Dr. Mario Loffredo
Systems and Technological Development Unit
Institute of Informatics and Telematics (IIT)
National Research Council (CNR)
via G. Moruzzi 1, I-56124 PISA, Italy
Phone: +39.0503153497
Mobile: +39.3462122240
Web: http://www.iit.cnr.it/mario.loffredo

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 10 Sep 2020 13:23:07 -0700

Hi Mario,

> Hope this make the rationale of Section 3 more clear.

Ah, I see the source of my confusion now -- "could be returned according to
different field sets" means that the structure of the topmost object
returned will vary with the field set in use, and "could be applied to
their related objects" means that it's permitted (expected, perhaps) for a
field set definition to include a specification for what fields of related
objects are returned with the query (if any related objects are returned at
all).  Thanks for the clarification!  

If I might propose an alternate wording for your consideration:

   Representation of second level objects within a field set produces
   additional considerations.  Since the representation of the topmost
   returned objects will vary according to the field set in use, the
   response may contain no relationships (e.g., for an abbreviated field
   set) or may contain associated objects as in a normal RDAP query
   response.  Each field set can indicate the format of the additional
   objects to be returned, in the same manner that the format of the
   topmost objects is controlled by the field set.

Thanks,

Ben

---

From: Mario Loffredo <[email-masked]>
Date: Fri, 11 Sep 2020 09:27:17 +0200

Hi Benjamin,

thanks a lot for your reply. Please find my comment below.

>     topmost objects is controlled by the field set.

[ML] Sounds good. I rewrite Section 3. Thanks a lot for your suggestion.

Best,

Mario

---

From: Mario Loffredo <[email-masked]>
Date: Fri, 11 Sep 2020 13:11:03 +0200

Hi Benjamin,

thanks a lot for your extensive review. Please fidn my comments inline.

> https://datatracker.ietf.org/doc/draft-ietf-regext-rdap-partial-response/
[ML] :-)
> nit: comma (and only one space) after "e.g.".
[ML] There is only one space after "e.g." in the XML version. Anyway, 
I'll put the comma.
> I assume this is a JSON datastructure, specifically?
[ML] OK. I'll specify "JSON data structure"
> Is it allowed to have more than one entry in the array set
[ML] Obviously not. Should I furtherly explain that only one field set 
must be applied by default? Isn't implicitly stated by the meaning of 
"applied by default"?
> forward-reference is enough.
[ML] I'll put the reference "(see Section 2.1.2)"
> It would be good (either here or there) to

[ML] In Section 4.2 of RFC7483bis 
<https://datatracker.ietf.org/doc/draft-ietf-regext-rfc7483bis/> is written:

The "value", "rel" and "href" JSON values MUST be specified.  All other
    JSON values are OPTIONAL.

Could be enough to report the same text?

> nit: I suggest "also" or "additionally be included".
[ML] OK. I'll change it to "MUST additionally be included"
> RDAP-specific connotations that make it a better word to use here.)
[ML] it is a required property as defined in Section 4.9 of RFC7483bis 
<https://datatracker.ietf.org/doc/draft-ietf-regext-rfc7483bis/>
> field indicating the "self" link relationship'.
[ML] OK.
> while defining field sets.
[ML] I'm convinced that the number and the content of the field sets 
defined in an RDAP profile should be agreed between producers and 
consumers. I expect that the field sets number and content will be the 
result of a tuning process.
> specific definition of "result limits" in RFCs 7482 or 7483.)
[ML] Yes, exactly. The value "result set truncated due to excessive 
load" is presented among the values defined in Section 10.2.1 of RFC7483 
for "Notice and remark Types". When a server returns such notice, it 
means that server limits have been exceeded.
> these potential gains are not actually realized.

[ML] Could you furtherly explain this concept?

> build a complete system).

[ML] Do you suggest to insert it among the "informative References"? I 
think that even if it isn't essential to implement the proposed 
capability, I believe it's needful to achieve a comprehensive 
implementation that takes into account the relationship between the 
availability and the content of each fiedl set and the user access profiles.

For this reason, I put it among the "Normative References"

> survey was not restricted to RDAP partial response, which might be worth

[ML] OK. Could be enough to update the sentence as in the following?

"Looking at the implementation experiences of partial response offered 
by data providers on the web,  two approaches are observed"

> (even though this strategy might just be "do whatever the existing code
[ML] OK. I change that sentence as you suggest.
> is is relationship to jCard specified somewhere else?
[ML] It is not a jCard specific term. It is a term generally used in the 
programming languages to refer to an array of arrays of which the member 
arrays can be of different sizes. The straightforward deserialization of 
a jCard produces a jagged array.
> ("Latter approach" also appears in the following paragraph.)
[ML] I'll replace "latter approach" with "the field sets approach".

Hope I forgot nothing.

Looking forward for your reply to my comments and proposed updates.

Best,

Mario

---

From: Benjamin Kaduk <[email-masked]>
Date: Fri, 11 Sep 2020 19:30:16 -0700

Hi Mario,

Also inline.

> must be applied by default? Isn't implicitly stated by the meaning of 

I think it's probably worth saying what the error handling is if more than
one is set -- treat the whole thing as malformed, or pick one?

> [ML] I'll put the reference "(see Section 2.1.2)"

Thanks!

> Could be enough to report the same text?

That would work, since presumably we don't want to make this document wait
until that one is done.

> <https://datatracker.ietf.org/doc/draft-ietf-regext-rfc7483bis/>

Sorry, I was trying to ask about "MAY also add any property providing
service information" -- specifically, about how that is conveyed in the
response.  (My assumption is just as a field of the topmost JSON object.)

> result of a tuning process.

I agree.  I am wondering if we should give advice to the producer, pointing
out that if done badly, a "smaller" field set might omit critical
information and lead the consumer to behave in a dangerous or unexpected
way.

> means that server limits have been exceeded.

I see now; thanks for the pointer and explanation.

> [ML] Could you furtherly explain this concept?

If the server implements partial responses, but all clients do not, the
server still has to spend resources producing full responses for all
clients.  The efficiency gains and risk reduction seem to be proportional
to the fraction of clients that support partial responses.

> For this reason, I put it among the "Normative References"

Your reasoning seems sound; please leave it where it is.

> by data providers on the web,  two approaches are observed"

I think so; thanks.

> a jCard produces a jagged array.

I see, thanks.  (Can you tell I mostly only program in C?)

> Looking forward for your reply to my comments and proposed updates.

I appreciate your responses and updates; they look good to me.

Thanks!

-Ben

---

From: Mario Loffredo <[email-masked]>
Date: Sat, 12 Sep 2020 20:13:04 +0200

Hi Benjamin,

thanks a lot for your quick response. My comments are inline.

> one is set -- treat the whole thing as malformed, or pick one?
[ML] OK. I'll clarify that a server MUST define only one field set as 
default.
> That would work, since presumably we don't want to make this document wait
[ML] OK. I'll add that sentence.
> response.  (My assumption is just as a field of the topmost JSON object.)
[ML] The term "service information" is used there to identify other 
information than an object property like remarks, notices and other 
possible metadata.
> information and lead the consumer to behave in a dangerous or unexpected
[ML] I think a similar sentence would imply a consequent clarification 
of the factors revealing that a field set is done badly.  I'm confident 
that the interaction between clients and servers will produce field sets 
with the right granularity.
> to the fraction of clients that support partial responses.
Yes, this is true. Anyway, I expect that servers will implement measures 
to discourage the massive submission of search queries requesting the 
full response. Moreover, I don't think that supporting partial response 
will be a big problem for clients because it consists merely in 
displaying less information than the full response.
> I appreciate your responses and updates; they look good to me.

Please confirm me that no other changes are needed so I can submit the 
new version

Best,

Mario

>
> -Ben
>
> _______________________________________________
> regext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/regext

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 12 Sep 2020 15:36:06 -0700

Hi Mario,

> thanks a lot for your quick response. My comments are inline.
[...]
> Please confirm me that no other changes are needed so I can submit the 

Thanks for the additional explanations; I have no further comments.
Please go ahead and submit.

-Ben

---

From: Barry Leiba <[email-masked]>
Date: Thu, 17 Sep 2020 12:34:03 -0400

Ben, version -15 was submitted.  Can you give it a look, please?

Thanks,
Barry

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 17 Sep 2020 21:26:57 -0700

Hi Barry,

Sure, just did.  Sorry it took a bit -- I've been a bit sick this week and
everything has been moving slowly.

-Ben