Subject: [T2TRG] draft-hartke-t2trg-ciri-00 review

From: Ari Keränen <[email-masked]>
Date: Mon, 21 Jan 2019 18:25:17 +0000

Hi,

I reviewed the CIRI -00 draft. Overall it looked very good to me but 
here's a few comments/questions.

2.1.  Options

    host.ip
      Specifies the host of the IRI authority as an IPv4 address
      (4 bytes) or an IPv6 address (16 bytes).

Do we need endianess considerations here? Or does CBOR take care of that?

2.2.  Option Sequences

     A sequence of options is considered _well-formed_ if:

Do we need the _emphasis_ here? Maybe in quotes since it's a defined 
term? Unless we want to follow the CDDL convention -- which perhaps 
makes sense. But then it's probably good to have a note along the lines 
of "New terms are introduced in _cursive_." in the terminology section.

     o  a "host.ip" option is followed by a "port" option;

Why is the port option mandatory? No default ports allowed?

3.  CBOR

       ciri = [?(scheme:    1, text),
               ?(host.name: 2, text //
                 host.ip:   3, bytes .size 4 / bytes .size 16),
               ?(port:      4, uint .size 2),
               ?(path.type: 5, path-type),
               *(path:      6, text),
               *(query:     7, text),
               ?(fragment:  8, text)]>

Isn't "text" too permissive type for most (all?) of the components?

Cheers,
Ari

---

From: Dave Thaler <[email-masked]>
Date: Mon, 21 Jan 2019 20:11:19 +0000

FYI, the IRI WG intended to update RFC 3987 and shut down because it could not reach
consensus on IRIs.  In particular, RFC 3987 does not match what major implementations
(i.e. browsers) actually do, and so there's a major incompatibility between current practice
and what RFC 3987 does.  There does exist at least one RFC 3987 implementation out there
in Windows WinRT APIs though, but I don't know if there's others.   In practice, the
recommendation from the internationalization community in the IETF tends to be to not
use IRIs.   That is, protocols should only ever put URIs (not IRIs) on the wire, and conversion
from internationalized strings to/from URIs is done at the UI layer.

With that in mind, the case for using IRIs (as opposed to URIs) in Constrained Restful
Environments seems to not have much real-world applicability since it goes against
the internationalization guidance.

It's fine for a research project, which T2TRG is allowed to do, but may not be very useful
in practice, from what I've seen.

Dave

P.S.: An example of one incompatibility between RFC 3987 and what is widely implemented
in browsers, is how percent encoding is interpreted.  In RFC 3987 it's percent-encoded UTF-8,
whereas in implementations it's percent-encoded in whatever the charset is of the document
(or whatever) you got the IRI out of, which may or may not be UTF-8.   Again this is just one
of many examples.

-----Original Message-----
From: T2TRG <[email-masked]> On Behalf Of Ari Keränen
Sent: Monday, January 21, 2019 10:25 AM
To: [email-masked]
Cc: [email-masked]
Subject: [T2TRG] draft-hartke-t2trg-ciri-00 review

Hi,

I reviewed the CIRI -00 draft. Overall it looked very good to me but here's a few comments/questions.

2.1.  Options

    host.ip

2.2.  Option Sequences

3.  CBOR

Cheers,
Ari
_______________________________________________
T2TRG mailing list
[email-masked]
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.irtf.org%2Fmailman%2Flistinfo%2Ft2trg&amp;data=02%7C01%7Cdthaler%40microsoft.com%7C8840ed3d4dce47e1bc6208d67fcdd224%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636836919304296169&amp;sdata=7HFctjFUzx0n8M03K7bSeWeFFYJ5TRpxINcWq%2Byadks%3D&amp;reserved=0

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 21 Jan 2019 21:31:33 +0100

Hi Dave,

I’m not sure that we need to carry all that baggage.

> (or whatever) you got the IRI out of, which may or may not be UTF-8.   Again this is just one

“Then don’t do that.”  For CORE, we already have decided to inhabit a UTF-8-only world.
We also may not need to use percent encoding at all for CIRIs.

In summary, I’m not sure the experience the Internationalization community has with IRIs carries very far in the simplified world created by CORE and CIRIs.

But I would sure be interested which problems IRIs created, in particular those that were not with legacy encodings.
Further pointers are very welcome.

Grüße, Carsten

---

From: Dave Thaler <[email-masked]>
Date: Mon, 21 Jan 2019 20:35:52 +0000

Once you have COAP/HTTP proxies, you pull in all that baggage.

Furthermore, the conventional wisdom is that there's no problem that
putting IRIs on the wire solves that isn't better solved by putting URIs
on the wire.

I understand you're asking for pointers, and I'll bring that up with the
IETF Internationalization Directorate that's now being formed.

Dave

-----Original Message-----
From: Carsten Bormann <[email-masked]> 
Sent: Monday, January 21, 2019 12:32 PM
To: Dave Thaler <[email-masked]>
Cc: Ari Keränen <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [T2TRG] draft-hartke-t2trg-ciri-00 review

Hi Dave,

E.g.:

> P.S.: An example of one incompatibility between RFC 3987 and what is 
> widely implemented in browsers, is how percent encoding is 
> interpreted.  In RFC 3987 it's percent-encoded UTF-8, whereas in implementations it's percent-encoded in whatever the charset is of the document

Grüße, Carsten

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 21 Jan 2019 21:37:53 +0100

Hi Dave,

thanks for bringing up this interesting point.

The draft essentially describes a CBOR variant of the way CoAP
serializes the Request URI: The URI is split into its components (+
the path component into its segments) and each component becomes one
"option". In these options, no percent encoding is performed. So, as I
see it, the following are all serializations of the same data:

URI:
   coap://example.com:5683/city/Montr%C3%A9al

IRI:
   coap://example.com:5683/city/Montréal

CoAP:
   Uri-Host: "example.com"
   Uri-Port: 5683
   Uri-Path: "city"
   Uri-Path: "Montréal"

CIRI:
   [1, "coap",
    2, "example.com",
    4, 5683,
    6, "city",
    6, "Montréal"
   ]

I don't think the problems of one serialization have much impact on
the standardization of a new serialization.

(What's the name of the data model of which URI, IRI, CoAP and CIRI
are serializations?)

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 21 Jan 2019 22:58:55 +0100

> Once you have COAP/HTTP proxies, you pull in all that baggage.

Right. Currently, converting a CIRI to CoAP options is defined as:

1. Recompose the CBOR-encoded IRI components to an IRI
2. Convert the IRI to a URI according to RFC 3987
3. Decompose the URI to CoAP options according to RFC 7252

It sounds like a good idea to rewrite section 4.2 to output a URI and
define the conversion as:

1. Recompose the CBOR-encoded IRI components to a URI
2. Decompose the URI to CoAP options according to RFC 7252

> putting IRIs on the wire solves that isn't better solved by putting URIs

The Semantic Web relies extensively on IRIs, and some formats like
Turtle put IRIs "on the wire" rather than URIs. E.g.:

   @prefix loc: <http://example.org/location/> .

   loc:Montréal a loc:city .

Would this be considered "UI layer"? If not, I'm not sure if URIs
would improve things:

   loc:Montr%C3%A9al a loc:city .

(Is that even valid?)

Since the CoRAL textual format [1] uses IRIs in a similar way, some
advice on IRIs in formats intended to be read and written by humans
would be very welcome.

Klaus

[1] https://tools.ietf.org/html/draft-hartke-t2trg-coral-06

---

From: Dave Thaler <[email-masked]>
Date: Mon, 21 Jan 2019 23:22:29 +0000

I agree that "Recompose the CBOR-encoded IRI components to a URI" could be
done without referencing RFC 3987 and without using the term "IRI" (as opposed to CIRI
which you can define).  Although I wonder if in so doing, you'd run into all the same
issues that failed to achieve consensus with the rfc3987bis.

And I also agree that draft-hartke-t2trg-coral-06 likely has the same issues because
it uses IRIs instead of URIs.

-----Original Message-----
From: Klaus Hartke <[email-masked]> 
Sent: Monday, January 21, 2019 1:59 PM
To: Dave Thaler <[email-masked]>
Cc: Carsten Bormann <[email-masked]>; [email-masked]; Ari Keränen <[email-masked]>; [email-masked]
Subject: Re: [T2TRG] draft-hartke-t2trg-ciri-00 review

1. Recompose the CBOR-encoded IRI components to an IRI 2. Convert the IRI to a URI according to RFC 3987 3. Decompose the URI to CoAP options according to RFC 7252

It sounds like a good idea to rewrite section 4.2 to output a URI and define the conversion as:

1. Recompose the CBOR-encoded IRI components to a URI 2. Decompose the URI to CoAP options according to RFC 7252

> putting IRIs on the wire solves that isn't better solved by putting 
> URIs on the wire.

The Semantic Web relies extensively on IRIs, and some formats like Turtle put IRIs "on the wire" rather than URIs. E.g.:

   @prefix loc: <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.org%2Flocation%2F&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cc9d0643be43d49fc77ab08d67febbc9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636837047803044411&amp;sdata=AiwjzFDEBICXw%2B%2BpJOUok5l1eTBaszbiQl8YW6n91Ng%3D&amp;reserved=0> .

Would this be considered "UI layer"? If not, I'm not sure if URIs would improve things:

(Is that even valid?)

Since the CoRAL textual format [1] uses IRIs in a similar way, some advice on IRIs in formats intended to be read and written by humans would be very welcome.

Klaus

[1] https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-hartke-t2trg-coral-06&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cc9d0643be43d49fc77ab08d67febbc9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636837047803044411&amp;sdata=HbsWjAZLY%2B6kw2qQrarD6pt2478qZsZcRyflKk5VoRE%3D&amp;reserved=0

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 22 Jan 2019 13:01:47 +0100

Hi Ari,

thanks a lot for your review!

> Do we need endianess considerations here? Or does CBOR take care of that?

Is there any protocol or format that does not encode 192.0.2.42 as
h'C000022A' or [2001:db8::42] as h'20010DB80000000000000000000042'? I
always thought that IP addresses do not have an endianness.

> of "New terms are introduced in _cursive_." in the terminology section.

Yes, the idea was to write terms defined by the document in cursive
where they are introduced. I'll add a note to the terminology section.

> Why is the port option mandatory? No default ports allowed?

This is relates to a tricky question regarding design goals.

As defined by RFC 3986, URIs can often be written in a number of ways:

* Schemes can be written in upper- or lowercase but are case-insensitive.
* Registered name are case-insensitive as well.
* Hex characters in percent encodings are case-insensitive as well.
* Schemes can define scheme-specific rules. E.g., in the case of "coap":
    * The port 5683 and an omitted port are equivalent.
    * The path "" and "/" are equivalent.

The primary function of CIRI References is to dereference them (where
RFC 3986 defines "dereferencing a URI" as using an access mechanism
determined by "URI resolution" to perform an action on the URI's
resource). For that, a client only needs to compare URI schemes, pass
registered names to a name resolution service, and know the default
ports of the protocols it actually implements.

The question is if we secondarily also want to be able to compare
CIRIs. Normally, because of all of these normalization rules, this is
quite tricky to do for all URI schemes (in particular if a URI scheme
has scheme-specific rules and is not recognized by an implementation).
However, if we restrict CIRIs to follow the normalization rules of
CoAP, then we can actually support any URI scheme with the same
normalization rules (such as HTTP) with ease. The only thing we need
to know to successfully compare two CIRIs with an unrecognized scheme
is - the default port.

So, at the cost of two bytes in a CIRI with a default port, we gain
that we can compare any supported CIRI even when the scheme is
unrecognized. Is it worth it? In my opinion, yes - if we want to be
able to compare CIRIs. On that, I don't have a good opinion yet. What
do you think?

> Isn't "text" too permissive type for most (all?) of the components?

RFC 3986 actually makes no restrictions at all on what characters can
be used in registered names, paths segments, queries and fragment
identifiers. (The only requirement is that some of them need to be
percent-encoded.) So, as I understand it, "text" (a.k.a. "tstr") is
the right match here.

The scheme is restricted with a regex in -00:

      ciri = [?(scheme:    1, text .regexp "[A-Za-z][A-Za-z0-9+.-]*"),
              ...

Klaus

---

From: Ari Keränen <[email-masked]>
Date: Wed, 23 Jan 2019 09:05:37 +0000

Hi Klaus,

> On 22 Jan 2019, at 14.01, Klaus Hartke <[email-masked]> wrote:
> 
> Hi Ari,
> 

Nothing obvious comes to my mind, but I remember in some other contexts (e.g., HITs in HIP specs; which are essentially IPv6 addresses) we were careful to remind about the byte order, especially for crypto operations. But perhaps that is not that relevant here, so I'm fine either way.

> where they are introduced. I'll add a note to the terminology section.

Sounds good, thanks!

> able to compare CIRIs. On that, I don't have a good opinion yet. What

Seems like a sensible design choice. But since it's non-obvious, should it be mentioned somewhere in the document?

>      ciri = [?(scheme:    1, text .regexp "[A-Za-z][A-Za-z0-9+.-]*"),

Does this mean also control characters like new-line are OK, or is there need for restricting that? Should we express the need for percent-encoding in CDDL? 

Cheers,
Ari

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 23 Jan 2019 18:57:09 +0100

> Seems like a sensible design choice. But since it's non-obvious, should it be mentioned somewhere in the document?

Yes. The section on comparisons is currently missing, mostly because
I'm not sure if we actually need to be able to compare. If not, we
could also support URI schemes with different normalization rules. So,
as mentioned in [1], it would be nice to have a discussion on what URI
schemes we want to support and whether comparisons are actually
needed.

> Does this mean also control characters like new-line are OK, or is there need for restricting that? Should we express the need for percent-encoding in CDDL?

I don't see anything in RFC 3986 that would prevent percent-encoded
control characters e.g. in a path segment:
<http://example.com/First%20Line%0D%0ASecond%20Line/>.

Since in CBOR text strings are prefixed by their length, we don't need
to do any percent encoding there:
[1, "http", 2, "example.com", 4, 80, 6, "First Line\r\nSecond Line", 6, ""]
(in CBOR diagnostic notation).

Klaus

[1] https://mailarchive.ietf.org/arch/msg/core/ITSAbkgxMhlgzwn-20Jtk6mfN04