Subject: [T2TRG] IRIs in CoRAL (was: draft-hartke-t2trg-ciri-00 review)

From: Klaus Hartke <[email-masked]>
Date: Mon, 4 Feb 2019 12:46:07 +0100

> And I also agree that draft-hartke-t2trg-coral-06 likely has the same
> issues because it uses IRIs instead of URIs.

Some further thoughts:

* In CoAP, the request URI is transported as a sequence of of CoAP
options that contain the different parts of an URI without
percent-encoding. For example, the URI
<http://example.com/city/Montr%C3%A9al> in a request to (http,
example.com, 80) would be encoded as an Uri-Path option containing
utf8(decode-percent-encodings("city")) = h'63697479' followed by an
Uri-Path option containing
utf8(decode-percent-encodings("Montr%C3%A9al")) =
h'4d6f6e7472c3a9616c'.

CoAP does not require any Unicode normalization be performed, so if a
client happens to make a request with an Uri-Path option with
utf8(nfd("Montréal")) = h'4d6f6e7472_65cc81_616c' where the server
expects an Uri-Path option with utf8(nfc("Montréal")) =
h'4d6f6e7472_c3a9_616c' (or vice versa), then the client will get a
4.04 Not Found error.

CoAP defines a conversion from CoAP options to URIs (and vice versa).
This conversion is purely syntactic, so an Uri-Path option with
h'4d6f6e7472_65cc81_616c' in the request URI would become
<http://example.com/city/Montr%65%CC%81al>.

CoRAL, in the binary format, does exactly the same for link targets
(except that the conversion of CIRI options is currently defined to be
to IRIs, which I'll replace with URIs in the upcoming
draft-hartke-t2trg-ciri-01).

* In Web Linking [RFC8288], the context and the target of a link are
IRIs. However, these are serialized as URIs on the wire in the "Link"
header field.

CoRAL, in the binary format, does exactly the same for link targets
(except that it uses CBOR instead of ASCII characters to delimit the
URI components on the wire).

* In RDF, concepts are named with globally unique Unicode strings. To
make the minting of these strings painless, they are restricted to the
syntax of an IRI. These IRIs are used purely as identity tokens (in
RFC3987 lingo) and are therefore compared character-by-character.

RDF recommends [1] that these IRIs avoid non-normalized forms such as
uppercase characters in scheme names, explicitly stated HTTP default
port, percent-encoding of characters where it is not required by IRI
syntax, and IRIs that are not in NFC. So for example the concept
identified by the IRI <http://example.com/city/Montréal> is not the
same as the concept identified by the IRI
<http://example.com/city/Montréal> if one of them isn't in NFC.

CoRAL, in the binary format, does exactly the same for link relation types.

Now, in M2M communication, I think we avoid all problems with
normalization etc.: Servers authoritatively manage the namespace of
their resources. If a client asks a server "Hey, server, what
resources do you have?" and the server responds with "I have a
resource at [6, h'63697479', 6, h'4d6f6e7472c3a9616c'].", then the
client can simply copy those bytes into its next request without ever
decoding them. As long as the server accepts its own output as input,
everything works.

When a client compares link relation types to locally stored strings,
it can use byte-for-byte comparison (as suggested by RFC3987) as long
as both the server and the client store the link relation types
exactly as they are defined.

The only issue left is when human users input IRIs as link relation
types or as link targets. In CoRAL, this happens in the textual
format. Interestingly, Turtle [2] doesn't seem to perform any kind of
input normalization, so human users are expected to write in perfect
NFC when they identify a concept by the IRI. (Maybe I'm missing
something?) If true, this seems like a bad user experience. However, I
think it would be a bad user experience as well if one has to write
<%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF> instead of <こんにちは>.

I would prefer to not invent anything new here and to just follow the
consensus if possible.

Klaus

[1] https://www.w3.org/TR/rdf11-concepts/#note-iris
[2] https://www.w3.org/TR/turtle/

---

From: Dave Thaler <[email-masked]>
Date: Mon, 4 Feb 2019 19:17:38 +0000

Inline below…

-----Original Message-----
From: Klaus Hartke <[email-masked]>
Sent: Monday, February 4, 2019 3:46 AM
To: Dave Thaler <[email-masked]>
Cc: [email-masked]
Subject: IRIs in CoRAL (was: draft-hartke-t2trg-ciri-00 review)

Some further thoughts:

* In CoAP, the request URI is transported as a sequence of of CoAP options that contain the different parts of an URI without percent-encoding.

In contrast IRIs do have some percent encoding, in three cases:

1)     To escape reserved characters

2)     When the byte sequence is not UTF-8 (e.g., binary bytes)

3)     To escape characters not appropriate in an IRI

See section 3.2 of RFC 3987 for details.

For example, the URI <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%25C3%25A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=4nYM1%2Bj2Zz52XgEGgnuBasmJdgkf5E38zr7cpBsm3IM%3D&amp;reserved=0> in a request to (http, example.com, 80) would be encoded as an Uri-Path option containing

utf8(decode-percent-encodings("city")) = h'63697479' followed by an Uri-Path option containing

utf8(decode-percent-encodings("Montr%C3%A9al")) = h'4d6f6e7472c3a9616c'.

CoAP does not require any Unicode normalization be performed, so if a client happens to make a request with an Uri-Path option with

utf8(nfd("Montréal")) = h'4d6f6e7472_65cc81_616c' where the server expects an Uri-Path option with utf8(nfc("Montréal")) = h'4d6f6e7472_c3a9_616c' (or vice versa), then the client will get a

4.04 Not Found error.

Most people would say that’s a problem (as you point out at the bottom of your message).

IRI’s (in RFC 3987) on the other hand do require normalization.

This conversion is purely syntactic, so an Uri-Path option with h'4d6f6e7472_65cc81_616c' in the request URI would become <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%2565%25CC%2581al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=A88h0r8EuhVyju889ip7swWUFgn2laYUjbXnSDVmO2k%3D&amp;reserved=0>.

CoRAL, in the binary format, does exactly the same for link targets (except that the conversion of CIRI options is currently defined to be to IRIs, which I'll replace with URIs in the upcoming draft-hartke-t2trg-ciri-01).

* In Web Linking [RFC8288], the context and the target of a link are IRIs. However, these are serialized as URIs on the wire in the "Link"

header field.

CoRAL, in the binary format, does exactly the same for link targets (except that it uses CBOR instead of ASCII characters to delimit the URI components on the wire).

* In RDF, concepts are named with globally unique Unicode strings. To make the minting of these strings painless, they are restricted to the syntax of an IRI. These IRIs are used purely as identity tokens (in

RDF recommends [1] that these IRIs avoid non-normalized forms such as uppercase characters in scheme names, explicitly stated HTTP default port, percent-encoding of characters where it is not required by IRI syntax, and IRIs that are not in NFC. So for example the concept identified by the IRI <https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> is not the same as the concept identified by the IRI <https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> if one of them isn't in NFC.

Now, in M2M communication, I think we avoid all problems with normalization etc.: Servers authoritatively manage the namespace of their resources. If a client asks a server "Hey, server, what resources do you have?" and the server responds with "I have a resource at [6, h'63697479', 6, h'4d6f6e7472c3a9616c'].", then the client can simply copy those bytes into its next request without ever decoding them. As long as the server accepts its own output as input, everything works.

When a client compares link relation types to locally stored strings, it can use byte-for-byte comparison (as suggested by RFC3987) as long as both the server and the client store the link relation types exactly as they are defined.

The only issue left is when human users input IRIs as link relation types or as link targets. In CoRAL, this happens in the textual format. Interestingly, Turtle [2] doesn't seem to perform any kind of input normalization, so human users are expected to write in perfect NFC when they identify a concept by the IRI. (Maybe I'm missing

something?) If true, this seems like a bad user experience. However, I think it would be a bad user experience as well if one has to write <%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF> instead of <こんにちは>.

It’s not just user experience, it might cause security vulnerabilities if such a string is used for security purposes.

See RFC 6943.

I would prefer to not invent anything new here and to just follow the consensus if possible.

One typical consensus in the IETF is to put URIs, not IRIs, on the wire.

Dave

Klaus

[1] https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Frdf11-concepts%2F%23note-iris&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049783845&amp;sdata=BqgcQRFWj1DpNuOU%2FNCCMvHRMwTDFmzJKtMujHcMsag%3D&amp;reserved=0

[2] https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Fturtle%2F&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049793804&amp;sdata=WS%2BWnFEFffWRohqs7EpOe4FZyiFrgKre89gY630mT98%3D&amp;reserved=0

---

From: Jim Schaad <[email-masked]>
Date: Mon, 4 Feb 2019 17:21:12 -0800

From: T2TRG <[email-masked]> On Behalf Of Dave Thaler
Sent: Monday, February 4, 2019 11:18 AM
To: Klaus Hartke <[email-masked]>
Cc: [email-masked]
Subject: Re: [T2TRG] IRIs in CoRAL (was: draft-hartke-t2trg-ciri-00 review)

Inline below…

-----Original Message-----
From: Klaus Hartke <[email-masked] <mailto:[email-masked]> > 
Sent: Monday, February 4, 2019 3:46 AM
To: Dave Thaler <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> 
Subject: IRIs in CoRAL (was: draft-hartke-t2trg-ciri-00 review)

Some further thoughts:

1.	To escape reserved characters
2.	When the byte sequence is not UTF-8 (e.g., binary bytes)
3.	To escape characters not appropriate in an IRI

For example, the URI < <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%25C3%25A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=4nYM1%2Bj2Zz52XgEGgnuBasmJdgkf5E38zr7cpBsm3IM%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%25C3%25A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=4nYM1%2Bj2Zz52XgEGgnuBasmJdgkf5E38zr7cpBsm3IM%3D&amp;reserved=0> in a request to (http, example.com, 80) would be encoded as an Uri-Path option containing

4.04 Not Found error.

[JLS] There also is an interesting question about case-folding for the paths as well.  Doing this for ASCII is not too hard, doing so once you go into UNICODE it becomes a huge problem.  I just looked and my implementation ignores case as well as normalization.

This conversion is purely syntactic, so an Uri-Path option with h'4d6f6e7472_65cc81_616c' in the request URI would become < <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%2565%25CC%2581al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=A88h0r8EuhVyju889ip7swWUFgn2laYUjbXnSDVmO2k%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fexample.com%2Fcity%2FMontr%2565%25CC%2581al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=A88h0r8EuhVyju889ip7swWUFgn2laYUjbXnSDVmO2k%3D&amp;reserved=0>.

header field.

RDF recommends [1] that these IRIs avoid non-normalized forms such as uppercase characters in scheme names, explicitly stated HTTP default port, percent-encoding of characters where it is not required by IRI syntax, and IRIs that are not in NFC. So for example the concept identified by the IRI < <https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> is not the same as the concept identified by the IRI < <https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fexample.com%2Fcity%2FMontr%C3%A9al&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636848776049783845&amp;sdata=0adMfMUJNNYHiHk9jZ%2Fh11NVM9Rh5eCYlIwKk3CQDUI%3D&amp;reserved=0> if one of them isn't in NFC.

See RFC 6943.

Dave

Klaus

[1]  <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Frdf11-concepts%2F%23note-iris&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049783845&amp;sdata=BqgcQRFWj1DpNuOU%2FNCCMvHRMwTDFmzJKtMujHcMsag%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Frdf11-concepts%2F%23note-iris&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049783845&amp;sdata=BqgcQRFWj1DpNuOU%2FNCCMvHRMwTDFmzJKtMujHcMsag%3D&amp;reserved=0

[2]  <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Fturtle%2F&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049793804&amp;sdata=WS%2BWnFEFffWRohqs7EpOe4FZyiFrgKre89gY630mT98%3D&amp;reserved=0> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2Fturtle%2F&amp;data=02%7C01%7Cdthaler%40microsoft.com%7Cf33e3c24c3e34af6d91f08d68a966e89%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636848776049793804&amp;sdata=WS%2BWnFEFffWRohqs7EpOe4FZyiFrgKre89gY630mT98%3D&amp;reserved=0