Subject: Re: [media-types] Media subtypes containing "+"

From: Manu Sporny <[email-masked]>
Date: Wed, 8 Jul 2020 13:15:37 -0400

Murray S. Kucherawy wrote:
> We had the applicant remove this text and instead suggest that they need to
> register "+ld+json" independently as a media type suffix, but that such a
> registration would likely be denied until such time as the community has
> had a chance to discuss its implications.

Hi Murray, Mark, and Julian,

Let me try to shed some light on how this choice is affecting the W3C
WGs working on the JSON-LD, Verifiable Credentials, and Decentralized
Identifiers specifications.

Way back in the day, when we were working on JSON-LD 1.0, we had to
register a media type and the suggestion given by the powers that be at
the time was to put the syntax we were using last. We were encoding
Linked Data in JSON format, so the natural choice became
"application/ld+json" as the media type and subtype.

Fast forward four years and we were working on Verifiable Credentials,
which have an abstract data model that could be expressed in various
formats (JSON-LD or CBOR-LD), but JSON-LD was the primary way, and we
didn't have a dire need to go do something like
"application/vc+ld+json", and we were concerned that attempting that
would just be rejected:

https://github.com/w3c/vc-data-model/issues/421

At that point, there was a suggestion that we do "jsonld" as the media
type, which would have had negative consequences on all of the software
that is deployed in production today and goes against the tribal
suggestion that "ld+json" is the right thing to do anyway. Doing
"jsonld" was just a hack to get around the question being asked in this
thread -- which is, how deep do the subtypes go?

https://github.com/w3c/json-ld-syntax/issues/287

We ended up not registering a MIME Type for Verifiable Credentials and
instead it was suggested, by the JSON-LD WG, that we register a "Profile
Type" for JSON-LD in this registry:

https://www.iana.org/assignments/profile-uris/profile-uris.xhtml

No one has done that to date, because you can't easily transmit a
Profile Type in a variety of the use cases we were looking at (such as
MIME Type associated with a file on disk based on file extension -- most
operating systems don't support more than just a string expressing MIME
Type).

Fast forward another four years and we're now engaged in the same
discussion in the W3C Decentralized Identifier Working Group (This isn't
going away, it's only going to get worse -- we have a handful of other
specs in the works with the same problem). In the DID WG specifically,
we're not going to be able to dodge the question like we did in the VCWG
and JSON-LD 1.1 WG because there are at least four MIME Types being
contemplated where JSON-LD Profiles won't work:

application/did+json
application/did+cbor
application/did+ld+json
application/did+ld+cbor

... and there is a growing need for a MIME Type for Verifiable
Credentials (as they're starting to be serialized to disk and there are
use cases that put these things in directories on desktops and laptop OSes):

application/vc+ld+json
application/vc+ld+cbor

So, we're getting cornered into having to pick something and it's going
to come down to:

application/did+jsonld
application/did+cborld

OR

application/did-ld+json
application/did-ld+cbor

OR (this seems to be the most tenable one)

application/did+ld+json
application/did+ld+cborld

The W3C JSON-LD 1.1 WG is at the end of it's life, and has not been able
to push this decision forward at IETF, so we can't easily register a new
MIME Type at this point. I'm sure there are some standards shenanigans
that could be played to do that between W3C Staff and IETF ADs, but
registering a new MIME Type for "jsonld" is broadly held as the wrong
thing to do. If we go down that path, IETF might consider rejecting all
new subtype submissions and JSON-LD will become a cautionary tale of
where subtypes went wrong... or perhaps there can be guidance given on
when *not* to use a subtype that would have prevented the JSON-LD folks
(specifically, me as the Editor at the time) from suggesting
"application/ld+json" in the first place.

The alternate path is that "application/did+ld+json" is allowed, with a
clear rationale on why that's a good thing. We have already tested a
variety of popular MIME Type parsers across multiple languages out there
in the wild and they seem to do the right thing.

We go into CR in the W3C DID WG in the next couple of months and we'd
like to register all of the relevant MIME Types for the Decentralized
Identifier specification by that time. We'd like to register at least
this one:

application/did+ld+json

What would break on the Internet if we were to do that? Are there other
options that we're not seeing?

-- manu

-- 
Manu Sporny - https://www.linkedin.com/in/manusporny/
Founder/CEO - Digital Bazaar, Inc.
blog: Veres One Decentralized Identifier Blockchain Launches
https://tinyurl.com/veres-one-launches

---

From: Nathan Rixham <[email-masked]>
Date: Wed, 8 Jul 2020 21:17:38 +0100

On Wed, Jul 8, 2020 at 6:15 PM Manu Sporny <[email-masked]>
wrote:

> "application/ld+json"

A very small point, which I realise is probably years too late, but in
usage "application/json+ld" is far more practical to use, for example in a
media selector, similarly "application/json+ld+did" follows the same
principal, subtyping reverse requires a great deal of both explantation and
code to handle, and gets ever more complex. Finding all "application/json*"
is much easier.

---

From: Manu Sporny <[email-masked]>
Date: Wed, 8 Jul 2020 16:44:02 -0400

On 7/8/20 4:17 PM, Nathan Rixham wrote:
> usage "application/json+ld" is far more practical to use, for example in
> a media selector, similarly "application/json+ld+did" follows the same
> principal, subtyping reverse requires a great deal of both explantation
> and code to handle, and gets ever more complex. Finding all
> "application/json*" is much easier.

Very true, which is why the first MIME Type attempted for JSON-LD was in
fact:

"application/json+ld"

... that is, until I was corrected by people that knew better. :)

Every now and then I go back and scan through a sea of specifications
and mailing lists in an attempt to understand the reasoning behind the
choice... find it (at least, that's what I keep telling myself), and
then promptly forget because the path is so twisted and meandering.

In any case, that ship sailed long ago.

-- manu

PS: Hi Nathan! It's been years -- hope you're well. :)

---

From: Martin J. Dürst <[email-masked]>
Date: Thu, 9 Jul 2020 08:28:19 +0900

On 09/07/2020 05:44, Manu Sporny wrote:
> .... that is, until I was corrected by people that knew better. :)
> 

Yes indeed. And the reason for the syntax is also quite clear and easy. 
It came up with XML. For XML, there isn't only application/, there are 
also image/, message/, and model/ (see 
https://www.iana.org/assignments/media-types/media-types.xhtml), and 
font/svg+xml would also be possible even though it isn't registered 
currently.

Similar considerations apply to other suffixes, in particular also those 
related to compression schemes. Major types and suffixes are not 
hierarchical. A suffix was easier to pick out than something in the middle.

Even for json, there's model/gltf+json, so if you want to deal with all 
json, you would have to match for "/json", not for "application/json", 
in a scheme that would but what are now suffixes at the start of the 
subtype.

Regards,   Martin.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 Jul 2020 12:55:28 -0700

Hi Manu, thanks for chiming in.

> Identifier specification by that time. We'd like to register at least

Mark is asking the right question, I feel: Do we know how well suffixes
have fared, in terms of having been a benefit?  I'm not sure how one would
go about determining this.  I would love to hear suggestions.

> options that we're not seeing?

I also don't know how to answer this, other than to claim that we tend to
err on the side of being cautious about making assumptions.  Having never
implemented a browser or something else that has to handle media types, I
don't know if I would look for a suffix by searching for a "+" from the
left or from the right, and thus I don't know what would break because the
results here are obviously quite different.

My inclination is to ask that we debate a draft that's an update to RFC
6838, which describes media type suffixes more precisely in a way that
explains how to use what looks like a nested suffix structure like this.
That is, make it clear that "application/did+ld+json" is to be interpreted
as a type of "application" and a subtype of "did" with a suffix of
"+ld+json", or a subtype of "did+ld" with a suffix of "+json", or a subtype
of "did" with two ordered suffixes.

Is someone from the W3C willing to put such a draft together?

-MSK

---

From: Martin J. Dürst <[email-masked]>
Date: Thu, 23 Jul 2020 10:47:14 +0900

> go about determining this.  I would love to hear suggestions.

I'm also not sure how we would do this. But one thing that occurs to me 
is that it may differ quite a bit depending on the actual suffixes. In 
particular, the benefits for +ld+json should be rather high, because 
JSON LD can be interpreted as RDF, and RDF data from different sources 
and with different vocabularies can be integrated rather easily (at 
least so the story goes). So I'm not surprised to see such a proposal 
come from such a WG.

Another point is that these suffixes are not only for machines. They may 
help humans, too. They may help people understand how a document is 
structured. And they may also help direct people who design new types to 
use existing types and build upon them, rather than invent their own 
clundges.

> What would break on the Internet if we were to do that?

It's easily possible that there is some software that is looking for the 
first or the last '+' in a subtype, with the expectation that it's the 
only one, or that checks that there is at most one '+'.

> Are there other

The 'correct' (according to a comment in the ABNF in RFC 6838, see 
https://tools.ietf.org/html/rfc6838#section-4.2) is that one should look 
for the last '+'.

      restricted-name-chars =/ "+" ; Characters after last plus always
                                   ; specify a structured syntax suffix

> explains how to use what looks like a nested suffix structure like this.

This seems to make sense.

> of "did" with two ordered suffixes.

I'd propose it be interpreted as two suffixes, along the following 
hierarchy:
application
application/json
application/ld+json
application/did+ld+json

The change of direction is unfortunate, but we already have it here and 
e.g. in URIs. And we have to be careful with the term 'subtype'. In RFC 
6838, that refers to everything after the slash, i.e. 'did+ld+json'.

Regards,   Martin.