Subject: Re: [media-types] I-D Action: draft-ietf-mediaman-suffixes-00.txt

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 16 Feb 2022 14:52:13 -0800

On Wed, Dec 1, 2021 at 2:53 PM <[email-masked]> wrote:

>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
> This draft is a work item of the Media Type Maintenance WG of the IETF.
>
>         Title           : Media Types with Multiple Suffixes
>         Authors         : Manu Sporny
>                           Amy Guy
>         Filename        : draft-ietf-mediaman-suffixes-00.txt
>         Pages           : 4
>         Date            : 2021-11-21
>
> Abstract:
>    This document updates RFC 6838 "Media Type Specifications and
>    Registration Procedures" to describe how to interpret subtypes with
>    multiple suffixes.
>

Some initial feedback:

The prose in the Abstract and elsewhere says this updates RFC 6838, but
it's not shown that way in the front page header.  Please add that.

I think Section 2 is fine, but I'm confused by Section 2.1.  Let's ignore
the multi-leveled thing for the moment.  I've always read "foo/bar+baz" as
"a foo/bar, encoded or expressed using baz".  Or put another way: Take this
payload and apply the inverse of "baz", and I should get a "foo/bar".  This
section seems to suggest the idea that if I don't know what a "foo/bar" is,
maybe I know what a "foo/baz" is, and I should try that.  Why is that the
right thing to do here?

For Appendix A, it's a little curious for an author to be thanking himself.

-MSK, just participating

---

From: Manu Sporny <[email-masked]>
Date: Thu, 17 Feb 2022 12:51:28 -0500

> it's not shown that way in the front page header.  Please add that.

Fixed in
https://github.com/rhiaro/draft-w3cdidwg-media-types-with-multiple-suffixes/commit/711f5391915265e4d7d81423c7df46ed55832a4c

> "a foo/bar, encoded or expressed using baz".  Or put another way: Take
> this payload and apply the inverse of "baz", and I should get a "foo/bar".
> This section seems to suggest the idea that if I don't know what a
> "foo/bar" is, maybe I know what a "foo/baz" is, and I should try that.  Why
> is that the right thing to do here?

Hrm, yes, I understand the issue you're raising. Section 2.1 attempts to
provide a mental model for people to think through how subtypes might be
applied. It sounds like the mental model doesn't gel with your one, which also
makes sense to me.

What we were going for was basically saying:

If you don't know what a foo/bar is, but you do know what a foo/baz is, then
you can process it as foo/baz (because there are known processing rules for
foo/baz).

Let's take Russ' recent example w/ application/svg+xml+gzip:

If a client gets a file off of the Internet, and it it sends it to some
processing back end, the back end can respond with two basic responses:

* I know how to process that media type, or
* I don't know how to process that media type.

So, let's say that the back end process doesn't need to understand all
application/* out there... it just needs to understand the basic encoding to
know what to do with it. So, let's say all your archival program wants to do
is store gzip files off of the Internet.

The way the current rules are written seem to indicate that an application
needs to understand the ENTIRE media type in order to process it. What the
current spec is trying to say is: That's not necessarily true, you can
understand part of the media type and process it.

So, instead of the archival system that stores gzip files going, "I don't know
what an 'application/svg+xml+gzip' bytestream is"... it can instead say "I
don't know what an 'application/svg+xml+gzip' bytestream is, and I don't know
what an 'application/xml+gzip' bytestream is, but I do know how to process an
'application/gzip' byte stream, so I'll do that.

Now, that all falls apart if someone can provide an example of where it is not
acceptable to do that sort of processing... or, clearly, if many people on
this mailing list revolt and suggest an alternative mental model for thinking
about all of this.

Does that make sense, Murray, or did I not do an adequate job of explaining
the mental model (or 'no' to both)?

> For Appendix A, it's a little curious for an author to be thanking

Snoop Dogg takes issue with your position, MSK:

https://www.youtube.com/watch?v=NfF3bThOW0Q

:P ... but seriously, it came from a time where Amy did the first draft of the
spec and I was trying to help convey what the spec should say. I've removed it
in the latest commit, because yeah, while it made sense a few years ago, it
doesn't make sense today:

https://github.com/rhiaro/draft-w3cdidwg-media-types-with-multiple-suffixes/commit/4d68c56d390192a88f5a21331af53512dc0d39f0

-- manu

-- 
Manu Sporny - https://www.linkedin.com/in/manusporny/
Founder/CEO - Digital Bazaar, Inc.
News: Digital Bazaar Announces New Case Studies (2021)
https://www.digitalbazaar.com/

---

From: Manu Sporny <[email-masked]>
Date: Thu, 17 Feb 2022 12:59:35 -0500

On 2/17/22 12:51 PM, Manu Sporny wrote:
> Let's take Russ' recent example w/ application/svg+xml+gzip

Murray, I screwed up explaining this... part of it being that there is no such
thing as application/svg+xml... it's image/svg+xml, which exposes the problem
that I think you're concerned about with the mental model explained in the spec.

Don't respond, let me take another shot at explaining, and then we can update
the spec text if we think it's a useful change.

-- manu

---

From: Manu Sporny <[email-masked]>
Date: Thu, 17 Feb 2022 14:44:28 -0500

On 2/17/22 12:51 PM, Manu Sporny wrote:
> Murray, I screwed up explaining this...

That'll teach me to try to fire off an email that requires precision right
before a scheduled call. Attempt #2, below...

Let's take Russ' recent example w/ image/svg+xml+gzip.

I think what we want to be able to say is that there are multiple valid ways
of processing content that is associated with that media type:

image/svg+xml+gzip - the consuming application understands application/gzip
and image/svg+xml

image/svg+xml - as long as some preliminary application can gunzip the data, a
secondary application that understands image/svg+xml can process it

application/gzip - some preliminary application understands gzip and that's
where the story ends... but perhaps that's fine because the application's job
is to just crawl the Internet and store all gzip'd data for archival purposes.

What Section 2.1 was attempting to do is to say "it's okay to process a media
type if you only understand part of the media type... media types are not
opaque"... and I believe this is where your concerns came in, Murray. The
section is clearly wrong when we apply its logic to something like
image/svg+xml+gzip, because +gzip can exist across a number of different
top-level types... it doesn't naturally map to "application/gzip".

So, the question then becomes, how do we reasonably map "+gzip" to
"application/gzip" via the Media Types[1] table?

For the "application/*" space, it's fairly easy (and Section 2.1 makes a lot
more sense). The application takes the subtype and just starts chopping off
items delimited by the "+" sign until you get a media type you understand:

application/did+ld+json - DID Document processors
application/ld+json - JSON-LD processors
application/json - JSON processors

However, as stated above, the mental model breaks down when you go outside of
"application/*". For example, how do we say that image/svg+xml+gzip can be
processed as "application/gzip" if an application understands that media type?

Perhaps the fix to this is to say that suffixes MUST be registered and
associated with a concrete media type? For example, we might need a new class
of entries the media types table:

Structured Suffixes Registry
----------------------------

+gzip -> application/gzip
+zip -> application/zip
+json -> application/json
+ld+json -> application/ld+json
+xml -> text/xml

Thoughts? What sort of horrors will doing that unleash?

-- manu

[1]https://www.iana.org/assignments/media-types/media-types.xhtml

---

From: Chris Lilley <[email-masked]>
Date: Thu, 17 Feb 2022 15:28:05 -0500

> secondary application that understands image/svg+xml can process it

This is the part that requires precision, with regard to right-to-left 
unpacking of suffixes.

There have been issues in the past (especially in non-HTTP contexts, 
such as file:// or data://) with an application receiving a gzipped SVG 
(.svgz) file and then simply saying "this is not a well formed XML 
document" because the preliminary application part was unspecified.

-- 
Chris Lilley
@svgeesus
Technical Director @ W3C
W3C Strategy Team, Core Web Design
W3C Architecture & Technology Team, Core Web & Media

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 18 Feb 2022 13:45:52 +0900

Hello Chris,

Glad to see you here.

> document" because the preliminary application part was unspecified.

Yes, I can imagine that. I checked out the SVG spec yesterday, because I 
was quite sure that it would contain detailed instructions about media 
types. In SVG 1.1, I found 
(https://www.w3.org/TR/SVG11/intro.html#MIMEType):

The MIME type for SVG is "image/svg+xml" (see XML Media Types 
[RFC3023]). The registration of this MIME type is in progress at the W3C.

It is recommended that SVG files have the extension ".svg" (all 
lowercase) on all platforms. It is recommended that gzip-compressed 
[RFC1952] SVG files have the extension ".svgz" (all lowercase) on all 
platforms.

In SVG 2.0 (Editor's Draft), I found 
(https://www.w3.org/TR/SVG2/conform.html#ConformingSVGServers):

Also, if the SVG file is compressed with gzip or deflate, conforming SVG 
Servers must indicate this with the appropriate header, according to 
what the protocol supports. Specifically, for content compressed by the 
server immediately prior to transfer, the server must use the 
"Transfer-Encoding: gzip" or "Transfer-Encoding: deflate" headers as 
appropriate, and for content stored in a compressed format on the server 
(e.g. with the file extension "svgz"), the server must use the 
"Content-Encoding: gzip" or "Content-Encoding: deflate" headers as 
appropriate.

This is correct in terms of HTTP,..., but it doesn't take into account 
Russ's case, which I understand is labeling a blob of gzipped SVG just 
with a MIME media type.

Regards,   Martin.

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 18 Feb 2022 14:55:11 +0900

Hello everybody,

> of processing content that is associated with that media type:

That should be okay, but it might be possible to just define a single 
way (remove suffixes from the right and each time you remove a suffix, 
apply the corresponding processing), and then say that merging steps or 
other kinds of optimizations that produce the same result are also okay.

So we would say that if you see image/svg+xml+gzip, then you first 
gunzip it, then you apply XML parsing, and then you apply SVG 
processing. Any combination of these steps is okay if the result is the 
same.

> is to just crawl the Internet and store all gzip'd data for archival purposes.

I would definitely want to avoid any kind of general operation on media 
type strings except the removal of suffixes from the right. What you do 
for something ending in +gzip may be the same or close to what you do 
for application/gzip, but there's no guarantee that this works in general.

> application/json - JSON processors

Again, please don't do that, because it doesn't work in the general 
case. So it doesn't work as a mental model.

> of entries the media types table:

I very much agree with registering things.

> Structured Suffixes Registry

I think the registry should map the +xyz suffix (as an example) to xyz 
processing, which may be xyz parsing or un-xyz-ing or something else.
The definition of this processing may be done by pointing to another 
media type, but that shouldn't be required.

So we would e.g. have a registry entry similar to the following.
+gzip -> Revert gzip compression (usually the command gunzip, similar
          to processing application/gzip)

Regards,   Martin.

---

From: Manu Sporny <[email-masked]>
Date: Fri, 18 Feb 2022 10:51:51 -0500

On 2/18/22 12:55 AM, Martin J. Dürst wrote:
> So we would say that if you see image/svg+xml+gzip, then you first gunzip
> it, then you apply XML parsing, and then you apply SVG processing. Any
> combination of these steps is okay if the result is the same.

Ok, great, that's good direction.

> for something ending in +gzip may be the same or close to what you do for 
> application/gzip, but there's no guarantee that this works in general.

I understand the general concern you have and I share it.

Do we have any concrete examples of some "+foo" being applied in different
ways across media types?

> Again, please don't do that, because it doesn't work in the general case.
> So it doesn't work as a mental model.

Ok, good, that approach to explaining the mental model is dead, then.

I'll rewrite the section using what you're suggesting, Martin, and see if it
resonates with people better.

> I very much agree with registering things.

Ok, so the concept of sort of structured suffix registry isn't off of the
table. I'll integrate that into the next iteration of the spec.

> processing, which may be xyz parsing or un-xyz-ing or something else.

Ok, I'll play around with some language to get that effect by going through
all the "+foo"s in the current registry and see if I can map all of them to
"processing" of some kind or another.

The only thing I'm worried about there is what you alluded to above, in that
"+foo" is not applied consistently across all media types. It's something
that's going to "quietly" fail unless we have a media type registrant step
forward and point out why the general rule doesn't work for their media type.

For example, "Oh, well, when we said +xml, what we really meant was XML with
the XLink extension)".

> The definition of this processing may be done by pointing to another media 
> type, but that shouldn't be required.

Ok, yes, that makes sense.

> So we would e.g. have a registry entry similar to the following. +gzip ->
> Revert gzip compression (usually the command gunzip, similar to processing
> application/gzip)

Right... I have enough to do a revision.

Anyone else on here think this is the wrong direction? It feels sane to me.
Russ? Murray?

-- manu

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 20 Feb 2022 00:11:47 +0100

Are there security concerns here?

It's long been a favorite game of virii and trojans to lie about what 
the inner type contains, and then try to leverage the fact that the 
inner type may dispatch on something different in order to induce 
unexpected processing.

For example, having a gzipped file "image/svg+xml+gzip", knowing that 
the XML parser will dispatch based on the XML declaration not the MIME 
type and instead putting a text/html file with embedded Javascript in there.

Or using the gzip "extract by name" facility and packing "virus.exe" 
inside "image/svg+xml+gzip", knowing that the next level of unpacking 
will dispatch on filename suffix?

These concerns exist (of course) with single-level suffix concatenation 
too, but multiple suffixes multiply the paths that can be taken.

I see that RFC 6838 section 4.6 does not note the particular danger of 
suffixed types; it may be a Good Thing to include a security section in 
this document that mentions the issue.

        Harald

---

From: Manu Sporny <[email-masked]>
Date: Sat, 19 Feb 2022 18:59:00 -0500

On 2/19/22 6:11 PM, Harald Alvestrand wrote:
> it may be a Good Thing to include a security section in this document that 
> mentions the issue.

Yes, excellent point, Harald. I'll add that as the first item in the security
considerations section.

Tracking that here:

https://github.com/rhiaro/draft-w3cdidwg-media-types-with-multiple-suffixes/issues/2

As somewhat of an aside, shouldn't we hand this github repository over to the
mediaman WG (or some other "IETF-owned" place)? Is there such a thing?

-- manu

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 20 Feb 2022 14:31:44 +0100

> mediaman WG (or some other "IETF-owned" place)? Is there such a thing?

It seems that it's up to me as a chair to request / create one. I 
haven't gotten around to it yet.

Good to know that your draft is in a github repo that we can move!