Subject: [yam] Updating multipart/report

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 15 Jul 2011 11:13:18 -0700

This seems to be a good time to introduce this question since YAM is talking about updating its charter.

The MARF working group has a requirement to be able to produce messages that contain abuse reports.  It currently uses multipart/report to do this.  However, multipart/report has a limitation that it has to be the outermost media type in a message.  This prevents MARF from being compatible with another system that can contain multiple reports per message.

I introduced this topic some months ago on apps-discuss, and a draft that is almost verbatim the same as RFC3462 minus the above limitation.  I remember Keith dissenting and Ned supporting, on the grounds that the limitation hasn't served any useful purpose. The topic died off after that, but I'd like to reintroduce it.

We could try to do this from within MARF but it seems to me that modifying a general-purpose email RFC in a working group with a specific application is probably a little sketchy.  It really belongs either in here or in APPSAWG, subject of course to whatever YAM's charter allows.

So, can/should YAM take it as a work item?

-MSK

---

From: Ned Freed <[email-masked]>
Date: Fri, 15 Jul 2011 14:37:15 -0700 (PDT)

> The MARF working group has a requirement to be able to produce messages that
> contain abuse reports.  It currently uses multipart/report to do this. 
> However, multipart/report has a limitation that it has to be the outermost
> media type in a message.

> This prevents MARF from being compatible with another
> system that can contain multiple reports per message.

> I introduced this topic some months ago on apps-discuss, and a draft that is
> almost verbatim the same as RFC3462 minus the above limitation.  I remember
> Keith dissenting and Ned supporting, on the grounds that the limitation hasn't
> served any useful purpose. The topic died off after that, but I'd like to
> reintroduce it.

With all due respect to Keith, this sort of blanket restriction is nothing
short of absurd. As you point out, it prevents re-use of multipart/report in
environments where reports have no need to be top-level and in fact need to be
able to be able to be gathered together. It doesn't even make sense for DSNs
and MDNs - suppose you want to forward one of these with a cover note to a
support team for analysis?

In fact we're lucky that essentially every client out there has summarily
ignored this requirement, because the absolute last thing we want it for
clients to screw around with the content of reports in any way, even minor
stuff like changing the subtype of the multipart. It's already hard enough to
get users to provide information needed to diagnose their problems; we really
don't need any more cooks stirring this particular pot.

To the extent there should be any sort of restriction on multipart/report, it
should be on initial DSN and MDN generation. In those cases it makes perfect
sense to say they have to be at the top-level and relays shouldn't mess with
that. But once the DSN or MDN is delivered, it's just another hunk of data. And
as for other uses of multipart/report, well, we imposed a lot of
well-intentioned restrictions like this in the MIME effort, and pretty much
without exception they've all proved to be bad ideas in the medium to long
term.

> We could try to do this from within MARF but it seems to me that modifying a
> general-purpose email RFC in a working group with a specific application is
> probably a little sketchy.  It really belongs either in here or in APPSAWG,
> subject of course to whatever YAM's charter allows.

> So, can/should YAM take it as a work item?

While I agree that this should be dealt with, I don't have an opinion
regarding venue, mostly because I confess to not really understanding
the purpose of either YAM or APPSAWG (the latter I only became aware of
recently) at this point. I do note that EAI is removing a much more
significant restrition (another one that was well-intentioned but
ultimately wrongminded) on encodings of subtypes of message, and this
is being done directly in the EAI WG.

				Ned

---

From: S Moonesamy <[email-masked]>
Date: Fri, 15 Jul 2011 15:50:17 -0700

Hi Ned,
>is being done directly in the EAI WG.

Prior to the chartering of APPSAWG, there was an informal discussion 
about the topic last year.  I have some recollection of the comments 
about the idea of having an area-wide working group.

A new charter [1] has been proposed for YAM.  The matter is on the 
agenda for the YAM WG session at IETF 81.  I hope that the 
participants will make judicious use of the face time.  The purpose 
of the YAM WG is not to pick a fight with the EAI WG.  It would be a 
bad idea anyway for obvious reasons. :-)

I forgot to mention (off-list conversation) to Murray that RFC 3462 
which is currently a Draft Standard is listed as a work item for this 
WG.  If an update which does not affect advancement is needed by the 
MARF working group, I personally am open to discussing about it in 
this working group.  I sent an email to Gregory Vaudreuil, the author 
of RFC 3462, about the proposed charter.  I have not received any reply yet.

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/yam/current/msg00642.html

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Jul 2011 19:35:27 -0400

--On Friday, July 15, 2011 14:37 -0700 Ned Freed
<[email-masked]> wrote:

>> This seems to be a good time to introduce this question since
>> YAM is talking about updating its charter.
> 
>> The MARF working group has a requirement to be able to
>> produce messages that contain abuse reports.  It currently
>> uses multipart/report to do this.  However, multipart/report
>> has a limitation that it has to be the outermost media type
>> in a message.
>...
> With all due respect to Keith, this sort of blanket
> restriction is nothing short of absurd. As you point out, it
> prevents re-use of multipart/report in environments where
> reports have no need to be top-level and in fact need to be
> able to be able to be gathered together. It doesn't even make
> sense for DSNs and MDNs - suppose you want to forward one of
> these with a cover note to a support team for analysis?
>...
> To the extent there should be any sort of restriction on
> multipart/report, it should be on initial DSN and MDN
> generation. In those cases it makes perfect sense to say they
> have to be at the top-level and relays shouldn't mess with
> that. But once the DSN or MDN is delivered, it's just another
> hunk of data.

You are stating a restriction on DSNs and MSNs, not
multipart/report.  That is probably an important distinction;
see below.

>  And as for other uses of multipart/report, well,
> we imposed a lot of well-intentioned restrictions like this in
> the MIME effort, and pretty much without exception they've all
> proved to be bad ideas in the medium to long term.
>...
> While I agree that this should be dealt with, I don't have an
> opinion regarding venue, mostly because I confess to not
> really understanding the purpose of either YAM or APPSAWG (the
> latter I only became aware of recently) at this point. I do
> note that EAI is removing a much more significant restrition
> (another one that was well-intentioned but ultimately
> wrongminded) on encodings of subtypes of message, and this is
> being done directly in the EAI WG.

I think that is the key, so let's take it a step further.  We
imposed a number of restrictions when the MIME spec was
developed.  As you said, they were well-intentioned.  Some were
to avoid cases or processing overhead that people were concerned
about, others to avoid various types of complexity.   EAI's need
to establish message/global and make it work by doing away with
the nested encoding restriction notwithstanding, it would really
be good to not do these piecemeal.  

Suggestion: someone with the motivation to make more of these go
away and the time to invest some effort should go through the
MIME specs and try to identify all of those restrictions.  Make
a list and make recommendations for each one as to whether the
restriction still seems to be justified based on what we've
learned in nearly 20 years.   With that list in hand, maybe we
can have a focused review discussion and get out a document that
updates the base MIME specification (and anything else
necessary) to sweep away whichever of these don't make sense and
to reaffirm any of those that are left.  It would be a serious
bit of housecleaning, but I think it would be worth the effort.

I imagine Ned already has a list, but a fresh set of eyes
producing one and then comparing it it his would give many of us
confidence that we are on the right track.

Murray, since you brought this up and have generated one draft
already, it seems to me that you are a half-step from
volunteering.  But maybe someone else would like to either do
the job or help.

best,
    john

---

From: Dave CROCKER <[email-masked]>
Date: Sat, 16 Jul 2011 17:15:38 -0700

> is being done directly in the EAI WG.

Well, it's really only doing it in the EAI context.

Anyhow, Murray's work item fits into YAM only by starting an entirely new and 
different working group.

It fits into APPAWG only if it doesn't really need a full-blown working group.

My guess is that it does not need a full-blown working group.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 16 Jul 2011 20:22:07 -0700

> -----Original Message-----
> From: John C Klensin [mailto:[email-masked]]
> Sent: Saturday, July 16, 2011 4:35 PM
> To: Ned Freed; Murray S. Kucherawy
> Cc: [email-masked]
> Subject: Re: [yam] Updating multipart/report
> 

I'd be willing to edit, but I'm sure my list is a fraction of what yours or Ned's might be.

Maybe we could talk about this during YAM or at least in the hallway track in Quebec City.

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Jul 2011 16:07:45 -0400

> hallway track in Quebec City.

I'm always available in hallways although I don't think Ned will
be.  I'd be happy to see it on the YAM agenda if there is time
-- not because I think it is part of YAM's mission, but because
a lot of the relevant people will probably be there.   Would the
YAM Chairs consider targeting having the agenda finished ten
minutes early so that those who are interested can take
advantage of being in the room for a discussion on this topic.

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Jul 2011 02:45:54 -0400

--On Friday, July 15, 2011 15:50 -0700 S Moonesamy
<[email-masked]> wrote:

>...
> I forgot to mention (off-list conversation) to Murray that RFC
> 3462 which is currently a Draft Standard is listed as a work
> item for this WG.  If an update which does not affect
> advancement is needed by the MARF working group, I personally
> am open to discussing about it in this working group.  I sent
> an email to Gregory Vaudreuil, the author of RFC 3462, about
> the proposed charter.  I have not received any reply yet.

Since it is conceivable that something depends on, and assumes,
the existing restriction, I don't see any way to remove it that
would not require a trip back through Proposed Standard.

   john

---

From: S Moonesamy <[email-masked]>
Date: Mon, 18 Jul 2011 03:27:56 -0700

Hi John,
>advantage of being in the room for a discussion on this topic.

Yes, that can be done.

Regards,
S. Moonesamy

---

From: Ned Freed <[email-masked]>
Date: Tue, 19 Jul 2011 08:06:35 -0700 (PDT)

> be.

No, I won't be there in person but I should be participating remotely.

> I'd be happy to see it on the YAM agenda if there is time

Seems reasonable to me.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Tue, 19 Jul 2011 08:07:58 -0700 (PDT)

> would not require a trip back through Proposed Standard.

This is one of the reasons why I prefer to see such restrictions lifted
in separate documents. Once the separate document (which can either be
one limited to removing the restriction or something more general) advances
the restriction can be removed from the original document without a reset.

More generally, this is another case of the stuff that John has been talking
about on the IETF list: Our processes have not kept up with the complexity of
our present documents. Back when documents were, on average, much simpler, it
may have made sense to reset an entire document or group of documents to
proposed simply to make a tweak like this, but these days it's causing more
problems than it solves.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Tue, 19 Jul 2011 08:20:10 -0700 (PDT)

> multipart/report.  That is probably an important distinction;

It's actually a little narrower than that - I'm saying that an "active"
DSN or MDN, that is, one sent to actually report on the status or
disposition of a mesage, should be required to be at top level.

But there should be no such restriction on, say, forwarding a DSN or MDN  I
have received to someone else. Indeed, I would actually like to require such
things NOT be sent at top level.

> be good to not do these piecemeal.

One problem is going to be that many if not most of the issues here lie in the
media type space, and that's not a MIME-only party any more.

In facy we should probably look over the media types registration draft
revision that's out now and see if there are any restrictions there that need
to go.

> confidence that we are on the right track.

Agreed.

				Ned

---

From: S Moonesamy <[email-masked]>
Date: Tue, 19 Jul 2011 09:54:49 -0700

>the restriction can be removed from the original document without a reset.

One of the questions when advancing documents along the Standards 
Track is whether it makes sense for such a move when there are 
changes that may affect the document status.  In this case, the 
alternatives are:

  (a) Have a Full Standard and a Proposed Standard document that 
lifts the restriction

  (b) Stick to the charter and work on the Full Standard document only.

  (c) Work on the Proposed Standard document only given that there is general
      agreement that the restriction should be lifted.

>proposed simply to make a tweak like this, but these days it's causing more

There can be some confusion in terms of which specification to 
implement when alternative (a) is chosen.  My take is that "the IETF 
is aiming at long-term stability of the specification and you can 
expect that the IETF won't change an implementation requirement 
overnight".  It's up to the implementer to determine whether to go 
with the tweak or not.  For all we know, the tweak may end up causing 
problems or it may require fine tuning.

With alternatives (a) and (c), the work would have to be done outside 
the working group.  One of the paragraphs from the proposed WG 
charter mentions that:

   "However the WG might reach consensus that certain changes have to be
    done in order to remove restrictions which were proven to be problematic
    in the field, or which restrict evolution of the protocols."

The working group would not be allowed to work on alternative (c) if 
its output is restricted to Full Standard documents 
only.  Alternative (a) is a viable approach if the working group is 
to consider the complexity of the documents identified as work items.

Regards,
S. Moonesamy

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 19 Jul 2011 09:58:15 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Ned Freed
> Sent: Tuesday, July 19, 2011 8:08 AM
> To: John C Klensin
> Cc: Ned Freed; S Moonesamy; [email-masked]

I'm concerned that this approach essentially leaves two versions of multipart/report around, because it's easy for someone to find the current RFC and not know about the other standards track document that updates it because it's not so noted on that RFC.  My preference would be to obsolete the old one with the new one.  I know that we update the RFC index with "updated by" information but people often don't know, or forget, to look there.

But if people think this approach will really work, then I'm happy to shrink mine down to a paragraph or two that removes the restriction and briefly explains why.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 19 Jul 2011 10:03:46 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of S Moonesamy
> Sent: Tuesday, July 19, 2011 9:55 AM
> To: Ned Freed; John C Klensin
> Cc: [email-masked]

I'd be fine with this.  We've been doing it that way with the main two mail documents since RFC282{1,2}.

>     in the field, or which restrict evolution of the protocols."

I'm in favour of a charter update that enables this sort of work.

---

From: Ned Freed <[email-masked]>
Date: Tue, 19 Jul 2011 14:19:02 -0700 (PDT)

> I'm concerned that this approach essentially leaves two versions of
> multipart/report around, because it's easy for someone to find the current RFC
> and not know about the other standards track document that updates it because
> it's not so noted on that RFC.  My preference would be to obsolete the old one
> with the new one.  I know that we update the RFC index with "updated by"
> information but people often don't know, or forget, to look there.

Such concerns may have some validity in the general case, but in this
particular case it's seriously overblown. For one thing, software that actually
implements  a restriction requiring multpart/report to only be used at
top-level is seen by users as just plain broken - to the point that it's very
doubtful it's honored by anyone except initial DSN/MDN generators - which is
the one place where the restriction does make sense and should be retained.

And if people are going to miss an updates clause, what makes you think they're
going to find the Obsoletes: any easier to locate? I still get stuff from
people that refers to RFCs 1891-1894 as the current specifications for this
stuff. It's not like the handling of obsoleted RFCs is materially different
from updated RFCs.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 20 Jul 2011 14:36:32 +0200

On 19/Jul/11 17:20, Ned Freed wrote:
> disposition of a message, should be required to be at top level.

I agree that it is somewhat simpler to deal with a traditional DSNs,
and that list managers may choke if presented with a container of many
of them.  Ditto for applications that expect an abuse report.
Is this the only reason to retain the top-level requirement?

Anyway, it is difficult to define "active", because the definition of
DSN seems to be confusing:

RFC 3461:
 A DSN is transmitted as a MIME message with a top-level content-type
 of multipart/report (as defined in RFC 3464).

RFC 3464:
 A DSN is a MIME message with a top-level content-type of
 multipart/report (defined in RFC 3461).

By substitution, I obtain the circular definition:

 A DSN is transmitted as a DSN.

If the  3461 definition had specified "active DSN", it would have yielded

 An active DSN is transmitted as a DSN.

which may make more sense.  But we probably want to say

 An active DSN is transmitted as a top-level DSN.

The wording looks much better for MDN, which is different from the
MIME type used to transmit it:

RFC 3462:
 When used to send a report, the Multipart/Report content-type must
 be the top-level MIME content type for any report message.

RFC 3798:
 A message disposition notification is a MIME message with a top-level
 content-type of multipart/report (defined in [RFC-REPORT]).  When

> things NOT be sent at top level.

+1, so it is clear they are related to some other sender.