Subject: [ietf-smtp] Fwd: I-D Action: draft-melnikov-smtp-metadata-00.txt

From: Alexey Melnikov <[email-masked]>
Date: Sun, 22 Mar 2015 15:48:20 +0000

Constructive feedback would be very much appreciated!

-------- Original Message --------
Subject: 	I-D Action: draft-melnikov-smtp-metadata-00.txt
Date: 	Sat, 07 Mar 2015 12:25:40 -0800
From: 	[email-masked]
Reply-To: 	[email-masked]
To: 	<[email-masked]>

A New Internet-Draft is available from the on-line Internet-Drafts directories.

         Title           : Simple Mail Transfer Protocol extension for relaying Metadata
         Author          : Alexey Melnikov
	Filename        : draft-melnikov-smtp-metadata-00.txt
	Pages           : 11
	Date            : 2015-03-07

Abstract:
    This memo defines an extension to the SMTP (Simple Mail Transfer
    Protocol) service whereby message metadata (such as Trace header
    fields, IMAP flags, Keying material, etc) can be transferred in
    separate containers similar to BDAT (RFC 3030, SMTP CHUNKING)
    command.  This allows clean separation of transaction related state
    from the message itself.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-melnikov-smtp-metadata/

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-melnikov-smtp-metadata-00

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 22 Mar 2015 17:24:47 +0100

I'm not sure this is constructive, but it's meant to be helpful.

AFAICT RFC3030 has not been widely adopted in practice. Why, do you think? 
Do you think the reason, whatever it is, will hurt the new extension 
unnecessarily?

Arnt

---

From: "John Levine" <[email-masked]>
Date: 22 Mar 2015 20:16:31 -0000

>AFAICT RFC3030 has not been widely adopted in practice. Why, do you think? 

I think it's because networks have gotten fast and reliable enough
that 8BITMIME and various ways to encode binary data (qp and base64)
are good enough.

An alternative to what this draft does is to define some new MIME
multipart types and wrap everything in a new message.  That has the
disadvantage that it's ugly, but the advantage that MTAs including
relays don't need to know about it and it's arguably* faster.

So I think it would work fine if people implemented it, but unless
there's a very concrete use case and people ready write and deploy
those implementations, I wouldn't bother.

R's,
John

* - arguably due to more round trips for more commands, arguably
mitigated by pipelining

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 22 Mar 2015 20:31:24 +0000

On 22/03/2015 20:16, John Levine wrote:
  [snip]
> relays don't need to know about it and it's arguably* faster.
Indeed.
> those implementations, I wouldn't bother.
Fair comment, see use case examples in the document.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 22 Mar 2015 20:38:41 +0000

On 22/03/2015 16:24, Arnt Gulbrandsen wrote:
> AFAICT RFC3030 has not been widely adopted in practice. Why, do you 
> think? Do you think the reason, whatever it is, will hurt the new 
> extension unnecessarily?
I think support for RFC 3030 CHUNKING extension is much more common 
these days, but I need to double check major SMTP server implementations.

There is really no reason for this proposal to require support for 
CHUNKING, so it can be refactored to work with DATA (I would not change 
syntax of the new command though). In general, I find that octet-counted 
blocks are easier to implement that something like dot-stuffing.

---

From: "John R Levine" <[email-masked]>
Date: 22 Mar 2015 16:13:31 -0500

> Fair comment, see use case examples in the document.

I don't understand the benefit of putting the headers in a separate chunk. 
Setting suggested IMAP flags seems easy to do in a header if someone wants 
to do that.

Again, if people want to do this in SMTP I wouldn't be opposed to it, but 
I'd really want to see some indication that there's some likelihood that 
it'll be implemented.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: Stephan Bosch <[email-masked]>
Date: Sun, 22 Mar 2015 22:14:49 +0100

Hi,

> Constructive feedback would be very much appreciated!

I think this document can benefit from a more extensive introduction. I
mainly like to know why a `clean separation of transaction-related state
from the message itself' is necessary. What can go wrong if it isn't
passed separately? What are the main benefits? What are the applications
where this is relevant? I can take a few guesses for things like keying
material, but the rationale for passing trace information like that is
much less obvious. I see some hints throughout the text, but I think
making this clear in the induction is important.

Also, what is the application of being able to pass IMAP flags in SMTP?
Normally, I wouldn't want to give the sender the ability to assign flags
directly. And how would these flags interact with Sieve imap4flags?

Regards,

Stephan.

> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 22 Mar 2015 21:48:55 +0000

> Hi,
Hi Stephan,
> I think this document can benefit from a more extensive introduction. 
> I mainly like to know why a `clean separation of transaction-related 
> state from the message itself' is necessary. What can go wrong if it 
> isn't passed separately? What are the main benefits? What are the 
> applications where this is relevant? I can take a few guesses for 
> things like keying material, but the rationale for passing trace 
> information like that is much less obvious. I see some hints 
> throughout the text, but I think making this clear in the induction is 
> important.
Fair comment. There might be some benefits related to providing privacy, 
especially if this data is encrypted, but the main message itself is in 
clear text.
I admit I haven't thought about this a lot and this might be a silly 
idea, but I thought putting this in a draft is much better than arguing 
about this in abstract as lots of people do :-).
> Also, what is the application of being able to pass IMAP flags in SMTP?
Message transfer between IMAP servers over SMTP. For example if I want 
to send you a message and let you know that it was forwarded and/or 
replied to.
> Normally, I wouldn't want to give the sender the ability to assign 
> flags directly. And how would these flags interact with Sieve imap4flags?
The idea is that the flags passed through this extension would replace 
the default empty set used when a Sieve script starts.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 22 Mar 2015 21:52:42 +0000

> I don't understand the benefit of putting the headers in a separate chunk.
See my reply to Stephan. I admit, I haven't decided myself how useful 
this might be.
> Setting suggested IMAP flags seems easy to do in a header if someone 
> wants to do that.
It bothers me that we keep minting new SMTP extensions. There was an 
SMTP/LMTP extension proposal for passing IMAP flags.
Stuff like DSN parameters could have been done as transaction state 
block 0. I know it is too late to change DSN, but I am wondering if 
doing something more generic would be useful.

The third use case is key material exchange a la DarkMail, but I will 
see if there is any interest in that idea.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 22 Mar 2015 23:20:20 +0100

Alexey Melnikov writes:
> I think support for RFC 3030 CHUNKING extension is much more 
> common these days, but I need to double check major SMTP server 
> implementations.

Neither Exim nor Postfix. What else counts as major or common nowadays? 
Gmail perhaps? It does chunking, but not binarymine.

Arnt

---

From: Steve Atkins <[email-masked]>
Date: Sun, 22 Mar 2015 15:26:53 -0700

On Mar 22, 2015, at 3:20 PM, Arnt Gulbrandsen <[email-masked]> wrote:

> Alexey Melnikov writes:
>> I think support for RFC 3030 CHUNKING extension is much more common these days, but I need to double check major SMTP server implementations.
> 
> Neither Exim nor Postfix. What else counts as major or common nowadays? Gmail perhaps? It does chunking, but not binarymine.

Oracle Communications Messaging Server, Hotmail, Google.

( http://esmtp.wordtothewise.com - forgive the terrible UI ).

Cheers,
  Steve

---

From: Ned Freed <[email-masked]>
Date: Sun, 22 Mar 2015 21:54:49 -0700 (PDT)

> Gmail perhaps? It does chunking, but not binarymine.

FWIW, we support client/server chunking but our binarymime support is server
only. (The logic here is that it's main use is for mobile client submission.)

It support calls are any indication there are several widely used chunking
implementations out there. (With some bugs.) Binarymime, not so much.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Mar 2015 13:17:59 -0700

The problem with BINARYMIME is that unless everyone supports it, someone
may have to transform the message... and that will break DKIM signatures
and make life annoying.  Hence my colleagues question about a mime aware
dkim canonicalization format as a pre-req for binarymime.

Brandon

---

From: "John Levine" <[email-masked]>
Date: 23 Mar 2015 21:43:27 -0000

>dkim canonicalization format as a pre-req for binarymime.

We considered it when we were doing the strict/loose c14n but decided
against it as a swamp.  If a relay is going to change MIME structures,
it can recode parts and reorder them while keeping the "same" contents
which will make it quite a challenge to describe what one has to do to
put them into canonical form.

There is a c14n registry to which new values can be added, so if you
want to propose a MIME c14n, the first step is to write it up in a draft.

R's,
John

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 23 Mar 2015 23:02:05 +0100

> see if there is any interest in that idea.

a fourth use case might be, for MLM's to make a diff between:

a) original message and
b) the message the MLM will distribute

and to send this diff along with the original DKIM-Signature in a 
'container'. The receiver MTA could reconstruct the original message and 
verify the original DKIM-Signature (and e.g. apply DMARC).

/rolf

---

From: "John Levine" <[email-masked]>
Date: 23 Mar 2015 22:50:17 -0000

>verify the original DKIM-Signature (and e.g. apply DMARC).

I'd think it'd be easier just to wrap the original message as a
message/rfc822 part.  The problem with that is that MUAs don't display
them very well.  Hard to say how hard that is to fix vs. putting code
in an MTA to handle this metadata-ized thing vs a shim in the MTA or MDA
that recognizes a wrapped message and unwraps it before delivery.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Tue, 24 Mar 2015 06:42:18 -0700 (PDT)

> dkim canonicalization format as a pre-req for binarymime.

Which is why I said the target use with SUBMIT, not SMTP relay.

Of course another way to look at it is that this is Yet Another Way that
DKIM/DMARC is incompatiable with our existing standards and infrastructure.
Calling for Yet Another Fix.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 24 Mar 2015 14:54:42 +0100

> Calling for Yet Another Fix.

Perhaps. Although in this case I suggest that BINARYMIME isn't part of our 
existing standards and infrastructure, for lack of implementation and 
deployment.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Tue, 24 Mar 2015 07:38:03 -0700 (PDT)

> existing standards and infrastructure, for lack of implementation and

Arnt, I'm sorry, but that's bunk. And I must say I'm getting pretty tired of
this specious line of argument.

RFC 3030 specifies a standards track protocol. That alone is sufficient to
refute your suggestion that this isn't part of our existing standards suite.

As for it not being part of our infrastructure, what you actually mean is that
"I haven't seen it in the wild and some google searches I did and some web
sites I visited turned up no signs of usage".

The problem with that is you don't know everything that's out there. (And
neither do I.) (And neither does anyone else on this list.) What we do know is
that the Internet is a big place, with lots of nooks and crannies we can't see
into.

And in this instance you now know (because I told you) that you missed a use
case you had no insight into: Mobile device submission. Which happens to be a
space where support in a single client can have a rather  large impact.

So your suggestion that there's no deployment is wrong as well.

Now, as it happens the existing deployment is a type that doesn't overlap
existing DKIM usage much if at all. So we haven't had to deal with the
interoperability problem we've created.

And that's OK, I guess, as long as nobody tries to expand the scope of DKIM to
end-to-end scenarios. If you try to do that there's going to be a conflict, and
this time it's going to be DKIM that's in the position of not being deployed in
the relevant space.

And the combination of that along with the apparent desire to use BINARYMIME
outside of the submission space being expressed by some major players here may
be sufficient cause to want to define an encoding-insensitive MIME
canonicalization for use with DKIM.

But more generally, I categorically reject the notion that apparent lack of
deployment of a standard is ipso facto sufficient cause to completely ignore
that standard and to deploy things that fail to interoperate with it. This
strategy is a a sure-fire recipe for creating ongoing operational problems.

Of course this doesn't mean we are bound to respect every standard we ever
created, unchanged, forevermore. But we have ways - technical, expository, and
procedural - for dealing with such conflicts when they arise, and my own
suggestion is that we use them.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Wed, 25 Mar 2015 13:31:55 -0700

Yesterday, 250k IPs we talked to for SMTP advertised BINARYMIME.  The
largest advertiser we saw was Hotmail/Outlook.

Trying 65.55.37.104...
Connected to mx1.hotmail.com.
Escape character is '^]'.
220 COL004-MC3F10.hotmail.com Sending unsolicited commercial or bulk e-mail
to Microsoft's computer network is prohibited. Other restrictions are found
at http://privacy.microsoft.com/en-us/anti-spam.mspx. Wed, 25 Mar 2015
13:24:42 -0700
EHLO foo
250-COL004-MC3F10.hotmail.com (3.21.0.161) Hello [72.14.229.81]
250-SIZE 36909875
250-PIPELINING
250-8bitmime
250-BINARYMIME
250-CHUNKING
250-STARTTLS
250-AUTH LOGIN
250-AUTH=LOGIN
250 OK

Not reporting BINARYMIME ourselves, but we don't reject binary messages at
the moment and it mostly works through the whole system, but we haven't
gone as far as actually validating the whole pipeline on corner cases to be
happy enough to start advertising it.

Supporting it just for MSA may be a good partial solution...

Brandon