Subject: Re: [ietf-smtp] DKIM andsmtp extension model (was: Re: [Proposal] confusing parts of the mail system, was 250-MARKDOWN)

From: Ned Freed <[email-masked]>
Date: Fri, 01 Feb 2019 00:01:11 -0800 (PST)

> In article <F5A0D1E3B5FC14EEFBB6C7E7@PSB> you write:
> >

> skipping over all the parts I agree with ...

> >...  When the EAI WG was
> >doing that work, it was clear that interactions with DKIM were
> >unspecified and fairly clear that DKIM was not going to work
> >with such addresses and message headers without some careful
> >thinking and specification adjustments. ...

> DKIM turns out to be OK on SMTPUTF8 mail.  So long as you do the DKIM
> signing after all of the cleanup that submission does, the signatures
> are fine.  If something later recodes or reorders body parts, the
> signatures will break, but that's hardly unique to EAI.  DKIM has some
> optional canonicalization rules for computing the hashes which turn
> out to be useless.

I strongly disagree. It depends on whether you're talking about headers or
bodies. Relaxeded header canonicalization is incredibly useful. Relaxed body
canonicalization, not so much.

> Hashes using the complicated fuzzy rules don't
> work any better than the simple rule that says hash everything
> verbatim except drop blank lines at the end of the message.

For messages bodies, yes. The situation with headers is different - the
relaxed case is easy to implement and gets mesages past certain
transformations that otherwise would break the signature.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 1 Feb 2019 12:26:08 +0100

On Thu 31/Jan/2019 04:46:02 +0100 John C Klensin wrote:
> 
> I think those kinds of questions are going to require a WG
> rather than, e.g., Pete and I getting together and putting
> drafts together with while everyone will be comfortable.  YMMD
> but be careful what you wish for or wish the ADs to wish for.

+1, it would be about time!

Best
Ale
--

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 1 Feb 2019 12:31:28 +0100

On Friday 1 February 2019 09:01:11 CET, Ned Freed wrote:
> The situation with headers is different - the

I'll second that.

The relaxed header spec us a nicely done compromise that preserves a large 
part of the value of the signature and is robust against a large part of 
the shit that happens in transmission. Not perfect on either count, but 
enough to be a net positive. The people who wrote the spec have good reason 
to be proud.

Arnt

---

From: "John Levine" <[email-masked]>
Date: 1 Feb 2019 12:32:01 -0500

>transformations that otherwise would break the signature.

I haven't noticed that but you see a lot more mail than I do.

Apropos this point, I stuck a new paragraph into the DKIM/SPF/DMARC
EAI update draft pointing out that the relaxed case folding applies
only to header field names which per RFC 5322 are ASCII only, so the
folding remains ordinary ASCII case folding. 

 The rest of the relaxing involves squashing white space and in mail
headers, EAI white space is the same as ASCII white space.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Fri, 01 Feb 2019 14:33:22 -0800 (PST)

> enough to be a net positive. The people who wrote the spec have good reason

Completely agree, and that's something we don't say often enough.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Fri, 08 Feb 2019 13:15:12 -0500

> +1, it would be about time!

Write a proposal.  Possibly propose a WG-forming BoF for either
Prague (cutoff date is today) or Montreal (FWIW, I won't be in
Prague but will probably be in Montreal - no idea about Pete's
schedule).  If possible, make a list of things you think need to
be changed or at least examined for changes.  

As you are doing that, note that after IETF LC on what became
5321 closed there were demands from an AD or two to make
editorial and organizational changes. Some of those changes
would have required completely reorganizing the document.
Partially because it was recognized that such a reorganization
would require starting the approval process over and probably
discussion in a WG, compromises were reached that involved a few
small changes and probably satisfied no one.  IMO, we should not
try to move forward unless those issues can be settled (or
preventing from arising again) very early in the process.  So
something along those lines should be reflected in your
proposals as well.  I don't remember there being similar
problems with 5322, but you should check.

FWIW, I personally believe that it would be a good idea to
completely reorganize and rewrite 5321 because the process by
which we folded 821, some topics covered by 1123, and the
extensions work together created a document that is hard to
follow as well as being too long.   While I think we could patch
5321 (as we patched 2821 to get it) without needing a very
intense review process, I assume that reviewing a reorganization
and rewrite and making sure we didn't inadvertently introduce
errors, requirements, or other substantive changes for which
there was no consensus would almost certainly require an active
and committed WG.

Also, either

* Think about doing formal implementation reports on 5321 and
5892 and, if any of the changes you think need to be made are
new and substantive rather than, e.g., clarifications or unused
features that should be deleted, make sure I-Ds are written,
posted, processed into Proposed Standards (and reflected in
those implementation reports) before you suggest we open 5321/
5322.

   --or--

* Figure out how to convince the community that it would be
worthwhile to produce Proposed Standard 5321bis / 5322bis with
the clear expectation that we would want to be working on
Internet Standard 5321ter / 5322ter six months or a year after
we got finished with the "bis" versions.  As you can probably
guess, I'd be strongly opposed to that for multiple reasons, but
I'm just one voice.

As I said, be careful what you wish for.

  best,
    john

---

From: "Bron Gondwana" <[email-masked]>
Date: Tue, 12 Feb 2019 04:27:45 -0500

> As I said, be careful what you wish for.

I will be in Prague, and hopefully most of Montreal (I will have to miss the Monday, annoyingly). I'm very interested in this work as well and happy to help find the right place to work on it.

Cheers,

Bron.

--
 Bron Gondwana, CEO, FastMail Pty Ltd
 [email-masked]

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 12 Feb 2019 12:13:16 +0100

On Fri 08/Feb/2019 19:15:12 +0100 John C Klensin wrote:
> --On Friday, February 1, 2019 12:26 +0100 Alessandro Vesely wrote:
>>> I think those kinds of questions are going to require a WG rather than,
>>> e.g., Pete and I getting together and putting drafts together with while
>>> everyone will be comfortable.  YMMD but be careful what you wish for or
>>> wish the ADs to wish for.>>
> Write a proposal.

:-)

> If possible, make a list of things you think need to be changed or at least
> examined for changes.

We have this funny situation where SMTP is a /de facto/ standard.  Just
bringing it to Internet Standard would be a Good Thing.  There are not so many
changes to be done.  You say you have notes.  Otherwise the YAM database is
still good.

> As you are doing that, note that after IETF LC on what became 5321 closed
> there were demands from an AD or two to make editorial and organizational
> changes. Some of those changes would have required completely reorganizing
> the document.

Reorganizing the document would make life difficult for implementors, which is
not worth doing for purely editorial or aesthetic reasons.

> Partially because it was recognized that such a reorganization would require
> starting the approval process over and probably discussion in a WG,
> compromises were reached that involved a few small changes and probably
> satisfied no one.  IMO, we should not try to move forward unless those
> issues can be settled (or preventing from arising again) very early in the
> process.

The new WG's purpose would be to consolidate the standard, not to explore
experimental rules.  Some parts of the document are observably obsolete, for
example Section 3.9.  In such cases, it would be preferable to state that some
features don't belong to the on-the-wire protocol, and content ourselves with
stating who is responsible for ruling what.

> FWIW, I personally believe that it would be a good idea to completely
> reorganize and rewrite 5321 because the process by which we folded 821, some
> topics covered by 1123, and the extensions work together created a document
> that is hard to follow as well as being too long.

It's here that I find it difficult to follow your reasoning.  Yeah, it would be
a fascinating idea, but would it also be a practical task to pursue?

> While I think we could patch 5321 (as we patched 2821 to get it) without
> needing a very intense review process, I assume that reviewing a
> reorganization and rewrite and making sure we didn't inadvertently
> introduce errors, requirements, or other substantive changes for which there
> was no consensus would almost certainly require an active and committed WG.

I agree an active and committed WG is necessary, but I doubt it'd suffice.

> Also, either
> 
> * Think about doing formal implementation reports on 5321 and 5892 and, if
> any of the changes you think need to be made are new and substantive rather
> than, e.g., clarifications or unused features that should be deleted, make
> sure I-Ds are written, posted, processed into Proposed Standards (and
> reflected in those implementation reports) before you suggest we open 5321/ 
> 5322.
> 
>    --or--
> 
> * Figure out how to convince the community that it would be worthwhile to
> produce Proposed Standard 5321bis / 5322bis with the clear expectation that
> we would want to be working on Internet Standard 5321ter / 5322ter six
> months or a year after we got finished with the "bis" versions.

I'd opt for the second one of those that you said.  But...

> As you can probably guess, I'd be strongly opposed to that for multiple
> reasons, but I'm just one voice.

Eh?  Well, certainly sooner or later there will be a 5321ter if SMTP won't die
before.  Perhaps six months or a year is a bit too early, but ten years after
would seem quite enough, wouldn't it?

Best
Ale
--

---

From: Hector Santos <[email-masked]>
Date: Tue, 12 Feb 2019 10:24:41 -0500

> a fascinating idea, but would it also be a practical task to pursue?

I think so.

RFC1123 was my "holy bible" for internet hosting. It was an extremely 
useful consolidated guide at the time for the early internet 
integrated hosting servers developers of FTP, TELNET and SMTP.

I think we need something like it again, at least for SMTP.  Take a 
look at RFC1123 section 5.4. SMTP REQUIREMENTS SUMMARY and similar 
tables for the other protocols.  These were so unbelievable helpful as 
a quick reference guide.  We need another modern summary like this 
again.  I think most seasoned developers just want to know the 
"protocol rules" and not waste time with historical design reasoning 
notes.  Implementators trust the cogs, who help write/mold the specs, 
are doing the right (IETF) engineering thing so just give us the MAY, 
SHOULD and MUST protocol product design rules.

RFC5321+ could be a reduced technical reference guide with straight 
forward state machine protocol and prototyping notes for both pure 
SMTP and ESMTP with add-ons such as ESMTP AUTH, STARTTLS, 8BITMIME, 
etc  Summary (Extended) Requirements/BCP Tables would be very helpful.

I wouldn't want to add anything new unless we wanted to add a section 
on "Mail Filtering and Load Controls" concepts which has probably been 
the biggest things since 2003 done with SMTP as developers began to 
add these features to their implementations in order to help customers 
were were getting hit hard with the unprotected return path spoofs and 
unsolicited spam.  Add-ons like GreyListing, SPF, DKIM and DKIM-based 
Domain Policy methods would be part of such a section discussion of 
the filters available. The biggest thing here would be the shift of 
real-time processing to the DATA state which had added significant 
session time overhead. But improved hardware has trumped those early 
performance concerns, so in general, we can do more with E/SMTP today 
than ever before.

I also agree that RFC5322+ should maybe be done at the same time, but 
not necessarily as a show stopper if IETF WG work for only RFC5321+ 
can be allocated.

-- 
HLS