Subject: [ietf-smtp] smtp extension model

From: John Bucy <[email-masked]>
Date: Mon, 22 Aug 2016 13:53:10 -0700

On Fri, Aug 19, 2016 at 3:30 PM, John R Levine <[email-masked]> wrote:

>
>> Yes, although with SMTPUTF8 we made a fairly clear distinction that
> you're on the bus or you're off, and an entire mail site either supports it
> or not.
> <https://www.ietf.org/mailman/listinfo/ietf-smtp>
>

This is a little surprising to me. I understood 6531 to allow

c: MAIL FROM: <utf8 sender> SMTPUTF8
s: 250 OK
c: RCPT TO: <legacy_application>
s: 553 5.6.7 <legacy_application> does not accept utf8 messages

cheers
john

---

From: "John R Levine" <[email-masked]>
Date: 22 Aug 2016 17:08:15 -0400

> s: 553 5.6.7 <legacy_application> does not accept utf8 messages

Ah, you're right.  There was a certain amount of back and forth about that 
during the process.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Claus Assmann <[email-masked]>
Date: Mon, 22 Aug 2016 14:55:36 -0700

> >c: MAIL FROM: <utf8 sender> SMTPUTF8
                ^
> >c: RCPT TO: <legacy_application>
              ^

See the fine RFCs.

---

From: John C Klensin <[email-masked]>
Date: Mon, 22 Aug 2016 18:48:18 -0400

--On Monday, August 22, 2016 13:53 -0700 John Bucy
<[email-masked]> wrote:

>> you're on the bus or you're off, and an entire mail site
>> either supports it or not.

Ignoring the surplus (and invalid) spaces after the colons...

Yes, it does.  That is the entire mail site associated with
<utf8 sender> supporting it and the mail site associated with
<legacy_application> not doing so, so I think it is consistent
with John L.'s comment.

If I were the relay or host accepting that connection, I'd
generate the "does not accept..." reply the moment I saw
SMTPUTF8 and/or a non-ASCII address (i.e., in response to the
MAIL command in the example above) but, unless something snuck
into 6531 when I wasn't looking, that is a matter of taste
rather than an absolute requirement.  In particular, the last
paragraph of Section 3.6 explicitly allows the error response
after RCPT.  There would be no choice other than to do it then
or after DATA (see that paragraph) in either of the following
circumstances (there might be others):

(i) The server supports SMTPUTF8 but, in spite of
<legacy_application> being a local address, the server knows it
cannot accept messages with non-ASCII addresses or headers.

(ii) The server supports SMTPUTF8 but <legacy_application> is
not a local address or is a local address associated with an
alias that points to another host, so the server is acting as
relay or forwarding the message and knows that the remote server
doesn't support SMTPUTF8.

Some cases similar to the above --particularly when the server
doesn't know but finds out later-- can also involve situations
that require accept-and-bounce.

best,
    john

---

From: John Bucy <[email-masked]>
Date: Fri, 26 Aug 2016 12:29:27 -0700

> doesn't support SMTPUTF8.

It seems like the best time for this to happen would be during submission
where the mua is in a much better position to deal with the failure than
some other mta in the middle. How evil would it be for the msa to remember
which hosts it's talked to recently that accept a given extension to
propagate back to its clients?

cheers
john

---

From: "John R Levine" <[email-masked]>
Date: 26 Aug 2016 15:35:04 -0400

> propagate back to its clients?

Not at all, assuming a reasonable version of "recent."  I would guess 
something on the order of a day would be recent, but you surely have or 
could collect data that can tell you how often the extensions offered by 
other MTAs changes to figure out a good TTL.

I don't know how this scales, but I think some MSAs also do call-forwards, 
connect to the next server during the submission session to see if it 
handles the required extensions.

Regards,

---

From: John C Klensin <[email-masked]>
Date: Fri, 26 Aug 2016 16:00:33 -0400

--On Friday, August 26, 2016 12:29 -0700 John Bucy
<[email-masked]> wrote:

>...
>...

> It seems like the best time for this to happen would be during
> submission where the mua is in a much better position to deal
> with the failure than some other mta in the middle. How evil
> would it be for the msa to remember which hosts it's talked to
> recently that accept a given extension to propagate back to
> its clients?

Yes, well...
First, a great many options about what should be done where were
extensively considered during the two stages of SMTPUTF8/ EAI
development.  What follows is a summary, probably a poor one, of
some of those discussions: if you or others need more
background, there is probably no substitute for the archives
which perhaps you could motivate getting back online.

That said, the MUA and MSA are the only entities in the system
that can really do something intelligent, particularly
determining and substituting ASCII-only addresses for ones that
are partially non-ASCII.  Those substitutions are obvious on the
backward-pointing (aka sender address) side if a user maintains
both types of addresses but, even on the forward-pointing side,
if the MUA or MSA can't figure it out, anything further
downstream can, at best, get a "you lose" message back to the
user (no matter how much information that message contains).

The MSA does need to be careful about caching for all of the
usual reasons, e.g., how to detect when something has changed.
But logic suggests that, one a site converts to and starts
advertising SMTPUTF8, it is unlikely to discontinue the service
so remembering where it was successfully accepted seems sensible
even if the code needs to be good enough to deal with a failure
to accept it later.

Some care is in order because, while I don't know anyone who has
done it yet, it would be perfectly reasonable for a delivery MTA
to keep a list of which addresses are UTF-8 capable down to the
webmail client or IMAP or POP server being used by that
particular user and to decline, at RCPT time, to accept SMTPUTF8
if the user couldn't or wouldn't read the messages.  So, if one
were being extra-careful, one would need to remember that
information on a per-recipient basis, not just a per destination
host one.

Similarly, it would be sensible for an originating MUA or MSA to
maintain an address book that contained all-ASCII alternative
addresses for users with addresses that require SMTPUTF8
capability so that, if it could be determined that extended
messages would not go through, the addresses could be rewritten.
It seems fairly obvious that users would like that more than
getting a bounce or error message that they then had to deal
with (including the delays).  However, implementing it well is
not as straightforward as one might guess at first glance.

best,
     john