Subject: [ietf-smtp] Fwd: New Version Notification for draft-crocker-email-deliveredto-05.txt

From: Dave Crocker <[email-masked]>
Date: Wed, 4 Aug 2021 19:16:40 -0700

This has changes intended to cover the existing use of the field.

For covenience, consider:

https://www.ietf.org/rfcdiff?url2=draft-crocker-email-deliveredto-05

d/

-------- Forwarded Message --------
Subject: New Version Notification for draft-crocker-email-deliveredto-05.txt
Date: Wed, 04 Aug 2021 19:14:26 -0700
From: [email-masked]
To: Dave Crocker <[email-masked]>

A new version of I-D, draft-crocker-email-deliveredto-05.txt
has been successfully submitted by Dave Crocker and posted to the
IETF repository.

Name:		draft-crocker-email-deliveredto
Revision:	05
Title:		Delivered-To Email Header Field
Document date:	2021-08-04
Group:		Individual Submission
Pages:		8
URL: 
https://www.ietf.org/archive/id/draft-crocker-email-deliveredto-05.txt
Status: 
https://datatracker.ietf.org/doc/draft-crocker-email-deliveredto/
Htmlized: 
https://datatracker.ietf.org/doc/html/draft-crocker-email-deliveredto
Diff: 
https://www.ietf.org/rfcdiff?url2=draft-crocker-email-deliveredto-05

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: "John Levine" <[email-masked]>
Date: 4 Aug 2021 23:41:50 -0400

>https://www.ietf.org/rfcdiff?url2=draft-crocker-email-deliveredto-05

It still is missing the point. The thing in the Delivered-To line is a
token, which usually is not an address or a mailbox, even though it
might look like one. The draft refers to "include each address
that led to the final delivery" or "Delivering the same message more
than once to the same address" or "The header field contains information about the
individual mailbox" but the thing in the header isn't an address or a mailbox.

I still don't see any reason to claim this is an experiment. We have
20 years of experience with Delivered-To. It seenm unlikely that
anyone plans to change the way they do or do not add the header at
this point.

Once again, a survey of existing practice could be useful.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Thu, 5 Aug 2021 05:38:10 -0700

> Once again, a survey of existing practice could be useful.

Once again, tThe one missing the point is you, John.

You continue to offer firm conclusions, without engaging in the detailed 
substance.

Send actual data.

The revisions cover the data available so far, which do not reflect your 
'usually'.

Data.  Actually examples, showing existing practice.

d/

---

From: Dave Crocker <[email-masked]>
Date: Thu, 5 Aug 2021 11:33:30 -0700

> Once again, a survey of existing practice could be useful.

John,

You are persisting in your attempt to avoid the use of detailed 
technical information.  Instead you are repeatedly invoking an appeal to 
authority, posing yourself as the authority.  You are repeatedly making 
firm assertions of definitive, dismissive conclusions, without showing 
your work.

So far, however, the data that have been provided do not substantiate 
your dismissive views.

On the other hand, since your conclusions make clear that you view 
yourself as having definitive knowledge about objective truth, here, it 
should not be difficult for you to document the factual basis for your 
conclusions.  And yet, for some reason, you have not done so, in spite 
of repeated requests.

I note that you were a solid supporter of this draft, notably changing 
only after you got some (frankly, very mild) pushback on 1 August.  I'm 
sure that's merely coincidental.  I'm equally sure that you would not be 
motivated by such pettiness.  Still, it's an unfortunate linkage.

What is more unfortunate is your persistent effort to attempt 
intimidation rather than constructive participation.  Bullying is a 
hallmark of insecurity.  This particular style of bullying is one that 
the IETF not only fails to deal with, but frankly rewards.  That, of 
course, is worse than unfortunate, since it encourages others to keep 
away and damages the quality of the work done in the IETF.

As for the last line of your above posting, you are encouraged to use 
your extensive knowledge on this topic and to produce that survey. 
Given the strength of your views about the header field, it won't take 
you much time.

d/

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Thu, 05 Aug 2021 21:28:18 -0400

> technical information.

[~/Mail] find . -type f -mtime +7300 -name '[0-9]*' | tee /tmp/over20 | xargs grep -i ^Delivered-to: > /tmp/deliveries
[~/Mail] wc /tmp/over20
 35421  35421 507694 /tmp/over20
[~/Mail] wc /tmp/deliveries
 13694  31246 801887 /tmp/deliveries

So literally *last century* 40% of the mail I kept around arrived with a
Delivered-To: tag.  None were local deliveries in my mailbox, and the vast
majority were various mailing lists (9,759 were various SecurityFocus lists,
2,306 from the NANOG list, and 1,629 from a total of 54 different sources
from literally every continent.

Now can we get rid of the "We don't have detailed technical information"
straw man to avoid admitting that this tag has been around for an
Internet-archaeological amount of time?

---

From: Dave Crocker <[email-masked]>
Date: Thu, 5 Aug 2021 18:39:57 -0700

On 8/5/2021 6:28 PM, Valdis Klētnieks wrote:
> So literally*last century*  40% of the mail I kept around arrived with a

That's nice.  However it does nothing to tell us what was IN those 
header fields.  That's the issue that is being contested.

Also, you didn't provide any source data, or a summary of the details 
about that source data, except where the fields were generated.

So, again, it does nothing to aid in resolving questions about adequacy 
of the current draft text.

> Internet-archaeological amount of time?

No.

Feel free to try again.  You might want to review the recent email 
discussion, to clarify what questions are at issue.

d/

---

From: "John R Levine" <[email-masked]>
Date: 5 Aug 2021 22:37:02 -0400

>> xargs grep -i ^Delivered-to: > /tmp/deliveries
>> [~/Mail] wc /tmp/over20
> That's nice.  However it does nothing to tell us what was IN those header 
> fields.  That's the issue that is being contested.

Glad to hear we agree that the sensible approach is to research and 
document the existing practice.

It's still fairly hard, though.  I expect that all of those headers 
contain something that looks like a mailbox, but there's no 
straightforward way to tell from outside whether it is a loop breaking 
token or an actual mailbox.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Dave Crocker <[email-masked]>
Date: Thu, 5 Aug 2021 20:00:24 -0700

> document the existing practice.

If only I'd thought to ask for data, rather than just spout a conclusion 
without proucing that data.

> token or an actual mailbox

That's ok, since that is yet another issue that makes no difference to 
the current version of the specification.

d/

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 8 Aug 2021 19:02:02 +0200

> This has changes intended to cover the existing use of the field.

I read the draft anew.  Some nits/ notes/ questions below:

*3.  Delivered-To*

s/am email delivery/an email delivery/.

The new syntax looks puzzling:

    "Delivered-To:"  FWS *phrase Mailbox *phrase CRLF
                     ; Mailbox is from [SMTP]

The presence of phrases around the Mailbox makes this field difficult 
to parse.  Without angle brackets, the presence of an unquoted "@" 
being the only difference.  It is puzzling because phrase usage is 
neither discussed nor exemplified, and IME not seen in the wild.

*5.  Security Considerations*

I'd explicitly note that users concerned abut divulging their final 
recipient address should be wary about forwarding messages as 
attachment, e.g. when complaining about spam, because the topmost 
Delivered-To:'s can betray secret addresses.  How about recommending 
that MUAs provide a tool to remove that info?

*7.  Experimental Goals*

It is not clear whether implementations that are already recording 
Delivered-To: into messages but not exactly as specified in the I/D 
are supposed to change their code in order to comply rather than 
comment to the mailing list about their discrepancy.

Best
Ale
--

---

From: Dave Crocker <[email-masked]>
Date: Sun, 8 Aug 2021 17:43:15 -0700

On 8/8/2021 10:02 AM, Alessandro Vesely wrote:
> The presence of phrases around the Mailbox makes this field difficult to 
> parse.  Without angle brackets, the presence of an unquoted "@" being 
> the only difference.  It is puzzling because phrase usage is neither 
> discussed nor exemplified, and IME not seen in the wild.

The draft is attempting cover existing practice.  We've had only a few 
examples with detail supplied.

It is clear that one of the popular uses is to record the RCTP-To 
address.  It is also clear that there are other strings recorded:

      Delivered-to: [email-masked]
      Delivered-To:[email-masked]

And one that I'm not finding now, which had some free-form text, 
preceding a string conforming to email syntax.  This is what motivated 
the 'enhanced' syntax.  I don't recall its having the angle-bracket, 
though yes, that creates a parsing challenge.

We've also had the comment:
> On 8/3/2021 10:10 PM, Viktor Dukhovni wrote:
>> In Postfix, the address that is presently recorded is the result of any
>> internal rewriting such as mapping"[email-masked]"  to some internal
>> "[email-masked]", that may not correspond to any public address
>> of the recipient. 

but we have not been shown a resulting Delivered-to example's details, 
to ensure covering its syntax in the draft.

> recipient address should be wary about forwarding messages as 

Because such a warning will have no effect.

The existing text notes that this field creates potential PII disclosure 
concerns.  PII disclosure is a complex topic and no short note, buried 
in an obscure specification, is going to improve system behaviors.

> that MUAs provide a tool to remove that info?

1. Same answer as above

2. It is thoroughly inappropriate for an IETF document to recommend the 
details of MUA behavior that are not specific to protocol compliance.

> Delivered-To: into messages but not exactly as specified in the I/D are 
> supposed to change their code in order to comply rather than comment to 
> the mailing list about their discrepancy.

Implementations that seek to conform to the specification should conform 
to the specification.  All other aspects of implementations are out of 
scope.

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 00:50:37 -0400

> neither discussed nor exemplified, and IME not seen in the wild.

Nobody puts unstructured phrases into "Delivered-To".  There are no
display names in this context, because the payload is derived
exclusively from an envelope address.  The "phrase" bits are redundant
and incompatible with current practice.

-- 
    Viktor.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 07:45:34 -0700

> and incompatible with current practice.

Folks,

I modified the syntax to the current version because I remembered see an 
example that had free text before the address, in the field.

I've just done an exhaustive review of messages here, about 
delivered-to, and am not finding that example.  I think I know what I 
mis-read, but in any case...

What I've observed from this latest review is:

      1.  In all cases, the Delivered-To field contains a string that is
          in the form of an email address; that is: Mailbox (from RFC
          5321)

      2.  The strings regularly appeared to be the same as was in the
          SMTP RCPT TO command

      3.  Sometimes they had an alternative semantic, per the references
          to an 'internal' form

      4.  These internal forms appear to be a variant of the address that
          was in the RCPT TO command

If any of the above assertions of fact do not match your own assessment 
of existing practice, please document the details.

(Given the nature of 'mapping' mechanisms, I'd expect #4 to be more 
restrictive than what is actually allowed by existing implementations, 
but again, please provide concrete details that make clear what the 
less-restrictive rule is.)

d/

---

From: "John Levine" <[email-masked]>
Date: 9 Aug 2021 11:13:06 -0400

>of existing practice, please document the details.

If this draft is going to go anywhere, it needs to be written and
edited by people who are familiar with the existing practice.

It is a waste of everyone's time to "document" or "prove" that
Postfix, Courier, Exim, qmail, et al do what they have been doing for
20 years.

R's,
John

PS: Once again, it would be quite reasonable to write up an informational draft about
the way Delivered-To actually works, no experiments needed.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 08:16:08 -0700

John,

Among your ploys to avoid providing documentation for your assertions of 
fact about this header-field, this one is probably the most creative.

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 11:37:53 -0400

>           in the form of an email address; that is: Mailbox (from RFC

* The message might never have been transfered via SMTP in the first place.
* The address in question is placed in a message header, rather than used
  in an SMTP command.

Therefore, Postfix uses an RFC*22 encoding, addr-spec per:

    https://datatracker.ietf.org/doc/html/rfc5322#section-3.4.1

but, IIRC Jeremy reported that when misconfigured by the operator,
Exim can emit an address-list per:

    https://datatracker.ietf.org/doc/html/rfc5322#section-3.4

>       2.  The strings regularly appeared to be the same as was in the

This is rather site-dependent.  For some systems this is always the
case, for others never the case, and there also cases where this depends
on the particular recipient.

>       3.  Sometimes they had an alternative semantic, per the references

This is not a different semantic, the address in Delivered-To is always
the output envelope recipient to which the message is delivered, rather
than the input envelope recipient that triggered the delivery.

In many (though perhaps not most) cases the address conveyed by the MTA
to the LDA is not equal to the input SMTP "RCPT TO" (or, more generally
the input envelope recipient address, when SMTP was not how the message
came in).

>           was in the RCPT TO command

They are not "variants" they are derived from the input envelope
recipient address, but can be any combination of:

    * A change in the domain part to effect a routing change.

    * A change in the local part to map from a First.Last or
      similar public identifier to a user account identifier, ...

    * An individual recipient of an email addressed to a group
      mailbox (a list alias).

    * Replace the local part with some LDA-private loop-detection token

-- 
    Viktor.

---

From: Ned Freed <[email-masked]>
Date: Mon, 09 Aug 2021 08:29:29 -0700 (PDT)

> and incompatible with current practice.

Unfortunately this is not the case. Here's one example, from last year:

  Delivered-to: mailing list [email-masked]

I don't know if this is a thing all Yahoo Groups do/did, but this
particular group did.

Here's an example of one with an interesting local part:

  Delivered-to: [email-masked]
  Delivered-to: "eastgate-eastgate:com-eastgate-list"@eastgate.com

Best not assume nobody uses quoted local parts. I also have to wonder
if the [email-masked] should have been exposed.

And another interesting one:

  Delivered-to: [email-masked]

I again have to wonder if the IETF really wanted that particular internal
address exposed.

In any case, the real question, I think, is not whether or not examples can be
found of incompatible use of this field - they can - but whether we want to
require implementation to support what is best a rare usage. Speaking for
myself, I think the answer should be "no".

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 11:57:48 -0400

> myself, I think the answer should be "no".

The (at least dominant if not exclusive) present use of the field is
loop detection.  In this case an MTA only needs to recognise the header
that it itself emits, and only for a matching addr-spec.  All other
forms can be safely ignored.

The only other requirement is to not strip extant "Delivered-To" headers
when adding another.  The order is not actually significant, but it is
best to prepend as with trace headers.

-- 
    Viktor.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 09:35:00 -0700

>      https://datatracker.ietf.org/doc/html/rfc5322#section-3.4

Interesting point.  Not surprisingly, I'd be quite happy to have RFC5322 
be the baseline reference, rather than RFC5321.

It does achieve independence of the transfer mechanism, which is useful, 
per your example.  There doesn't seem to be any inherent need to be 
anchored to SMTP, especially given some existing practice of 'internal' 
transformations, which occur after SMTP has left the scene (assuming it 
was ever present.)

Anyone object to this change, making the syntax use RFC 5322 addr-spec?

> on the particular recipient.

Well, #2 was phrased essentially as a statistical observation, which 
means it isn't intended to cover all cases.

So I think your comment moves us to #3, below...

> the input envelope recipient address, when SMTP was not how the message

OK.  The current draft does not actually declare details about the 
semantics of the string, other than the very general:

      "The header field contains information about the individual mailbox
       that is used to determine the immediate location for that
       transition."

It might be ok to leave this without more detail.  Simpler and less 
constraining.

If we add more detail, why would it be important to the use of the 
specification?

Perhaps it would be helpful to add something non-normative, like:

      The information can be the address specified by the message author
      or by a later mapping to an Internet address by an intermediary, or
      it can be an internal reference to the mailbox, made by the system
      effecting final delivery.

Thoughts?

>      * Replace the local part with some LDA-private loop-detection token

(I assume LDA means local delivery agent?)

Continuing the idea of adding non-normative description, perhaps adding 
further, to the above "Perhaps" candidate text:

      The internal reference can be produced according to whatever rules
      the local system imposes.  Examples include: using a different
      domain name due to further internal routing, or mapping the local
      part to a user account identifier.

I left off the reference to a token because, so far, we have not seen an 
example of such an arbitrary string, in spite of repeated references to 
them.  (Note that the text does not prohibit random tokens, within the 
syntax of an email address, but without an example of current use, it is 
too theoretical to include in the list of examples.)

I left off the alias reference because, really, that's just an example 
of mailing list behavior.  (Yes, I understand that the implementation is 
quite different, but especially for this topic, the external observables 
are the same.)

d/

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 09:45:07 -0700

>    Delivered-to: mailing list [email-masked]

ahh, thanks.  That's an example of what I thought I'd seen but couldn't 
find this morning.

> myself, I think the answer should be "no".

Different semantics, within the framework of the same, email-address 
syntax, seems a relatively minor point to me, and is arguably covered 
adequately by the generic prose already in the draft, about the nature 
of the string.

On the other hand, the difference between allowing arbitrary text 
(phrase) in the field's payload seems a big deal to me.  As noted, it 
messes with parsing, given the absence of the usual delimiters.

Having the spec formally exclude an existing practice is obviously a big 
deal.

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 13:07:21 -0400

> If we add more detail, why would it be important to the use of the 

The key unanswered question is what the "audience" for the header
content.

If it is just MTAs for loop detection or else diagnostic information for
a human reader, then the format need not be strictly specified, each MTA
records what it wants to record, and then deals with its format should
the message come back around.

The human reader can read the tea leaves as best he can.

If the draft's intent is to expose the header for MUA consumption, then
it would need to be more clearly specified, but this is not compatible
with existing practice, and that role is played by various other
headers, and perhaps specifying a common header to consolidate:

    - X-Original-To: (Postfix)
    - Envelope-To: (IIRC Exim)
    - ...

would be a more useful exercise than trying to saddle Delivered-To
with a new purpose.

*WHY* are we talking about this draft???  What actual problem is it
trying to solve?

> (I assume LDA means local delivery agent?)

Yes, a.k.a. MDA.

> example of such an arbitrary string, in spite of repeated references to them.

Even if not currently done, Postfix is under no obligation to record
the actual address rather than some obfuscated hash, ...

> quite different, but especially for this topic, the external observables 

The reason that list expansion came up, is that the original text
repeatedly mentioned Delivered-To recording the *input* envelope
recipient (via RCPT TO), but list expansion replaces an input
envelope recipient with multiple output envelope recipients, and
in some cases these are then the "Delivered-To:" addresses in
the various recipients' individual message copies.

-- 
    Viktor.

---

From: Jeremy Harris <[email-masked]>
Date: Mon, 9 Aug 2021 21:15:19 +0100

>      https://datatracker.ietf.org/doc/html/rfc5322#section-3.4

If we're speaking of a Delivered-To: header, then no I did not.
Exim does not currently have a specific facility for generating
one.

Perhaps you were thinking of my description of Envelope-to:

(Naturally, you can configure to construct any custom header,
and do it that way with whatever content you wish)
-- 
Cheers,
   Jeremy

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 9 Aug 2021 22:41:11 +0200 (CEST)

> content

+1

A nice example of the proverbial solution in search of a problem.

/rolf

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 13:52:36 -0700

> A nice example of the proverbial solution in search of a problem.

 From the Introduction, what about this is insufficiently clear or complete?

> Given aliasing, mailing lists, and the like, the transit of a message
> from its author to a final recipient might include a series of
> submission/delivery events. Also, the delivery process can produce
> local address transformations. It can be helpful for a message to
> have a common way of indicating each delivery in the handling
> sequence, and to include each address that led to the final delivery.
> This can aid in the analysis of a message's transit handling. An
> additional use can be for detecting a delivery sequence loop. With a
> loop, the same copy of a messages transits the same email address
> more than once. This is different from having the message simply
> transit the same MTA more than once, which might be necessary, such
> as when it is processed through a mailing list; an MTA services many
> addresses. It is also different from having two copies of the same
> message arrive at the same, ultimate mailbox, having been originally
> posted to two different addresses.
> 
> Delivering the same copy of a message more than once, to the same
> address, is almost certainly not an intended activity. An example of
> a problematic arrangement would be to send a message to mailing list
> listA, where listA contains an entry for listB, and listB contains an
> entry for listA. The message will enter an infinite loop. Loop
> detection for email can be a complicated affair. The Delivered-To
> header field provides helpful information with a definitive
> indication that this copy of a message has been delivered to a
> specific mailbox.

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 16:53:09 -0400

> Perhaps you were thinking of my description of Envelope-to:

Yes, thanks for the correction.  Just to be sure I'm not missing
some important detail, I think you're saying that in Exim there
is no "Delivered-To:"-based or similar loop detection mechanism,
or least nothing built-in.

And the "Envelope-To:" header is for fetchmail and the like to
demultiplex multi-drop mailboxes.  It is not a loop detection
feature.

-- 
    Viktor.

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 17:28:49 -0400

>  From the Introduction, what about this is insufficiently clear or complete?

All of it.  What is the new experimental use of the header?

> > It can be helpful for a message to have a common way of indicating
> > each delivery in the handling sequence, and to include each address
> > that led to the final delivery.

Useful to whom and to what end?

> > This can aid in the analysis of a message's transit handling.

OK: 1. Human reading tea leaves.

> > An additional use can be for detecting a delivery sequence loop.

OK: 2. Existing loop detection use-case.

So the only proposed uses are the current ones, which is reassuring, but
also rather invites the question of what exactly is then the the nature
of the proposed "experiment"?

Instead of promposing an "experiment" why not just clarify the existing
uses above, and making it clear that the header is a trace header, and
that its payload SHOULD be an "addr-spec" from RFC5322.  The draft can
then be informational.

-- 
    Viktor.

---

From: "John R. Levine" <[email-masked]>
Date: 9 Aug 2021 17:29:07 -0400

>> Exim does not currently have a specific facility for generating one.
>>

Right, the header described in draft-crocker-email-deliveredto-05 is in 
fact Envelope-To, not Delivered-To.

As far as I can tell, based on Viktor, Sam, Jeremy, and me descring the 
operation of software we know in detail, Delivered-To never records an 
envelope address.  When the string looks like a mailbox, that's a 
coincidence of implementation where the internal form happens to resemble 
the inbound envelope and the MTA's address translations were not too 
complicated.

Dave has pointed out that Gmail also adds a Delivered-To address, but I 
have looked and found no description of it at all, just some speculation 
in ServerFault.  I would guess that Google is familiar with existing 
Delivered-To and implemnted it the same way, as a loop breaker.  But 
that's just a guess.  Does anyone have any documentation or personal 
knowledge of what Gmail's header does?

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Aug 2021 15:31:46 -0700

On 8/9/2021 2:28 PM, Viktor Dukhovni wrote:
> All of it.

Really, you didn't understand a single word of those two paragraph?

Perhaps you can offer some text that you'd find more helpful?

 >  What is the new experimental use of the header?

We've gone over that.  I gave a lengthy response.  It's fine that you 
don't agree with it, but it's not helpful to keep repeating your 
unhappiness with it.

> Useful to whom and to what end?

To anyone who wants the information.  Again:

    -  what specifics are you looking for?

    -  what additional or different detail do you think is needed?

> OK: 1. Human reading tea leaves.

I don't understand what that means for this document.

> of the proposed "experiment"?

asked and answered, as the lawyers like to say.

> that its payload SHOULD be an "addr-spec" from RFC5322.  The draft can

Let's focus on the substance of the document and worry about process and 
labels later (or never).

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 9 Aug 2021 20:32:25 -0400

> fact Envelope-To, not Delivered-To.

Well, yes and no.  On the one hand the subtext seems to be to try to
present a usable envelope recipient address to various tools, but on
the other hand the text is specifically about "Delivered-To" and
includes the loop-detection case.

I keep asking about the motivation for the draft, what problem is
it intended to address...  And yet, the only response I get is
requests to identify specific issues in the text.

But it is not possible to know whether the text is solid or not, unless
I know what the problem being solved is.

I am told the problem statement is in the introduction, but that is
either not the case, or the motivation is too weak to support a useful
draft.  It just states some generalities about how it could be useful
to record the delivery address, with little apparent explanation of
what tangible benefit this brings over existing practice.

Are we trying to make "Delivered-To" loop detection more uniform across
M[TD]As?  Are we trying to support multidrop mailboxes?  Something else?

If there's nothing beyond what's in the draft, then it is about time to
make it clear that it is no experiment, and is just trying to document
current practice.

-- 
    Viktor.

---

From: Richard Clayton <[email-masked]>
Date: Tue, 10 Aug 2021 01:53:15 +0100

In message <[email-masked]>, Viktor Dukhovni
<[email-masked]> writes

>M[TD]As?  Are we trying to support multidrop mailboxes?  Something else?

sadly no-one seems to be picking up on promoting the use of opaque
strings for the detection of loops which is (a) privacy preserving
(which cannot be dismissed out of hand) (b) more robust (since machines
often think they have the same name, when they should not (@local)) and
(c) is to a large extent what is happening already in that discussion
often says that only the particular MTA can possibly understand what the
"human readable" strings actually mean, viz they are already opaque ...

myself, when I look for loops I am staring at my own logs and perhaps
on-the-wire totals rather than trying to pin down a sample email and
examine all the headers --- the last of these is particularly hard when
you are running a mail system for "other people" and you have to get a
lawyer to hold your hand before you can inspect actual traffic ... viz I
don't generally care what other MTAs put into headers, I care what they
send me (or refuse to accept)

>make it clear that it is no experiment, and is just trying to document

which, if it is entirely practice that occurs within a single MTA is not
something that is usefully standardised across MTAs -- unless one is
trying to promote some Best Practice in which case settling on practices
which are more acceptable in this century is, I suggest, desirable

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: "John Levine" <[email-masked]>
Date: 10 Aug 2021 16:08:56 -0000

According to Richard Clayton  <[email-masked]>:
>something that is usefully standardised across MTAs --

We know of at least three MTAs with the loop breaking semantics, qmail
which originated it, Postfix, and Courier which adapted the idea.

I know that few people still run qmail but it is my impression that large parts
of it lives on on hidden in other software.

Even if it's not all that popular, documenting its current use could discourage
people from reusing the same name for a different purpose which as we've seen
may be a problem.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Aug 2021 11:36:56 -0700 (PDT)

> adequately by the generic prose already in the draft, about the nature

I'm talking about syntax. Not semantics. Requiring that implementations parse
what is effectively a new string format - AFAIK there is no  other place in any
of our specifications where

   FWS *phrase Mailbox *phrase

is used - may not be a significant burden to folks who spend a lot of time
dealing with email syntax, but to people who just want to hack something
together using some random email routine library, it's an issue. Having looked
at a lot of code written this way, the likely outcome is that they will call
something that's "close", which will probably handle  the case without the
phrases but will fail or produce bogus output when they are.

And as far as semantics are concerned, your syntax assumes that the 
the prefix and suffix parts are not, in fact, part of the mailbox. That
may or may not be correct.

> Having the spec formally exclude an existing practice is obviously a big

When formalizing things we often exclude rare practices that greatly
complicate things, especially when those practices are semantically
ambiguous.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Aug 2021 12:10:45 -0700

On 8/12/2021 11:36 AM, Ned Freed wrote:
>>> require implementation to support what is best a rare usage.
>> Speaking for myself, I think the answer should be "no".
> 
>> Different semantics, within the framework of the same,
>> email-address syntax, seems a relatively minor point to me, and is
>> arguably covered adequately by the generic prose already in the
>> draft, about the nature of the string.
> 
> I'm talking about syntax. Not semantics. Requiring that
> implementations parse what is effectively a new string format - AFAIK
> there is no  other place in any of our specifications where
> 
> is used - may not be a significant burden to folks who spend a lot of
> time dealing with email syntax, but to people who just want to hack
> something together using some random email routine library, it's an
> issue. Having looked at a lot of code written this way, the likely
> outcome is that they will call something that's "close", which will
> probably handle  the case without the phrases but will fail or
> produce bogus output when they are.
> 
> the prefix and suffix parts are not, in fact, part of the mailbox.
> That may or may not be correct.
> 
>> (phrase) in the field's payload seems a big deal to me.  As noted,
>> it messes with parsing, given the absence of the usual delimiters.
> 
>> Having the spec formally exclude an existing practice is obviously
>> a big deal.
> 

I would very much prefer the syntax to just be Mailbox. But what with 
the excited concern for covering established practice, this alternative 
syntax seemed necessary.

However, given your enthusiastic prompting -- which I will take as 
support -- I'll revert the syntax, but add some text noting the issue.

Anyone objecting strongly is encouraged to comment with explanations for 
why just having the field contain a string in the form of an email 
address is not sufficient.

d/

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Aug 2021 12:13:55 -0700

On 8/12/2021 12:10 PM, Dave Crocker wrote:
> I would very much prefer the syntax to just be Mailbox.

Forgot that there is the suggestion to switch to RFC 5321 as the base, 
which I'll do.

That means the reference here should have been addr-spec.

d/

---

From: Jeremy Harris <[email-masked]>
Date: Thu, 12 Aug 2021 21:02:29 +0100

On 09/08/2021 21:53, Viktor Dukhovni wrote:
>  I think you're saying that in Exim there

That's correct.

> demultiplex multi-drop mailboxes.  It is not a loop detection

I'm not sure why the option for it was added; before my time.
As I said, it's off by default.
-- 
Cheers,
   Jeremy

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Aug 2021 15:51:50 -0700 (PDT)

> the excited concern for covering established practice, this alternative

Something I managed to find one or two occurences of by looking through tens of
thousands of old messages is rather different than the behavior implemented
several of the most widely used MTAs.

It's also different in that the use of the field in those MTAs is strictly for
loop detection, which makes the format a local matter. The draft aims for a
different - and in some cases disjoint - logging usage, which really calls for
a common syntax.

This does not make me particularly happy, BTW. Our MTA doesn't currently
support this particular type of loop detection, but if I were to add it the way
I'd prefer to do it would produce a result that's a bit too big to put in a
mailbox. I'd end up having to use a different field. A looser syntax would
be handy.

There's an alternate way to do the loop detection that's much simpler and would
fit in a mailbox (!), but I haven't managed to convince myself that it never
expose something that shouldn't be exposed.

> address is not sufficient.

For logging purposes it's a restriction you want to have. For loop detection
it's a restriction you don't need, but if you're going to use the same
field name for both the need for legible logging has to take precedence.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Aug 2021 18:08:20 -0700

On 8/12/2021 3:51 PM, Ned Freed wrote:
> 
> There's an alternate way to do the loop detection that's much simpler
>  and would fit in a mailbox (!), but I haven't managed to convince
> myself that it never expose something that shouldn't be exposed.

If you feel like writing text to describe it -- including
tenativeness/caveats -- I'd be glad to add it.  I'm reworking some text,
so I think there will be a place that can have such text added easily.

d/

---

From: "John Levine" <[email-masked]>
Date: 12 Aug 2021 22:49:44 -0400

>different - and in some cases disjoint - logging usage, which really calls for

As far as we know, all of the existing usage is for loop detection, so it would
be reasonable to say the contents of the header is a token or sequence of tokens
with semantics local to the MTA that adds the header.  I suppose we could say something
like what we say with Message-ID, to use a local domain to avoid collisions.

What is not reasonable is to overload this existing header with new, incompatible
semantics and no way to tell the difference.  The obvious solution, of course,
is to use a different header name.  If the goal is to record the SMTP envelope
address, Envelope-To seems to match what Exim does.  If the goal is something else,
make up a new header name.  It's not like they're hard to invent.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Fri, 13 Aug 2021 11:02:19 -0700

> be reasonable to say the contents of the header is a token or sequence of tokens

The strings we have documentation for do not look nearly as abstract as 
the term token implies.  Rather, they look like email addresses.

When you produce fielded examples of strings that look more opaque 
and/or random, a change in terminology might be worth considering.

> with semantics local to the MTA that adds the header. 

The strings we have documentation for sometimes demonstrate local 
idiosyncrasy, but only sometimes.

For the most part, they seem to reflect the transit destination email 
address.

> I suppose we could say something

The current syntax already specifies use of a domain name.

> semantics and no way to tell the difference. 

Please provide details that substantiate a claim of incompatibility.

The current spec supports documented existing use, except for the 
example with preceding commentary free-form text.

  The obvious solution, of course,
> make up a new header name.  It's not like they're hard to invent.

Constantly repeating the same statement of your preference is 
constructive how, exactly?

d/

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 13 Aug 2021 20:09:06 +0200

> semantics and no way to tell the difference.

Without formal overloading, there is no reason to exclude the 
possibility that an MTA records (virtual) mailbox addresses in that 
field.   Local MTA scripts can use the values of Delivered-To: fields, 
if they can be interpreted correctly.  Also MUAs can take advantage of 
any mailbox values found there, as a sponger.  For example, 
Thunderbird has a configuration item named catchAllHeaders which can 
be set to a list such as "envelope-to, x-original-to, to, cc, 
delivered-to".

How about:

     "Delivered-To:"  FWS addr-spec / opaque-tokens FWS CRLF
?

After all, interoperability with MUAs and local operations is one of 
the experimental goals.

Best
Ale
--

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Aug 2021 11:47:36 -0700 (PDT)

>      "Delivered-To:"  FWS addr-spec / opaque-tokens FWS CRLF

This makes no sense to me at all. Nothing prevents you from using address
syntax - specifically the local part - to encode a value that only you can make
sense of. (My only concern is the length limit, but nobody else has echoed this
sentiment.)

Indeed, the ability of the domain to signal that this is your token, rather
than someone elses that might be similar enough to fool you, is pretty useful.

> After all, interoperability with MUAs and local operations is one of

IMO this is the opposite of interoperability, and given that no evidence
has been presented of anyone using this format, I see no reason to make
such a change.

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Fri, 13 Aug 2021 15:04:46 -0400

On Fri, Aug 13, 2021 at 11:02:19AM -0700, Dave Crocker wrote:

> The strings we have documentation for do not look nearly as abstract
> as the term token implies.  Rather, they look like email addresses.

The above shows a profound unwillingness to look beyond the apparent
syntax to the semantics committed to by the MTAs that are using this
header.  The extant implementations of "Delivered-To" make no promise
to record anything recognisable to any party other than the MTA.

At least in the case of Postfix, we specifically go to the trouble of
optionally prepending a separate "X-Original-To" header which records
the input address that led to the recipient, which in contrast to
"Delivered-To" is specifically intended for consumption by downstream
user agents.

It is not acceptable to violate the interface *contract* and saddle the
"Delivered-To" header with new semantics, whereby the recorded address
is anything other than an opaque token.  If that's the inention, you can
end the experiment now.  No such commitment will be made in Postfix, the
header payload will remain a private nonce for exclusive use by the MTA.

No matter how many times you repeat that "Delivered-To" is in practice
observed to be some variant of an actual email address, Postfix will
retain the right to use some other semantics that, while conforming to
an "addr-spec", may hold something that only the generating MTA can
generate and recognise.

-- 
    Viktor.

---

From: Viktor Dukhovni <[email-masked]>
Date: Fri, 13 Aug 2021 15:20:13 -0400

On Fri, Aug 13, 2021 at 08:09:06PM +0200, Alessandro Vesely wrote:

> Also MUAs can take advantage of any mailbox values found there, as a
> sponger.  For example, Thunderbird has a configuration item named
> catchAllHeaders which can be set to a list such as "envelope-to,
> x-original-to, to, cc, delivered-to".

The "Delivered-To" header is NOT for MUA consumption.  Any MUA using
"Delivered-To" for anything other than display to the user as diagnostic
information is violating the contract, and deserves whatever
misbehaviour ensues when it seems something new and/or unexpected.

-- 
    Viktor.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 15 Aug 2021 14:18:46 -0700

> to record anything recognisable to any party other than the MTA.

Documenting existing practice means documenting practice that actually 
exists.

Hypothesizing about what might sometime be done by someone somewhere, 
when there is no evidence it will be done -- nevermind that it has not 
yet been done -- is not documenting existing practice.

And by the way, the document is quite clear that loop detection is only 
one use it can be put to and that loop detection often involves a 
complex of methods.

So I don't really know what you mean about looking to the semantics, 
since the document supports the semantics of current practice.

Since you disagree, please be specific. What semantics are there, in 
current practice, that are not supported by the current draft?

> "Delivered-To" is specifically intended for consumption by downstream

The creation of other, independent header fields, has nothing to do with 
the technical details of this field.

Since you disagree, please be specific about the details of the current 
draft that need to be changed, to reflect current practice for THIS 
header field.

(If this sounds the same as the request, above, it's because it's the 
same request.)

> It is not acceptable to violate the interface *contract* and saddle the

That's nicely forceful and legalistic language, but there isn't a 
contract.  There is a goal of being compatible with existing practice, 
which this draft is.

> is anything other than an opaque token.

Sorry but the existing practice is the syntax of an email address and, 
often, actually the email transport recipient address.

Claiming otherwise might satisfy one's sense of abstraction, but goes 
beyond existing practice.

> an "addr-spec", may hold something that only the generating MTA can

Implementors are always free to deviate from a specification.

Whether that defeats interoperability is a different matter.

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 15 Aug 2021 17:51:52 -0400

> Documenting existing practice means documenting practice that actually 

When documenting *interfaces*, we do not document implementation
artefacts of what the interface happens to be doing.  We document
the interface contract, i.e. what the interface is *expected* to do.

When I say that "Delivered-To" is an opaque token private to the MTA,
this is not an empirical statement.  It is a statement of fact about the
contract between Postfix and the rest of the world.

> yet been done -- is not documenting existing practice.

There is no hypothesis.  See above.
 
> one use it can be put to and that loop detection often involves a 

Other than loop detection, and forensic evidence trails, what else
motivates the draft?

> since the document supports the semantics of current practice.

What is the nature experiment?  If you're documenting current practice,
how can the draft be "Experimental"?

> current practice, that are not supported by the current draft?

What is the motivation for producting the draft?  What problem are you
actually tryign to solve?

> draft that need to be changed, to reflect current practice for THIS 

Only if you're able to respond with a cogent reason for the draft to
exist, I haven't seen one yet, and no the listing of a few plausible
uses in the draft is not such a reason.

There needs to be a higher layer reason for defining the header at this
time, and clarity around whether any semantics are being imputed, or
we're just documenting existing syntax.

If the goal is to clarify existing syntax, then:

    * Change the "Experimental" status, to "Informational".

    * Make it clear that the value is opaque, and user agents
      can make no assumption about the payload.

> contract.  There is a goal of being compatible with existing practice, 

If it binds the MTA to produce tokens that an MUA will be able to
parse and act on, then it is NOT compatible with existing practice.

> often, actually the email transport recipient address.

No existing practice is the interface supported by the MTA, and the
MTA's freedom to make any and all changes that don't violate that
interface contract.

> beyond existing practice.

Surprisingly, you're conflating implementation with specification.

> Implementors are always free to deviate from a specification.

No, specifications of existing practice can't ignore the interface
contracts of existing implementations, or else they're not documenting
existing practice.

In any case, the payload of the produced strings is sufficiently
variable given the various kinds of rewriting in play, that even without
the tokens going as opaque as say hex-encoded HMAC values the MUA cannot
treat the value as an identifier from which to extract a usable
recipient address.

-- 
    Viktor.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 15 Aug 2021 21:05:12 -0700

> When documenting *interfaces*, we do not document implementation

This is a protocol/format specification, not an interface specification. 
  It documents what goes over the wire, as IETF typically does.

> the interface contract, i.e. what the interface is *expected* to do.

That isn't what IETF specifications do.

> When I say that "Delivered-To" is an opaque token private to the MTA,

That's nice, but does not represent the observable existing behavior. 
And what /you/ mean is less of an issue that was actual existing 
practice is.

> contract between Postfix and the rest of the world.

There is behavior, not a contract.

> Other than loop detection, and forensic evidence trails, what else

It's already stated in the draft.

> how can the draft be "Experimental"?

Victor, it is not helpful for you to keep asking a question or 
challenging a point that has already been repeatedly answered.

> actually tryign to solve?

Read the draft.

It would also be nice for you to answer the questions /I/ am asking /you/.

> parse and act on, then it is NOT compatible with existing practice.

Ahh.  So you mean the specification should specify behaviors.  It does that.

> MTA's freedom to make any and all changes that don't violate that

So there is already a written 'contract' -- presumably you mean 
'specification' that all existing systems conform to?

Absent that, there is no contract.  There is only existing practice. 
The document seeks to specify that practice, but in a larger context of 
utility.

The document contains technical details.  Other than the abstractions of 
contract and token, you have yet to point at technical details in the 
draft that are not correct.

> Surprisingly, you're conflating implementation with specification.

Nope.

d/

---

From: Ned Freed <[email-masked]>
Date: Mon, 16 Aug 2021 12:14:43 -0700 (PDT)

> And what /you/ mean is less of an issue that was actual existing

We need to be very careful about drawing conclusions like this from
"observable" data. My data was collected from mailboxes delivered to by an MTA
that isn't configured to insert Delivered-to: fields. This means that almost
all of Delivered-to: fields I'm seeing are ones inserted in messages forwarded
by a mailing list. In such cases it's expected that the field values will
either some sort of list address or one of my externally visible addresses.

But things would be different if I was using Postfix, or if I had my MTA
configured to do what Postfix is documented to do when delivering to local
addresses. In this case I would be getting addresses of the form:

    uid+folder+specialuse@lmtp-back-end

Such an address is meaningless to anything but the LMTP server itself,
and in fact would not be accepted as valid by anything except that
server (which of course is not reachable by anything other than authorized
LMTP clients).

I'm not sure the address qualifies as "opaque", which in fact is a bit of a
problem, since tacky or even NSFW folder names aren't exactly unheard of. And
exposure of account UIDs can be a security issue.

But it's definitely private to the MTA.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Mon, 16 Aug 2021 15:30:40 -0700

On 8/16/2021 12:14 PM, Ned Freed wrote:
> "observable" data.

There have been repeated requests for additional examples.  None has 
been forthcoming.

The draft conforms to those examples, other than the prefatory freeform 
text in the one mailing list example.

Interestingly, the new, competing specification also uses addr-spec.

So apparently it is competing on description rather than substance.

d/

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 17 Aug 2021 11:04:44 -0400

> On 16 Aug 2021, at 6:30 pm, Dave Crocker <[email-masked]> wrote:
> 

I guess IPv4 should have been specified by writing down the header
format, but not the semantics of any of the values, which would not
be "substance".

The substance of "Delivered-To" is private use as a loop detection
sentinel.  The syntax is an implementation detail.  Elevating the
syntax and downplaying the semantics is not progress.

-- 
	Viktor.