Subject: Re: [ietf-smtp] Possible cont4ibution to moving forward with RFC5321bis SMTP

From: Hector Santos <[email-masked]>
Date: Thu, 02 Jan 2020 10:47:59 -0500

On 12/31/2019 1:23 PM, Keith Moore wrote:
> On 12/31/19 12:33 PM, Hector Santos wrote:
>
>> I have two SMTP compliancy-based deterministic filters:
>>
>> - Machine name ip-literal matching connecting ip because SMTP tells
>> us it is defined as the IP address of the connecting client, and
>
> This is something that should be clarified in 5321bis, IMO.

+1.

You pointed out much of this, I see four basic issues:

a) Get rid of it,
b) Double down on its original purpose and field definition,
c) Clarify whether either a FQDN or IP-Literal can be used, and
d) Provide rejection insights focused on SMTP compliancy.

With hosted end-users, the false positives seen with NATs has been 
addressed with the SUBMIT protocol or some other client authentication 
that raised the SMTP bar and allowed for receiver restrictions.

With SUBMIT, the wcSMTP server will relaxed EHLO validation since the 
session is expected to be ESMTP AUTHenticated.  It was the first thing 
seen when I added the IP-literal check. I did private and publicly 
noted to the SUBMIT editors the implementation experience because 
SUBMIT would allow for "authorized" EHLO validation and rejection. 
But to me, it would be unnecessary under an expected ESMTP AUTH session.

-- 
HLS

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 12:18:04 -0500

> that raised the SMTP bar and allowed for receiver restrictions. 

In these days of complete IPv4 address space exhaustion, it can not be 
safely assumed that there is no NAT between the MSA and the MX SMTP server.

Keith

---

From: Hector Santos <[email-masked]>
Date: Thu, 02 Jan 2020 12:41:30 -0500

On 1/2/2020 12:18 PM, Keith Moore wrote:
>> addressed with the SUBMIT protocol or some other client
>> authentication that raised the SMTP bar and allowed for receiver
>> restrictions.

> safely assumed that there is no NAT between the MSA and the MX SMTP

Correct.

My approach is how to deal with the exceptions to a well-defined SMTP 
protocol element.   The good intention exceptions will normally 
address the issue promptly, reports are made, including server 
whitelisting, client authentication, etc. The bad intention exceptions 
don't give a hoot.

So by applying frontend SMTP compliancy-based osmosis filters first, 
you will expect to see fast detection and correction of the good 
stream, leaving behind the bad stream and less complaints.  That is 
exactly what has happen, what I have experienced, my customers have 
experienced, in the 17 years of applying these SMTP compliancy filters.

Thanks

-- 
HLS

---

From: Dave Crocker <[email-masked]>
Date: Thu, 2 Jan 2020 10:06:10 -0800

On 1/2/2020 9:18 AM, Keith Moore wrote:
> no NAT between the MSA and the MX SMTP server.

"MX Server"?

Since MX is used for destination side delivery, I'm not sure which MTA 
you mean, since MSAs aren't modeled as talking to the delivery site. 
Please clarify.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 13:51:09 -0500

> you mean, since MSAs aren't modeled as talking to the delivery site. 

I mean a server that accepts incoming mail on behalf of a mail domain.  
Normally they are advertised using MX records.  Sorry if that wasn't 
clear; I thought "MX server" would be an unambiguous shorthand.

To me the point at which a message crosses a boundary between mail 
domains - i.e. from an MTA forwarding outgoing mail from one domain, to 
an MTA accepting incoming mail for what is likely a different domain, is 
interesting.   It's especially important for the standards to get that 
interface right, because that interface is what lets us send mail from 
arbitrary senders to arbitrary recipients and have a reasonable 
expectation of that mail getting there.

If MTAs A and B are both managed by the same mail domain, and A sends 
mail to B, it's not a problem whether B accepts EHLO with ip domain 
literals.   If B accepts them, fine, because it's only internal traffic 
anyway.   If B rejects EHLO with ip domain literals, that's also fine, 
because that same domain can presumably also arrange for A to use a DNS 
name there.    It's when the mail is relayed between mail domains that 
the standards need to provide guidance, because when sending domain and 
receiving domain have different assumptions about what is right, it 
causes unnecessary failures.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 14:00:54 -0500

> experienced, in the 17 years of applying these SMTP compliancy filters.

I get the impression that you think that a mismatch of IP address 
between the client and server, for legitimate (e.g. non-spam) traffic, 
is an exceptional case that can be dealt with via exceptional means.

Even if this is an exceptional case today, I do not believe it will 
continue to be an exceptional case for the future of IPv4. I see the use 
of NAT in IPv4 traffic increasing for as long as there is a public IPv4 
Internet.

(Use of IPv6 address literals could be a different case, but I don't 
want to wade into that swamp today.)

You can obviously do what you like with your own product, and you can 
probably change the behavior of your product relatively quickly should 
you need to.   But RFCs should be stable documents and should try to 
provide the right advice for the long term.   So I suspect that 5321bis 
should make it clear that IPv4 addresses can't be expected to be the 
same between client and server.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Thu, 2 Jan 2020 11:08:21 -0800

On 1/2/2020 10:51 AM, Keith Moore wrote:
> To me the point at which a message crosses a boundary between mail domains
> 

ahh.  ack.

You meant something like: "the sequence of hops starting with the MSA 
and ending with the receiving MTA at the destination site".

I'd misunderstood and thought you meant that the MSA interacted directly 
with the receiving MTA.

d/

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 2 Jan 2020 20:09:35 +0100

On Thursday 2 January 2020 19:51:09 CET, Keith Moore wrote:
>  It's when the mail is relayed between mail domains that the 
> standards need to provide guidance, because when sending domain 
> and receiving domain have different assumptions about what is 
> right, it causes unnecessary failures.

I feel that a brief digression might help to prolong this discussion.

One way to disagree is about IP address family.

If I remember correctly, nothing (else) in 5321/2 requires a v6-only host 
to understand anything about v4. Running a v6-only host is stupid at this 
point, perhaps slightly less stupid in five years, but even so I don't 
think the next RFCs should feature an IPv4 requirement for address 
literals, and nothing for the main parts of the protocols.

Arnt

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 14:34:12 -0500

On 1/2/20 2:08 PM, Dave Crocker wrote:

>> I mean a server that accepts incoming mail on behalf of a mail 
>> domain.  Normally they are advertised using MX records.  Sorry if 
>> that wasn't clear; I thought "MX server" would be an unambiguous 
>> shorthand.
>>
>> domains

Actually I was trying to focus on the single hop between the last MTA of 
an originating domain and the first MTA of the destination domain.

> I'd misunderstood and thought you meant that the MSA interacted 
> directly with the receiving MTA. 

That is possible (and in some cases desirable IMO).   But it's neither 
required, nor what I understand to be typical of today's operations.

(At the same time I question the assumption that all uses of IP address 
literals in EHLO can reasonably be confined to submissions.)

Keith

---

From: Dave Crocker <[email-masked]>
Date: Thu, 2 Jan 2020 12:19:54 -0800

> an originating domain and the first MTA of the destination domain.

then referencing MSA continues to confuse me.

Might be worth looking at RFC 5598, Sections 2.2 and 2.3.

> That is possible (and in some cases desirable IMO).  

Not really, though this highlights the difference between network 
architecture and network implementation.  The former is an abstraction, 
with lots of possible implementation configurations.

If the box doing the MSA also talks to the receiving MTA, then that 
first box is also an MTA.

d/

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 15:30:51 -0500

> then referencing MSA continues to confuse me.

It's part of the discussion history but I don't think it's worth 
re-iterating.

> first box is also an MTA.

Sure.   But we're not always clear about when we're talking about an 
abstraction, vs. when we're using one of these terms to talk about an 
implementation or a service.

For lots of reasons, I expect that such ambiguities will persist.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Thu, 2 Jan 2020 12:48:08 -0800

On 1/2/2020 12:30 PM, Keith Moore wrote:
> But we're not always clear about when we're talking about an 

The established terms are actually rather precise and refer to the 
networking abstraction, not an implementation.

d/

---

From: Keith Moore <[email-masked]>
Date: Thu, 2 Jan 2020 16:10:49 -0500

> networking abstraction, not an implementation.

Good luck trying to get everyone to use these or any other terms, in 
exactly the way you want them to use them, on any particular occasion.

As far as I can tell, people use words imprecisely, and to mean slightly 
different things in different contexts, because it's so often useful or 
necessary for them to do so.  There's nothing new or odd about this.

Keith

---

From: "John Levine" <[email-masked]>
Date: 2 Jan 2020 17:17:24 -0500

>exactly the way you want them to use them, on any particular occasion.

The people I know don't have any trouble understanding that the same system, and
sometimes the same program, is an MSA when it's talking to an internal submission
client, and an MTA when it's talking to another MTA found using an MX.

You might want to reread RFC 5598 so you use the terms consistently
with the way everyone else does.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 2 Jan 2020 17:33:37 -0500

>literals, and nothing for the main parts of the protocols.

Assuming we keep address literals as part of SMTP (which I think would
be a mistake) rather than submission (where I don't care), I think the
as-if rule applies.

If a host doesn't get v4 connections, it doesn't need to recognize v4
address literals.  Similarly, if it doesn't get v6 connections it
doesn't need to recognize v6 literals.  Standards are about
interoperation, and cases that never happen don't matter for interop.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Thu, 2 Jan 2020 19:40:42 -0800

> interoperation, and cases that never happen don't matter for interop.

For defining a common core, the question is what is to be supported by 
everyone, all the time.  Supported is not the same as used, thereby 
distinguishing between requirements for interoperability technical 
specifications versus constraints of operational policy.

Is there any believe that any version of SMTP will be expected to 
operate without support for domain names?  (Again note, I said support, 
not use.)  Without support for IPv4?  v6?

These questions need to be answered in terms of what is desired for 
products being offered, not for operational choices in particular 
environments.

For example, if there really is a view that some SMTP products 
reasonably could operate only supporting IPv6 address, then after we get 
through the discussion of pragmatics that justify recommending 
standardization of that choice, we should consider how to create a 
common SMTP that works for that choice as well as the more general, open 
Internet use.

For this particular example, I'd say that the base protocol spec would 
have to say something generic like 'text' and then leave the constraints 
on the text to particular specifications that tailor things.  mumble...

d/

---

From: Keith Moore <[email-masked]>
Date: Fri, 3 Jan 2020 10:11:15 -0500

> On Jan 2, 2020, at 2:09 PM, Arnt Gulbrandsen <[email-masked]> wrote:
> 
> Running a v6-only host is stupid at this point,

I don’t share that assumption for multiple reasons.  One is that the Internet is a diverse place and there are lots of corner cases.  It’s not the case that every internet host needs to interact with hosts using ipv4.   Another is that v6 networks are likely to support some sort of NAT solution which permits v6 clients to contact v4 servers as long as those servers have dns names and public ipv4 addresses. 

In general I’d recommend that applications that can be IP version agnostic, be so.  As applied to SMTP I think that means accepting both IPv4 and IPv6 syntax, but not actually caring what the peer’s address is.

> perhaps slightly less stupid in five years, but even so I don't think the next RFCs should feature an IPv4 requirement for address literals, and nothing for the main parts of the protocols.

I would like for 5321bis to provide clear guidance to implementors rather than having a practical necessity for them to implement protocol features that have been removed from the standard.  I expect that smtp and http will be the last applications to not need ipv4 support somewhere.  So I don’t think it’s helpful to remove v4 address literals from the smtp protocol.   However it might be worth relaxing the syntax so that at least from a protocol perspective, smtp ehlo doesn’t care what kind of argument is used.  

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 3 Jan 2020 10:22:16 -0500

> On Jan 2, 2020, at 5:33 PM, John Levine <[email-masked]> wrote:
> 
> doesn't need to recognize v6 literals.

Emphatically disagree, because of NATs.  It’s counterproductive to gratuitously impair smtp to prevent it from being used across ipv4/IPv6 boundaries, across boundaries between public and private ipv4 networks, and across boundaries between private ipv4 networks.  It’s also counterproductive to put something into a specification that implementors have to ignore for pragmatic reasons.

Keith