Subject: Re: [ietf-smtp] EHLO domain validation requirement in RFC 5321

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 05:49:29 -0400

On 9/28/20 7:35 PM, John R Levine wrote:

>> Actually I expect there will be systems where just 25/<ipv4-address> is
>> statically NAT46’d to a 25/<IPv6-address> and all the outbound 
>> traffic goes
>> back through the NAT64 with no linkage to the IPv4 address except for 
>> those
>> established by the inbound connections.
>
> Hmmn.  I suppose that if I wanted to set up a mail system to maximize 
> the chance that my mail would fall afoul of well established 
> anti-abuse metrics and get thrown on the floor, that'd be a good plan. 

Please cite these "well established anti-abuse metrics" because they 
should not be accepted as valid without question.

If you refuse to do so, it should be assumed that they are either not 
well established or that they are not valid.   Or perhaps, they are 
themselves a form of abuse.

Keith

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 4 Oct 2020 19:13:10 +0200

On Sunday 4 October 2020 11:49:29 CEST, Keith Moore wrote:
> Please cite these "well established anti-abuse metrics" because 
> they should not be accepted as valid without question.

Actually, a significant set of email senders do question and do not accept 
them, so you're in a lot of company here, even if it may not be very good 
company ;)

But there's another aspect. Who accepts? These things are the content of 
.procmailrc files and sieve scripts. Who has authority to accept or not 
accept the validity of what I write in my sieve script? I believe that's a 
local matter, out of scope for the IETF, and therefore off-topic for all 
IETF lists.

Arnt

---

From: Sam Varshavchik <[email-masked]>
Date: Sun, 04 Oct 2020 13:31:18 -0400

Arnt Gulbrandsen writes:

>> Please cite these "well established anti-abuse metrics" because they should  
>> not be accepted as valid without question.
>
> But there's another aspect. Who accepts? These things are the content  
> of .procmailrc files and sieve scripts. Who has authority to accept or not  

Actually, this stuff is frequently checked during SMTP, and an undesirable  
result results in a rejected EHLO, MAIL FROM, RCPT TO, or even DATA. The  
original subject matter of this thread, the EHLO domain validation, is  
rarely done in a post-receipt phase, like in .procmailrc.

But your question is equally valid: who has the authority to accept or not  
accept the validity of how I configure my mail server with respect of the  
criteria it uses to accept or reject public E-mail? And this seems to be on  
topic, since the current document seems to require someone to accept the  
validity of certain parts of SMTP, and prohibit those bits from being used  
to determine that the E-mail is unwanted, and reject it.

---

From: Richard Clayton <[email-masked]>
Date: Sun, 4 Oct 2020 18:52:13 +0100

In message <[email-masked]>,
Arnt Gulbrandsen <[email-masked]> writes
>them, so you're in a lot of company here, even if it may not be very good 

"metrics" means measurement -- it's more heuristics as to how you should
configure and run your system.  About 7 years ago I helped put together
a consensus view of what these heuristics might be for production
systems (ie ones where you really cared whether they worked).

It never got published, not least because the first few pieces of
excellent advice were not seen as something that it was desirable by the
potential publisher to endorse.

That doesn't in my view (and those who helped compile the list) stop
this from being excellent advice -- it's kind of a clue test, if you
cannot obtain an IPv4 address then you probably should not be sending
email at scale. One day that will change, and standards writers may wish
to look into the far future so will not include short-term
considerations in their output. For the next few years however:

*  Use a static IPv4 address for your email system

*  Do not share this IPv4 address with user machines

*  Do not host your email system ‘in the cloud’

*  Make sure that your IP address is not listed in the PBL

*  Provide an MX record

*  Provide meaningful and consistent reverse DNS

*  Your system should say HELO (or EHLO) with its hostname

*  Keep your software completely up-to-date

*  Use a submit port with effective authentication or strict
   IP access controls

*  Limit outgoing email volumes

*  Accept reports of problems with your systems

*  Review the mail system logs on a regular basis

*  Ensure your system is highly reliable

*  Don’t create backscatter

*  Maintain a good reputation

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 14:05:46 -0400

On 10/4/20 1:13 PM, Arnt Gulbrandsen wrote:

> of .procmailrc files and sieve scripts. Who has authority to accept or 
> not accept the validity of what I write in my sieve script? I believe 
> that's a local matter, out of scope for the IETF, and therefore 
> off-topic for all IETF lists. 

The rules that individuals and even enterprises use to filter their own 
incoming email is their own business, and I agree that's out-of-scope 
for IETF.

But that's not what I was talking about and I don't think that's what 
John was talking about either.   He referred to "well established 
anti-abuse metrics".   If they're well established, where are they 
documented?   I don't think it's out of scope for IETF to examine them 
or express opinions about them.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Sun, 4 Oct 2020 11:19:16 -0700

On 10/4/2020 11:05 AM, Keith Moore wrote:
> If they're well established, where are they documented?

In the United States, it is common for people to operates with an 
expectation that going faster than the speed limit is safe, up to 5 or 
10 miles per hour.  In practical terms, this convention is 
well-established.  Go find it documented.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 14:21:11 -0400

Thanks for providing a list, though I wonder if this is the same as the 
list that John referred to.

I do suspect that the list could use some updating.   For example:

On 10/4/20 1:52 PM, Richard Clayton wrote:
> For the next few years however:
>

IMO this should change to support the reality that IPv4 addresses are 
getting scarcer by the day, especially in some parts of the world.  
(Especially given the inertia that likely exists with such rules, 
changing the rules now may be necessary to ensure smooth operation in a 
year or two)

e.g.

- Provide inbound SMTP service at one or more public IPv6 addresses.

- Also, if possible, provide inbound SMTP service at one or more public 
IPv4 addresses.

- Provide at least outbound SMTP relay using public IPv6.

- If possible, relay outbound SMTP mail using a public address of the 
same type as the destination address, rather than through a NAT46 or NAT64.

> *  Do not share this IPv4 address with user machines
Do not share addresses of client or server (inbound or outbound) SMTP 
with user machines.
> *  Do not host your email system ‘in the cloud’
I'm not sure what this actually means or why it's still a bad idea.   
Cloud hosting makes a lot of sense for various reasons.
> *  Make sure that your IP address is not listed in the PBL

I suspect that this is something that sites will have less and less 
control over in the future, at least in IPv4 space, especially given the 
"marketplace" in IPv4 prefixes and the need to have different sites' 
addresses in different IPv4 subnets (also has to do with limitations of 
DNS in-addr.arpa delegation).

> *  Your system should say HELO (or EHLO) with its hostname

Could use better definition of "its hostname".   Suspect you mean EHLO 
name should match PTR lookup of client's source IP address.

IMO that might be a bit limiting - I would really like to see TLS with 
client certs used for relay in the future if the details can be worked 
out, and in that case EHLO name should probably match client cert name.

> *  Keep your software completely up-to-date
up-to-date != minimal bugs.
> *  Use a submit port with effective authentication or strict
Could be reworded, I think.   Ideally a site would not relay any 
unauthenticated mail, no matter how it got into the site's mail system.
> *  Accept reports of problems with your systems
Is there a more recent standard for doing so than postmaster@?
> *  Maintain a good reputation

Anyway, looks like a good start, though I wonder what "well established 
anti-abuse metrics" others might have in mind.

I'd support an effort to update and formalize these.

Keith

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 14:28:31 -0400

> well-established.  Go find it documented.

It's easy to find documentation about how highways are engineered to 
permit safe operation at certain speeds (slightly greater than statutory 
speed limits), also taking into account road conditions, traffic volume, 
etc.   The "expectation" you refer to is not merely an informal 
convention, it's a result of explicit design.

But I'm not your trained monkey, and I'm not bound to jump through 
whatever arbitrary hoops you hold up,so you can Google that for 
yourself.   I suspect that you don't actually care, you're just trying 
to insult me again.

And congratulations, you have once again tried to divert the list away 
from its scope of Internet email issues, for what purpose I can only guess.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Sun, 4 Oct 2020 11:35:20 -0700

> convention, it's a result of explicit design.

Actually, it has little or nothing to do with road engineering. 
Objectively, in fact, anything over about 55 mph is extremely unsafe, 
given typical human reaction behaviors, at the list.

So this is primarily a matter of operating convention by the folk who 
police the roads.  Unwritten but highly reliable.  Except when it isn't.

> yourself.   I suspect that you don't actually care, you're just trying 

Keith, the above is pure ad hominem.  That violates IETF rules.

It would be nice to see those rules enforced.  The failure is also a 
matter of convention that violators rely on.  Quite successfully.

d/

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 14:39:27 -0400

On 10/4/20 2:35 PM, Dave Crocker wrote:

>> permit safe operation at certain speeds (slightly greater than 
>> statutory speed limits), also taking into account road conditions, 
>> traffic volume, etc.   The "expectation" you refer to is not merely 
>> an informal convention, it's a result of explicit design.
>

Whatever.  I found reliable sources that stated otherwise.   But this 
just illustrates that jumping through hoops that you hold up is pointless.

Keith

---

From: "John Levine" <[email-masked]>
Date: 4 Oct 2020 19:24:29 -0000

In article <[email-masked]>,
>list that John referred to.

It's close enough.

>getting scarcer by the day, especially in some parts of the world.  

It's true, if you want a /16, you have to pay a lot of money for it.
On the other hand, you can rent a VPS with a unique public IPv4
address from any of a zillion places for a few dollars a month, so
"scarce" is a matter of perception.

Keeping in mind that the point of a standard is to interoperate, I
think it would be a good idea to offer advice that actually helps, you
know, to interoperate. The reality is that if you send mail from a
static address with matching forward and reverse DNS that doesn't look
generic and EHLO as that DNS name, you'll have a lot more success
getting your mail delivered than otherwise. (That's equally true for
IPv6, of course.) It'd be an a self-inflicited injury for us to tell
people otherwise.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Richard Clayton <[email-masked]>
Date: Sun, 4 Oct 2020 20:46:11 +0100

In message <[email-masked]>,
Keith Moore <[email-masked]> writes

>getting scarcer by the day, especially in some parts of the world.  

you may wish it to change (and I am sure it will in time) ... but a
consensus view (albeit from 2013, but I would expect it was much the
same in 2020) is that you will have much more success delivering email
from a static IPv4 address than from an IPv6 address

>changing the rules now may be necessary to ensure smooth operation in a 

the inertia is I suspect merely in the people whose views go to the
consensus as to what is "a wise way to set up your email"... it may be
that they miss changes, but I doubt that you will do considerably worse
by using IPv4 for some time to come

>DNS in-addr.arpa delegation).

I think you may misunderstand the nature of the PBL ... this is
basically telling you that if you are using IPv4 addresses handed out by
a consumer ISP then you are going to have to ensure that they don't
settle for a quiet life for their abuse@ team by listing all their
assets

>name should match PTR lookup of client's source IP address.

the document I'm quoting from has a paragraph or so of explanatory text
accompanying each of the bullet points -- so although those bullet
points should resonate with everyone here, to make really good use of
the advice you would need the whole thing

>IMO that might be a bit limiting - I would really like to see 

you miss the point -- the list is what you should do for success today.
It is not a manifesto for how the world should be

that said, of course there is value in identifying where success is hard
to achieve and so we should be promoting initiatives to address that

>Is there a more recent standard for doing so than postmaster@?

if you are not reading abuse@ and security@ as well (and paying
attention to email coming in to pretty much any email address in whois
data (for IP or domains) then more fool you

- -- 
richard                                                  Richard Clayton

Those who would give up essential Liberty, to purchase a        Benjamin
little temporary Safety, deserve neither Liberty nor Safety.    Franklin

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 16:17:08 -0400

> from a static IPv4 address than from an IPv6 address

Oh, no doubt.   But I am of the understanding that the situation with 
respect to IPv4 address availability is changing and that the rate of 
change may well accelerate.

(Of course at some point public IPv4 addresses may be quite plentiful 
again because they are no longer very useful.)

The purpose of a standard specification is not to be reactionary, it is 
to specify what is needed for the protocol/service to work well now and 
in the future.   Operational practice does change over time (e.g. web 
traffic is now much more likely to use TLS and/or IPv6 than it was even 
a few years ago.)

So today, the right answer is that you need to support sending from IPv4 
if at all possible, sending from IPv6 is less essential.   But in the 
future IPv6 should be the norm and IPv4 should be phased out.   IMO it's 
valuable for there to be a standard recommendation for how to smoothly 
make that transition in email.   I doubt I have exactly the right words 
(and I didn't really try to nail them down) but I do think it's worth 
discussing in IETF.

> by using IPv4 for some time to come

Why should MTAs be saddled with having to use a legacy and increasingly 
obsolete Internet service?   Why should enterprises in parts of the 
world with less available IPv4 space have to jump through extra hoops to 
exchange email because of obsolete assumptions?   Isn't it worthwhile to 
encourage MSPs to make explicit transition plans to migrate to IPv6?

> settle for a quiet life for their abuse@ team by listing all their

I don't know that I'm specifically familiar with the PBL but I have been 
of the understanding that blackhole lists often operated on "reputation" 
and that the "reputation" of IP addresses were sometimes sullied by past 
usages, thereby penalizing users/sites who were subsequently assigned 
those same addresses.

Can we reasonably expect that there will continue to be a clean 
separation between IPv4 addresses used by consumers and IPv4 addresses 
used by enterprises?

> the advice you would need the whole thing

Ah, good to know.   I'd be pleased to read it if it were made available.

> that said, of course there is value in identifying where success is hard just

A document that says what you should do for success today is indeed 
useful.   IMO a document that outlined a transition (mostly to IPv6 but 
perhaps in other ways also) would also be useful, and that's what I'm 
arguing for.

> data (for IP or domains) then more fool you

ok, fair.

Keith

---

From: "John Levine" <[email-masked]>
Date: 4 Oct 2020 17:46:02 -0400

>Cloud hosting makes a lot of sense for various reasons.

It's a bad neighborhood, since you can expect your neighbors to be
poorly managed botted spam-spewing web servers. It varies by cloud
provider but the median is pretty bad.

>control over in the future, ...

It would likely be helpful to spend a few moments reading about what
the PBL is. "Less and less control" is very unlikely.

>Is there a more recent standard for doing so than postmaster@?

Um, RFC 2142 published 23 years ago.

R's,
John

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 17:57:14 -0400

> provider but the median is pretty bad.

Is it really fair to assess senders based on their "neighborhoods"?    
At what point does this depart from common sense and into the realm of 
pure prejudice?  ("That IP address is from across the tracks, which is a 
bad part of the net.")

And again, what does "in the cloud" actually mean?   Is renting a server 
in a rack at some hosting provider really better than renting a VM, or 
is it necessary to originate mail from your own address block that's 
routed to your enterprise network?

In most respects outsourcing of server provisioning, maintenance, and 
connectivity has become normal, widely accepted, often recommended 
practice.   Why should email be different?

It's hard to escape the impression that a lot of spam filters are based 
on imposing completely arbitrary restrictions on senders, on the belief 
that "good senders" will know which hoops they have to jump through (and 
have sufficient funding to do so) while "bad senders" won't.

> Um, RFC 2142 published 23 years ago.

ok fine.   I was wondering if some sort of other channel for this had 
been established in the last couple of decades, that I hadn't heard of.

Keith

---

From: Sam Varshavchik <[email-masked]>
Date: Sun, 04 Oct 2020 19:09:01 -0400

Keith Moore writes:

> Is it really fair to assess senders based on their "neighborhoods"?    At

Life's not fair.

Whether it's fair, or not, if someone wishes to evaluate an individual IP  
address based on its "neighborhood", a.k.a., the hosting provider, it is  
their prerogative to do so. Their mail server, their rules.

Or, if someone decides to willingly outsource their evaluation to a third  
party provider, it is their prerogative to do so as well.
 
> what point does this depart from common sense and into the realm of pure  
> prejudice?  ("That IP address is from across the tracks, which is a bad part  
> of the net.")

Yes, it is prejudice. So?

I'm prejudiced against Chinanet, China Unicom, Digital Ocean, and a few  
others. All I see from those providers are dictionary attacks, and spam. And  
no response to abuse complaints. So, goodbye. Is it fair to the lessors of  
the IP addresses that do not launch dictionary attacks or spam outbursts?  
Yes, it's unfair. Well, that's life. Sorry. I don't have to the time to keep  
track of bad IPs, on a one by one basis. I have other things that are more  
important on my priority list.

There happens to be some entities who do not like the side of railroad  
tracks that I live in, by the way. Or, they outsource their mail filtering  
to third parties that carry that opinion. I never whined about how unfair it  
is. And it never entered my mind to complain to them as well. I recognized  
that it's their mail server, their rules. They are free to take care of  
their business, and I'm free to take care of mine, in whichever way I see  
fit.

> practice.   Why should email be different?

Who said it should be?

> It's hard to escape the impression that a lot of spam filters are based on  
> imposing completely arbitrary restrictions on senders, on the belief that  
> "good senders" will know which hoops they have to jump through (and have  
> sufficient funding to do so) while "bad senders" won't.

Yes, they may seem to be arbitrary to an outside observer. But, they must  
have merit to whoever is using that spam filter. If it didn't have any  
merit, then they would not be put into place, by definition. And the only  
ones whose opinion counts, with respect to their mail server, is them. If it  
makes sense, to them, to enable EHLO domain verification (dragging this  
subject matter back into the thread kicking and screaming), then they're  
going to do that no matter what verbiage remains in the successor to RFC  
5321. You can take that to the bank.

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 19:39:05 -0400

On 10/4/20 7:09 PM, Sam Varshavchik wrote:

> Whether it's fair, or not, if someone wishes to evaluate an individual 
> IP address based on its "neighborhood", a.k.a., the hosting provider, 
> it is their prerogative to do so. Their mail server, their rules.

As long as they're only handling their own mail, I'd agree.

> Or, if someone decides to willingly outsource their evaluation to a 
> third party provider, it is their prerogative to do so as well.

Sure, though one hopes that the third party operates by published 
criteria that helps the client evaluate that provider's effectiveness 
and sanity.

>> what point does this depart from common sense and into the realm of 
>> pure prejudice?  ("That IP address is from across the tracks, which 
>> is a bad part of the net.")
>

At least you're willing to admit it.

See, I'm fine with penalizing operators that have clearly established 
bad reputations through bad behavior.   I'm not so fine with penalizing 
operators that merely have the misfortune to have "bad IP addresses", or 
send traffic from "bad neighborhoods".

If there's a threshold for responsible behavior such that an operator's 
traffic should be trusted, it seems like that threshold should be widely 
known so that all operators can decide for themselves whether to meet 
that threshold or not.   If, on the other hand, it's completely 
arbitrary, well that's not a recipe for reliable mail service.

> I'm prejudiced against Chinanet, China Unicom, Digital Ocean, and a 
> few others. All I see from those providers are dictionary attacks, and 
> spam. And no response to abuse complaints. So, goodbye. Is it fair to 
> the lessors of the IP addresses that do not launch dictionary attacks 
> or spam outbursts? Yes, it's unfair. Well, that's life. Sorry. I don't 
> have to the time to keep track of bad IPs, on a one by one basis. I 
> have other things that are more important on my priority list.
>
> tracks that I live in, by the way. Or, they outsource their mail 
> filtering to third parties that carry that opinion. I never whined 
> about how unfair it is. And it never entered my mind to complain to 
> them as well. I recognized that it's their mail server, their rules. 
> They are free to take care of their business, and I'm free to take 
> care of mine, in whichever way I see fit.

The purpose of IETF (and therefore this list) is to promote 
interoperability, not to degrade it or shield those who do so.

>> practice.   Why should email be different?

This part of the discussion was about a rule to not host mail servers 
"in the cloud".   To me "cloud" means the kinds of outsourcing I listed 
above.   Or would you define "cloud" differently?

>
>> It's hard to escape the impression that a lot of spam filters are 
>> based on imposing completely arbitrary restrictions on senders, on 
>> the belief that "good senders" will know which hoops they have to 
>> jump through (and have sufficient funding to do so) while "bad 
>> senders" won't.
>
> Yes, they may seem to be arbitrary to an outside observer. But, they 
> must have merit to whoever is using that spam filter. If it didn't 
> have any merit, then they would not be put into place, by definition.

I disagree.  I've seen lots of spam filtered for meritless reasons, I've 
seen lots of examples of operators engaging in completely arbitrary 
behavior.

> And the only ones whose opinion counts, with respect to their mail 
> server, is them. 

Emphatically disagree.   And that path leads to a thoroughly 
dysfunctional mail system, in which senders cannot assure the reliable 
delivery of mail even if it's legitimate content sent from a willing 
sender to a willing recipient.    Which is an accurate description of 
what we have today.

> If it makes sense, to them, to enable EHLO domain verification 
> (dragging this subject matter back into the thread kicking and 
> screaming), then they're going to do that no matter what verbiage 
> remains in the successor to RFC 5321. You can take that to the bank.

Yes, people will defend their own stupidity and irresponsibility until 
their deathbeds.     That doesn't mean that IETF should support it.

Keith

---

From: Sam Varshavchik <[email-masked]>
Date: Sun, 04 Oct 2020 20:34:24 -0400

> As long as they're only handling their own mail, I'd agree.

It's unclear to me how they could end up handling something that's not their  
own mail. Doing so would, at least, involve hijacking DNS records to repoint  
someone else's mail to their own mail server. But that would fall waaaaaay  
out of scope, here.

> Sure, though one hopes that the third party operates by published criteria  
> that helps the client evaluate that provider's effectiveness and sanity.

That's neither here, nor there. One can publish anything. Whether they abide  
by what they publish, in practice, it's a different story. But everyone  
should make their own decisions, for themselves.

To my knowledge, none of the widestly used, and most popular third party E- 
mail reputation services have any kind of authoritative, published manifesto  
about how they go about doing what they do. Anything beyond very, very  
general and non-specific description of what they're all about, and what  
their lofty goals are. Some are more specific than others, but nobody will  
actually state, for the record, exactly how they get from point A to point B.

And that has always been the way as far as I can remember.

And they are still used.

And this has been a frequent criticism from their detractors, and I suspect  
that you're using this rigorous standard as an indirect way of attempting to  
discredit all third party E-mail reputation services. Well, be it as it may.  
But this does not discredit them in my eyes. But that's only because I think  
most of them are already discredited, for other unrelated reasons. I stopped  
using all of them, some time ago, after I concluded that things have changed  
sufficiently, over the years, for them to be worth anything, any more.

But I certainly have no objection to anyone else still using them. It's a  
free world. Who am I to tell someone who they should or should not accept  
mail from, and why?

> See, I'm fine with penalizing operators that have clearly established bad  
> reputations through bad behavior.   I'm not so fine with penalizing  

You are entitled to your opinion. But you need to understand that, insofar  
as the operators of those mail servers go, they don't really value your  
opinion that much. Most of them don't even know who you are. Or who I am.  
They only see what's incoming on their port 25, and they will make their  
decision accordingly, and they won't really care whether someone else across  
the world thinks of it.

> interoperability, not to degrade it or shield those who do so.

And specifying how EHLO/HELO, MAIL FROM, RCPT TO, and DATA works, on a  
purely technical level – this can't be any more interoperable than it  
already is.

Whether or not someone accepts mail from someone else is a matter of policy,  
not interoperability.

Interoperability is promoted by having a rigid, unambiguous, technical  
specification for something. That spells everything out in detail. SMTP is  
much more interoperable than other mail protocols that better be left  
unsaid, which had poor specs from the beginning that created many  
interoperability headaches, for decades. As far as that goes, I think that  
SMTP is remarkably interoperable.

But I don't think there's anything technical in requiring any particular  
validation or non-validation criteria, or a policy, for the sender's IP  
address, or a domain, that falls outside of SMTP's strict boundaries. That's  
a matter of policy, and not a technical specification, and bears no  
relevance on interoperability.

>> merit, then they would not be put into place, by definition.
>

You concluded that it was meritless to yourself. You did not make that  
conclusion for anyone else.

> seen lots of examples of operators engaging in completely arbitrary behavior.

Arbitrary in your eyes. In their eyes, it's unclear how arbitrary or not  
arbitrary it was. You cannot know that without asking them and getting their  
answer. They may have perfectly legitimate reason for engaging in the  
behavior you deemed to be arbitrary. You don't know that.

>> And the only ones whose opinion counts, with respect to their mail server,  
>> is them.
>
> Emphatically disagree.

This has been alluded to before, briefly. Back in the 1990s, there was a  
rather …vocal group who proferred the notion that mail server administrators  
have no standing to control E-mail sent or received by their users. That  
their users had some kind of a right to receive all E-mail unfettered and  
that mail administrators have no right, of some sorts, to block it for any  
reason.

Let's just say that the group of individuals I'm referring to – and some on  
this list know exactly who they are – were not saying much different from  
what you are saying here. … and at about this point I started describing how  
well their ideas were received, what everyone thought of them and how  
everyone else assessed their … faculties, but when I reread it I realized  
that this came off too negative, so I decided to delete all of that, and  
just get to the bottom line:

No matter how much someone asserts to the contrary, the administrators of  
mail servers will always have complete, and have the only say, as to their  
mail servers' administrative policies. And there's nothing that anyone will  
be able to do about it. No matter how much anyone else, you or anyone else,  
thinks of the merits of their work. Sorry. Write any requirement into the  
next SMTP specification, that attempts to dictate policy. It won't work.

> And that path leads to a thoroughly dysfunctional  
> mail system, in which senders cannot assure the reliable delivery of mail

This has never been the case, sorry.

E-mail, as it stands, has never been a reliable, guaranteed message delivery  
medium. At most, E-mail has always been on a best-effort basis.

>> If it makes sense, to them, to enable EHLO domain verification (dragging  
>> this subject matter back into the thread kicking and screaming), then  
>> they're going to do that no matter what verbiage remains in the successor to  
>> RFC 5321. You can take that to the bank.
>
>
> Yes, people will defend their own stupidity and irresponsibility until their  
> deathbeds.     That doesn't mean that IETF should support it.

Well, I'm somewhat skeptical that they're looking for IETF's support on  
anything. And I don't really know what "IETF should support it" means in  
this context. Is IETF about setting everyone's mail acceptance mail  
policies, by decree?

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 21:23:06 -0400

On 10/4/20 8:34 PM, Sam Varshavchik wrote:

>>> Whether it's fair, or not, if someone wishes to evaluate an 
>>> individual IP address based on its "neighborhood", a.k.a., the 
>>> hosting provider, it is their prerogative to do so. Their mail 
>>> server, their rules.
>>
> It's unclear to me how they could end up handling something that's not 
> their own mail.

That's what MSPs do - they handle mail on others' behalf.

Let me use an analogy.  If I buy Internet service from some ISP, I 
expect them to deliver the IP packets that I originate to their 
destinations, and I expect them to deliver IP packets from other sources 
to my network.   What I don't expect is that the ISP will randomly drop 
or mangle IP packets based on arbitrary criteria. The IP stack 
(including everything above IP) expects that networks will make a best 
effort to deliver packets and largely depends on that behavior.

> That's neither here, nor there. One can publish anything. Whether they 
> abide by what they publish, in practice, it's a different story. But 
> everyone should make their own decisions, for themselves.

That's not a way to get reliability or interoperability.

>
> To my knowledge, none of the widestly used, and most popular third 
> party E-mail reputation services have any kind of authoritative, 
> published manifesto about how they go about doing what they do. 
> Anything beyond very, very general and non-specific description of 
> what they're all about, and what their lofty goals are. Some are more 
> specific than others, but nobody will actually state, for the record, 
> exactly how they get from point A to point B.
>

And this is precisely why email is now unreliable.

It used to be that IETF was about people trying to make the network and 
networked applications work well for everyone.   But somehow this list 
is now populated by at least a few people who are loudly defending their 
"right" to screw it up by whatever means they choose, for whatever 
reasons they choose, without having to answer to anyone for doing so.    
Assuming you have some good intention behind your arguments (and I, 
perhaps naively, expect that you really do), it's hard to see what it is.

> You are entitled to your opinion. But you need to understand that, 
> insofar as the operators of those mail servers go, they don't really 
> value your opinion that much. 

Well, I happen to think that email, as an interpersonal messaging 
service, is worth preserving.   Maybe they don't.

> purely technical level – this can't be any more interoperable than it 
Email isn't just a protocol, it is a service.   You can conform to the 
SMTP specification to the letter and the email service still be unreliable.
>
> Whether or not someone accepts mail from someone else is a matter of 
> policy, not interoperability.

You're not even arguing for policy.   You're arguing for completely 
arbitrary behavior.

> specification for something. That spells everything out in detail. 
> SMTP is much more interoperable than other mail protocols that better 
> be left unsaid, which had poor specs from the beginning that created 
> many interoperability headaches, for decades. As far as that goes, I 
> think that SMTP is remarkably interoperable.
>
> But I don't think there's anything technical in requiring any 
> particular validation or non-validation criteria, or a policy, for the 
> sender's IP address, or a domain, that falls outside of SMTP's strict 
> boundaries. That's a matter of policy, and not a technical 
> specification, and bears no relevance on interoperability.

Emphatically disagree.   If users can't trust that reasonable mail that 
they send to correct addresses of willing recipients will be delivered, 
email doesn't interoperate by any reasonable definition.

> conclusion for anyone else.

Well, you're so busy trying to define "merit"  in a circular and 
meaningless fashion, that I don't think I could ever convince you anyway.

> Arbitrary in your eyes.

Arbitrary in users' eyes, and users are generally powerless to do 
anything about it other than maintain multiple email accounts and try 
them randomly until something works or they give up.

> This has been alluded to before, briefly. Back in the 1990s, there was 
> a rather …vocal group who proferred the notion that mail server 
> administrators have no standing to control E-mail sent or received by 
> their users. That their users had some kind of a right to receive all 
> E-mail unfettered and that mail administrators have no right, of some 
> sorts, to block it for any reason.

Some of those people were *right*, even if they didn't make their 
arguments with perfect precision.     They were arguing for a *reliable 
email network* as viewed by users.

What if telephone calls were arbitrarily blocked by the telephone 
network for no apparent reason, maybe because the caller was from a "bad 
neighborhood"?   Would people keep relying on the telephone network, or 
would they just give up?   (Sure, recipients often ignore calls for 
which the calling number isn't known to them, but that's not the network 
blocking the calls)

I do remember some of the anti-spam discussions in the 1990s and how 
chaotic and hopeless they were.   There was a tremendous amount of 
naivete among most of the participants, and a lot of belligerence among 
some, especially those trying to make a "land grab" for part of the 
space whether by patent filings or other means.    I basically bailed, 
not having the time to keep up with them, hoping that the discussions 
would get saner later.

But today is not the 1990s, and conditions are somewhat different now 
than they were then, in ways that might be useful if people actually 
want to create a reliable email service.    Of course, maybe some people 
today still stand to benefit from perpetuating the chaos that currently 
exists.

>
> No matter how much someone asserts to the contrary, the administrators 
> of mail servers will always have complete, and have the only say, as 
> to their mail servers' administrative policies. And there's nothing 
> that anyone will be able to do about it. No matter how much anyone 
> else, you or anyone else, thinks of the merits of their work. Sorry. 
> Write any requirement into the next SMTP specification, that attempts 
> to dictate policy. It won't work.

Well maybe there are some subset of MSPs that actually want to 
contribute to making email more reliable, and maybe those MSPs are 
willing to try to build consensus around policies to that end. As for 
the ones trying to sabotage mail for whatever reasons, I don't think 
IETF can do anything about them, but we can always hope that the ones 
that care about reliable service replace them.

So maybe there should be an RFC "How To Make Your Email Work Reliably If 
That's Actually What You Want To Do".   Because I think Internet email, 
at least post-821/822, is still far better at its core than every other 
messaging system out there.   Better metadata, better searching, better 
archiving, more flexibility. What has happened to Internet email is 
nothing short of tragic, and I would like to see it fixed rather than 
left to rot.

>
>> And that path leads to a thoroughly dysfunctional mail system, in 
>> which senders cannot assure the reliable delivery of mail
>
> E-mail, as it stands, has never been a reliable, guaranteed message 
> delivery medium. At most, E-mail has always been on a best-effort basis.

Best-effort worked fairly well back when the biggest reasons for failure 
were bad MTA configuration (sendmail.cf files were especially 
notorious), bad DNS configuration, and persistent failure of the IP network.

Today's email is nowhere nearly a best-effort service.

>> Yes, people will defend their own stupidity and irresponsibility 
>> until their deathbeds.     That doesn't mean that IETF should support 
>> it.
>
> Well, I'm somewhat skeptical that they're looking for IETF's support 
> on anything. And I don't really know what "IETF should support it" 
> means in this context. Is IETF about setting everyone's mail 
> acceptance mail policies, by decree?

No, IETF is about trying to make recommendations that will facilitate 
and improve service for users.   Which seems like the opposite of what 
you're arguing for.

Keith

---

From: Sam Varshavchik <[email-masked]>
Date: Sun, 04 Oct 2020 21:50:52 -0400

Keith Moore writes:

> « HTML content follows »

Then, it's their mail that they're handling, not someone else's.

> Let me use an analogy.  If I buy Internet service from some ISP, I expect  
> them to deliver the IP packets that I originate to their destinations, and I  
> expect them to deliver IP packets from other sources to my network.   What I  
> don't expect is that the ISP will randomly drop or mangle IP packets based on  
> arbitrary criteria.   The IP stack (including everything above IP) expects  
> that networks will make a best effort to deliver packets and largely depends  
> on that behavior.

If you set up a server on port 25, in order to receive mail, and your ISP is  
blocking your port 25, then you would certainly have an issue to raise with  
them.

> That's not a way to get reliability or interoperability.

Then don't use them, yourself.

But if others choose to use them, you are in no position to judge whether  
their email is reliable or interoperable. On they can make that decision for  
themselves.

> networked applications work well for everyone.   But somehow this list is now  
> populated by at least a few people who are loudly defending their "right" to  
> screw it up by whatever means they choose, for whatever reasons they choose,

I will certainly insist that I have full and complete right to "screw" up my  
own email in whichever way I wish.

>> opinion that much.
> Well, I happen to think that email, as an interpersonal messaging service, is  
> worth preserving.   Maybe they don't.

Email has always been like that. It is no worse off now than it always was.

> Email isn't just a protocol, it is a service.   You can conform to the SMTP  
> specification to the letter and the email service still be unreliable.

Yes, and it will always be the case, with email in its current form.

> You're not even arguing for policy.   You're arguing for completely arbitrary  
> behavior.

I am arguing that individual mail server operators are free to set whatever  
policies they wish, for their mail servers. It is their arbitrary right to  
do so and nobody has any standing to make them do otherwise.

>> address, or a domain, that falls outside of SMTP's strict boundaries.  
>> That's a matter of policy, and not a technical specification, and bears no  
> Emphatically disagree.   If users can't trust that reasonable mail that they  
> send to correct addresses of willing recipients will be delivered, email  
> doesn't interoperate by any reasonable definition.

And if the same users want a pony, they won't get that either, I'm afraid.

How is that's interoperable? "Interoperable" means that the sender and the  
recipient can communicate. In this case, they communicated perfectly, and  
the recipient simply rejected the sender's E-mail.

Interoperable would mean that the recipient intended to receive E-mail, but  
due to an error in interoperability the message failed to get delivered.  
Interoperable does not mean guaranteed E-mail delivery. Sorry, it does not.

The notion that, as a mail sender, one is entitled to have their mail  
received by their intended recipient, and a failure to do so means that E- 
mail is not interoperable, is quite flawed. One has never had that guarantee  
in the first place, and never will. Not under the current, /interoperable/ E- 
mail protocol. It's simply not designed for that, and never will.

> Well, you're so busy trying to define "merit"  in a circular and meaningless  
> fashion, that I don't think I could ever convince you anyway.

No, you defined "meritless" as a conclusion that you reached. I pointed it  
out. And I precisely staked my position that neither myself, nor you, nor  
anyone else, can judge the merits of someone's decision for their incoming  
mail.

> Arbitrary in users' eyes,

and in anyone other than the operators', and it's only their opinion that  
matters, sorry.

>> rather …vocal group who proferred the notion that mail server  
>> administrators have no standing to control E-mail sent or received by their  
>> users. That their users had some kind of a right to receive all E-mail  
>> unfettered and that mail administrators have no right, of some sorts, to  
>> block it for any reason.
>
> Some of those people were right, even if they didn't make their arguments  
> with perfect precision.     They were arguing for a reliable email network as  
> viewed by users.

Yeah, some people just want a pony. They're not getting one.

> What if telephone calls were arbitrarily blocked by the telephone network for  
> no apparent reason, maybe because the caller was from a "bad neighborhood"?  

Telephone is not E-mail.
 
> But today is not the 1990s, and conditions are somewhat different now than  
> they were then, in ways that might be useful if people actually want to  
> create a reliable email service.    Of course, maybe some people today still  
> stand to benefit from perpetuating the chaos that currently exists.

SMTP is still the same today as it was back then, and it works the same way.

> Well maybe there are some subset of MSPs that actually want to contribute to  
> making email more reliable, and maybe those MSPs are willing to try to build  
> consensus around policies to that end.   As for the ones trying to sabotage  
> mail for whatever reasons, I don't think IETF can do anything about them, but  
> we can always hope that the ones that care about reliable service replace  
> them.

That's neither here, nor there. As I said, any policy diktats will fail.

> Best-effort worked fairly well back when the biggest reasons for failure were  
> bad MTA configuration (sendmail.cf files were especially notorious), bad DNS  
> configuration, and persistent failure of the IP network.
>

Really? That's news to me. Who exactly is foolish enough to guarantee mail  
delivery, pray tell me?

> No, IETF is about trying to make recommendations that will facilitate and  
> improve service for users.   Which seems like the opposite of what you're  
> arguing for.

I'm not arguing anything. I'm simply pointing out that attempting to dictate  
policy decisions in technical specifications is a fools' errand. That's the  
capsule summary of my initial message that started this thread. And I'm also  
disputing any notion that E-mail has been, at any time, or will ever be, a  
guaranteed communication medium. Your users can demand their E-mail to be  
delivered as much as they want, but it's not going to be up to them, and it  
is not going to be up to you. If someone doesn't want to receive mail from  
you, there's nothing you can do about it. Call it lack of interoperability,  
call it anything you want, but you're not going to mandate anyone to receive  
email from anyone else. This is simply not going to happen. You can either  
continue to make demands, that will be fruitless, or work with making  
everyone /aware/ that E-mail is what it is, so it is not a surprise to  
anyone.

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 22:09:40 -0400

Ok, basically it's become clear that your intention is a DoS attack 
either on this list or on Internet email.

---

From: Sam Varshavchik <[email-masked]>
Date: Sun, 04 Oct 2020 22:29:34 -0400

Keith Moore writes:

> Ok, basically it's become clear that your intention is a DoS attack either  
> on this list or on Internet email.

I don't know what you're referring to. Even if I wanted to, I have no means  
to launch a DoS attack against anyone.

You're vastly overestimating my mere mortal capabilities.

---

From: Keith Moore <[email-masked]>
Date: Sun, 4 Oct 2020 22:49:54 -0400

> You're vastly overestimating my mere mortal capabilities.

Well that, at least, is good news.

Keith

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 5 Oct 2020 12:53:28 +0200

>> Um, RFC 2142 published 23 years ago.

I'm surprised John didn't mention RFCs 7480-7484.  Nowadays you can get the 
email address of an ISP's abuse team by just feeding a simple command line 
program with the culprit IP address.

No need to tentative postmaster or abuse addresses at some guessed domain names 
or network abuse clearinghouses.

Oh, well, some ISPs have no abuse team or don't publish its email address. 
Whether LIRs should mandate responsive abuse teams is yet another 
purists-vs-realists quibbling subject.

Best
Ale
--