Subject: Re: [ietf-smtp] [Emailcore] Proposed ESMTP keyword RCPTLIMIT

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 07:59:09 -0700 (PDT)

The idea of a limits extension has come up several times before, but
for various reasons never went forward.

I think this extension is an important thing to have, so I've put together the
beginnings of a specification:

  https://datatracker.ietf.org/doc/draft-freed-smtp-limits/

(I've tried to incorporate most of the ideas I've seen on the list, but have
probably missed some.)

Since this isn't directly relevant to emailcore, furthr discussion should be
taken to the ietf-smtp list, which I have cc'ed.

				Ned

P.S. One thing that's definitely missing is an acknowldgements section.
Apologies for the omissiom, but I ran out of time. It will be the first thing I
add to -01.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 15 Mar 2021 08:30:02 -0700

>    https://datatracker.ietf.org/doc/draft-freed-smtp-limits/

Thanks for doing this.

Quick comments:

In the Introduction, I suggest adding:

      SMTP servers have always been able to announce a limit, in a 
reply, which means that the client first needed to issue a command.  The 
mechanism specified here avoids the overhead of that interactions, by 
announcing limits prior to any substantive interaction.

Also:

      The Limits mechanism provides an announcement at the start of a 
session.  Some servers vary their limits, as a session progresses, based 
on their obtaining more information.  The Limits mechanism does not 
cover in-session limitation changes.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 15 Mar 2021 14:12:05 -0200

> On Mar 15, 2021, at 12:59 PM, Ned Freed <[email-masked]> wrote:
> 

Thanks.  I noted that parameters for the "LIMITS" keywork are ";" delimited,
that's not the common pattern in EHLO keywords, where e.g. SASL mechanisms,
are space-separated.  It seems unlikely that limits will be anything but
boolean or numeric, and so ";" looks a bit out of place to me.

It feels more like MIME rather than SMTP. :-) :-)

-- 
	Viktor.

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 08:58:06 -0700 (PDT)

> Thanks for doing this.

No sweat.

> announcing limits prior to any substantive interaction.

Nice. Added.

> cover in-session limitation changes.

This was on the to-do list, but I was struggling with the language. This helps.

However, I did have another idea here: It would be possible for a server to
indicate that a limit change has occured through the use of a special enhanced
status code - probably limited to ones on successful repsonses. This would
instruct the client to reissue EHLO at the next opportunity to obtain updated
limits.

I think this is overengineered and opted not to include it, but I'd like
feedback from others on the point.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Mon, 15 Mar 2021 09:29:20 -0700

> feedback from others on the point.

I'll amend your assessment:  I think this would be /extremely/ 
overengineered.

Specifying mechanisms for highly nuanced behavior is worthwhile when 
there is a well-understood problem, widely viewed as being substantial, 
and therefore, there is good motivation for adopting the added complexity.

Most of the time, enhancement mechanisms, like this, already have 
significant adoption barriers.  So keeping them as simple as possible is 
to be highly preferred, over making them more nuanced.

d/

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 09:19:44 -0700 (PDT)

> are space-separated.

(I was wondering if anyone would notice this.)

It may be a common pattern - and in fact it's what I used initially. But then I
checked, and it's actually syntactically invalid according to RFC 5321:

   ehlo-line      = ehlo-keyword *( SP ehlo-param )

   ehlo-keyword   = (ALPHA / DIGIT) *(ALPHA / DIGIT / "-")
                    ; additional syntax of ehlo-params depends on
                    ; ehlo-keyword

   ehlo-param     = 1*(%d33-126)
                    ; any CHAR excluding <SP> and all
                    ; control characters (US-ASCII 0-31 and 127
                    ; inclusive)

I couldn't believe this was how it was originally specified, but sure enough,
it's what RFC 1869 says as well.

I'm now pondering what to do about my XCLIENT implementation...

> It seems unlikely that limits will be anything but

I could imagine a comma-separated value list. Maybe.

> It feels more like MIME rather than SMTP. :-) :-)

It seemed like a reasonable fallback syntax ;-)

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 15 Mar 2021 14:36:18 -0200

> On Mar 15, 2021, at 2:19 PM, Ned Freed <[email-masked]> wrote:
> 

If your XCLIENT is to be compatible with Wietse's then the supported
elements should be SPACE-separated:

	http://www.postfix.org/XCLIENT_README.html#protocol

220 server.example.com ESMTP Postfix
EHLO client.example.com
250-server.example.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-XCLIENT NAME ADDR PROTO HELO
250 8BITMIME

-- 
	Viktor.

---

From: "John Levine" <[email-masked]>
Date: 15 Mar 2021 13:48:28 -0400

>   ehlo-line      = ehlo-keyword *( SP ehlo-param )

I must be missing something.  It looks to me like a keyword followed by a
space separated list of parameters.

Like, say, this:

220 smtp.gmail.com ESMTP e190sm12987109qkd.122 - gsmtp
ehlo pedro
250-smtp.gmail.com at your service, [2001:470:8d03:c0d:503e:8cc:f15b:bf7d]
250-SIZE 35882577
250-8BITMIME
250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
quit

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 11:33:23 -0700 (PDT)

You're right. I missed the implication of the first line of ABNF. Will fix.

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 12:40:51 -0700 (PDT)

I think that's one decisively in the "thumbs down" column. Pity we don't have a
reaction mechanism to indicate that ;-) ;-)

				Ned

> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp

---

From: John C Klensin <[email-masked]>
Date: Mon, 15 Mar 2021 16:28:36 -0400

--On Monday, March 15, 2021 08:58 -0700 Ned Freed
<[email-masked]> wrote:

>>       SMTP servers have always been able to announce a limit,
>>       in a reply, which means that the client first needed to
>> issue a command.  The mechanism specified here avoids the
>> overhead of that interactions, by announcing limits prior to
>> any substantive interaction.
> 
> Nice. Added.

I wonder.  Along with the "first digit" rule, SMTP has been
fairly clear that SMTP clients should not depend on trying to
parse or interpret reply text except for VRFY and EXPN for which
syntax is actually given for replies.  Assuming this is the sort
of announcement you and Dave have in mind, suppose the client
sends
   MAIL FROM:<[email-masked]>
and the server responds
   250 OK.  No more than 20 recipients
would we seriously expect the client to interpret and use that
information?

In addition, if the answer were "yes", what would prevent a
connection-opening response of 
  220 SNAFU SMTP Server, v 666, no more than 25 recipients
allowed

which is no less an announcement and of course avoids having to
wait for any commands to be issued, even EHLO.

     john

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 14:26:51 -0700 (PDT)

> syntax is actually given for replies.

Which establishes the precedent for parsing replies given a signal
to do so.

> Assuming this is the sort

Of course not, but how about:

250 2.5.314159 RCPTMAX=n 

>   220 SNAFU SMTP Server, v 666, no more than 25 recipients allowed

> wait for any commands to be issued, even EHLO.

You can't repeat the announcement so you need the ability to include the
response in EHLO, which is the obvious way to revise the limits after AUTH or
STARTTLS. And EHLO is required in all three cases, so aside from not being
soonest in one case there's no downside I can see.

I guess I could write all these reasons down, but at some point text about
design choices becomes a hindrance to understanding the acutal protocol.
I think all this cross that line, whereas Dave's text did not.

				Ned

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 15 Mar 2021 22:40:13 +0100

Hi, Ned,

> Apologies for the omissiom, but I ran out of time. It will be the first thing I

In chapter 4 there is a copy/paste error: all three paragraphs within 
this chapter refer to:

"Description: RCPTMAX specifies the maximum number of [...]"

This should be:

"Description: MAILMAX specifies the maximum number of [...]" (par. 4.1)

and

"Description: RCPTDOMAINMAX specifies the maximum number of [...]" (par. 
4.3)

which makes me think you wrote par. 4.2 before you wrote the other 
paragraphs :-)

/rolf

---

From: Sam Varshavchik <[email-masked]>
Date: Mon, 15 Mar 2021 17:41:53 -0400

Dave Crocker writes:

> significant adoption barriers.  So keeping them as simple as possible is to  
> be highly preferred, over making them more nuanced.

That's my take too. I am not sure what I'll gain from doing this in the  
first place, not to mention any extra complexity of this sort.

Without explicitly defined limits: I'm just picking a reasonable cap and  
duplicating each message, so their number of recipients is under the cap.  
This happens much earlier in the processing, even before attempting to find  
a willing MX to talk to.

MXes which do not announce their limits are going to dominate for some time.  
So, I still need to do this for the foreseeable time. So, what does this  
SMTP extension buy me? The only thing I can think of: not duplicating  
messages when I don't need to.

But the thing is: I don't recall anyone raising this to me, as a problem.  
This is not on my radar screen as a problem that wants to be solved.

---

From: Ned Freed <[email-masked]>
Date: Mon, 15 Mar 2021 15:00:38 -0700 (PDT)

> which makes me think you wrote par. 4.2 before you wrote the other

Got me on that one!

				Ned

---

From: "John Levine" <[email-masked]>
Date: 15 Mar 2021 19:46:47 -0400

>This is not on my radar screen as a problem that wants to be solved.

I wouldn't think that your users would be very interested, but bulk mailers are desperate for
any hints about how much mail they can stuff at people without getting blocked or rate limited.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Mon, 15 Mar 2021 23:33:15 -0400

--On Monday, March 15, 2021 14:26 -0700 Ned Freed
<[email-masked]> wrote:
 
>> >>       SMTP servers have always been able to announce a
>> >>       limit, in a reply, which means that the client first
>> >>       needed to issue a command.  The mechanism specified
>> >> here avoids the overhead of that interactions, by
>> parse or interpret reply text except for VRFY and EXPN for
>> which syntax is actually given for replies.
> 
> Which establishes the precedent for parsing replies given a
> signal to do so.

Except that I thought (although I can't find it right now) that
5321 called  that out as an explicit exception.   Certainly they
and the forwarding and "try instead" replies are the only ones
for which an exact syntax is discussed.    But I think the key
phrase above is "signal to do so", with which I agree.  See
below.

> Of course not, but how about:

And that gets into a bit of tension between the "first digit"
rule in 5321 and extended codes, perhaps particularly in the
case of MAIL, where the fourth paragraph of Section 3.3 ("This
command tells...") says, apparently very specifically 'If
accepted, the SMTP server returns a "250 OK" reply", not "250
<whatever it feels like> which the client is expected to read
and understand".  But if that needs to be addressed at all, it
should be addressed in 5321bis, not in your draft... as long as
the draft does not increase the confusion.  See below.

>> which is no less an announcement and of course avoids having
>> to wait for any commands to be issued, even EHLO.
 
> You can't repeat the announcement so you need the ability to
> include the response in EHLO, which is the obvious way to
> revise the limits after AUTH or STARTTLS. And EHLO is required
> in all three cases, so aside from not being soonest in one
> case there's no downside I can see.

Good reason, but I think I wasn't clear about the point I was
trying to make, which was not about the design --with which I'm
quite happy-- but with Dave's specific paragraph which, in its
form as above, I think could add to the confusion.  Why?
Because, as soon as you say "SMTP servers have always been able
to announce a limit, in a reply...", it seems to me the
immediate question is 

	'Sure, they can "announce" whatever they like, but how
	would that work in practice? If, for example, the client
	sends a MAIL comment and gets back "250..." or even
	"2...", and moves on to the next command because _any_
	2yz response to the MAIL commend implies "ok, move on to
	the next command"'.  

The server could equally respond to a MAIL command with "250 Is
it not a lovely day?".  That would constitute an announcement of
sorts, but the expectation that it would be acted on would be
the same: in the absence of some spec that goes beyond 5321,
both are really comments, not announcements.   Now, if the
paragraph had said, e.g., "...limit, in a reply with an enhanced
status code,..." I probably would not have batted an eyelash
because, at that point, it would be clear that the server was
asking the client to do something different than "get as far as
250 and then move on".    But...
 
> I guess I could write all these reasons down, but at some
> point text about design choices becomes a hindrance to
> understanding the acutal protocol. I think all this cross that
> line, whereas Dave's text did not.

We are in complete agreement with your first sentence.  All I've
suggesting is that, absent some additional words, Dave's text
could add confusion and become a hindrance.   I am agnostic as
to whether it would be better to add the words (few of them and
perhaps using the example above) or to drop the text.  The key
design choice, IMO, is to have a facility for announcing these
limits that is precisely defined (which you have, IMO, done) and
that a client that is aware of and interested in that particular
extension keyword will know, rather exactly, now to interpret
it.  That is what the extension mechanism is all about.  

To push that explanation a bit further, we could, in principle,
define an extension of, e.g.,

 MANDATORYEXTENDEDCODES
or
 YOUBETTERREADREPLYTEXT

in such a way that the server would advertise it and, if the
client did not include some sort of confirmation as part of the
MAIL command or whatever command followed the EHLO response, the
server might find it perfectly reasonable to respond

  55z Then I don't want to talk with you, go away
   (554 would be close, but not exact)

IMO, that would be a fairly dumb idea, but those are the only
way I can think of to make an "announcement" as part of a reply
reliable.  That ultimately just demonstrates another reason why
your proposal is a better idea, but also shows, IMO, how
slippery the slope gets if you start discussing alternate
strategies... and brings me back to the line you describe above
and about which, again, I agree.

    john

---

From: Claus Assmann <[email-masked]>
Date: Tue, 16 Mar 2021 07:11:39 +0100

> Which establishes the precedent for parsing replies given a signal

draft-santos-smtpgrey has something like that for graylisting:
421 4.7.0 Come back later. retry=123
                           ^^^^^^^^^

-- 
Address is valid for this mailing list only, please do not reply
to it direcly, but to the list.

---

From: Ned Freed <[email-masked]>
Date: Tue, 16 Mar 2021 07:27:50 -0700 (PDT)

> 5321 called  that out as an explicit exception.

Even if it did, unless it also said "no subsequent standard can create similar
exceptions", I don't see why it would matter. RFC 2034 created the the
exception for including enhanced status code on top of RFC 821. And  it's been
pointed out that there's a draft that would create another exception.

> Certainly they

Enhanced status codes provide a pretty powerful signalling mechanism.

> the draft does not increase the confusion.  See below.

I'm afraid that's actually a bug in RFC 5321, irrespective of what
emerges from this discussion: It's a bug because any server
supporting the ENHANCEDSTATUSCODES extension isn't going to send
that response. And that includes a *lot* of servers.

As for the "first digit" rule being in tension, I don't think it is. The first
digit rule doesn't say "never ever attach semantics to the rest of the reply".
Rather, it's about the immediate effect of a reply on the SMTP client's
behavior not being based on anything more than the first digit.

The obvious example of deep analysis of SMTP replies is mailing list
managers, which have to decide whether or not a failure should cause the
address to be removed from the list. This "hard" or "soft" bounce determination
can be quite sophisticated - take a look at Sisimai sometime.

> 250 and then move on".    But...
 
I think this can be addressed by making it clear that a special
syntax would have had to have been defined.

> and about which, again, I agree.

The way RFC 2034 made the appearance of ENHANCEDSTATUSCODES reliable was to
announce it in EHLO. There was a counterproposal at the time, from Dan
Bernstein, to just "sprinkle the codes in replies using a # prefix", but we
went with the EHLO announcement approach. Which admittedly does make this a bit
circular...

Even so, it seems to me that calls to explain design choices over and
over are on the rise. This seems to me like one where a little
text helps more than it hurts.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 17 Mar 2021 11:46:25 +0100

> 421 4.7.0 Come back later. retry=123

That could be a fair idea.  Strawman below, along with a few nits.

Best
Ale

--- draft-freed-smtp-limits-01.md	2021-03-17 11:01:19.392667148 +0100
+++ draft-freed-smtp-limits-02.md	2021-03-17 11:20:30.316669820 +0100
@@ -24,6 +24,7 @@
    ABNF: RFC5234
    SMTP: RFC5321
    PIPELINING: RFC2920
+  GREYLISTING: RFC6647

  informative:
    STARTTLS: RFC3207
@@ -204,9 +205,9 @@

  Name: MAILMAX

-Value syntax: %x30-39 0*5DIGIT ; 0 not allowed, 6 digit maximum
+Value syntax: %x31-39 0*5DIGIT ; 0 not allowed, 6 digit maximum

-Description: RCPTMAX specifies the maximum number of transactions
+Description: MAILMAX specifies the maximum number of transactions
  (MAIL FROM commands) the SMTP will accept in a single session. The
  count includes all MAIL FROM commands, regardless of whether they
  succeed or fail.
@@ -218,7 +219,7 @@

  Name: RCPTMAX

-Value syntax:  %x30-39 0*5DIGIT ; 0 not allowed, 6 digit maximum
+Value syntax:  %x31-39 0*5DIGIT ; 0 not allowed, 6 digit maximum

  Description: RCPTMAX specifies the maximum number of RCPT TO commands
  the SMTP will accept in a single transaction. It is not a limit
@@ -233,9 +234,9 @@

  Name: RCPTDOMAINMAX

-Value syntax:  %x30-39 0*5DIGIT ; 0 not allowed, 6 digit maximum
+Value syntax:  %x31-39 0*5DIGIT ; 0 not allowed, 6 digit maximum

-Description: RCPTMAX specifies the maximum number of different domains
+Description: RCPTDOMAINMAX specifies the maximum number of different domains
  that can appear in a recipient (RCPT TO) address within a single
  SMTP session. This limit is imposed by some servers that bind to
  a specific internal delivery mechanism on receipt of the first
@@ -243,6 +244,20 @@

  Security Considerations: See {{Seccons}}

+GREYLISTING Limit
+-------------
+
+Name: GREYLISTING
+
+Value syntax:  %x31-39 0*5DIGIT ; 0 not allowed, 6 digit maximum
+
+Description: GREYLISTING specifies the minimum number of seconds
+that a client should wait before retrying to submit the same message.
+The presence of this limit implies that the client MAY receive a
+transient 4xx response.  See {{GREYLISTING}}
+
+Security Considerations: See {{Seccons}}
+
  Security Considerations            {#Seccons}
  =======================

---

From: Ned Freed <[email-masked]>
Date: Wed, 17 Mar 2021 10:44:04 -0700 (PDT)

Typos fixed.

> +GREYLISTING Limit

The primary purpose of this document is to create the extension and associated
registry. The only reason some initial entries are included is that it's very
helpful to have some examples to emulate. Building the entire registry is not
the goal, because doing so requires that we reach consensus on every limit
people can think of. This is a sure-fire recipe for document failure.

If past discussions are any indication, getting consensus on how to handle
greylisting is going to be difficult. I don't want to make this document
dependent on that.

That said, I don't see why this limit necessarily has anything to do  with
greylisting. It's about how to long to wait before retying after a transient
failure, irrespective of why the delay occurred. If could be greylisting, it
could be exceeding a rate or connection limit, it could be one of the sources
for validating recipients is down, it could be the spam filtering system is
down, etc. RETRYDELAY would be a much better name for it.

The question then becomes is such an announcement - one that applies to all
transient failures, useful. My sense is that the utility is marginal given the
lengthy list of possible causes and the fact that the best retry period for
different causes could be very different, which I note that the approach taken
in the original proposal allows.

				Ned

---

From: Jeremy Harris <[email-masked]>
Date: Wed, 17 Mar 2021 21:19:43 +0000

>    https://datatracker.ietf.org/doc/draft-freed-smtp-limits/

Thanks for taking this on, Ned.

I've thrown together an implementation, now exposed on the net.
If anyone wants to try interop, please contact me.

-- 
Cheers,
   Jeremy

---

From: John C Klensin <[email-masked]>
Date: Wed, 17 Mar 2021 21:06:21 -0400

--On Wednesday, March 17, 2021 10:44 -0700 Ned Freed
<[email-masked]> wrote:

> Typos fixed.
> 
>> +GREYLISTING Limit
>> +-------------
>> +
>> +Name: GREYLISTING
>> +
>> +Value syntax:  %x31-39 0*5DIGIT ; 0 not allowed, 6 digit
>> maximum +
>> +Description: GREYLISTING specifies the minimum number of
>> seconds +that a client should wait before retrying to submit
>> the same message. +The presence of this limit implies that
>> the client MAY receive a +transient 4xx response.  See
>> {{GREYLISTING}}
> 

FWIW, strong agreement.

And, before we go overboard on this, I would encourage everyone
to review and think about the wise advice [1] in the last two
paragraphs of Section 2.2.1 or RFC 5321.

Personally and wearing no hats, I like the idea of a new
"LIMITS" extension and EHLO keyword.   However, I think we could
easily go overboard and start adding limits parameters on a
principle closer to "might be a neat idea" than to "strong
demonstrated need and evidence of willingness to both advertise
and use".     

If I thought anyone would pay attention, I'd argue for a much
stronger requirement for a new parameter than "Specification
required".  I don't, but I think that, not only should we be
careful that any parameters included in the spec are of clear
applicability and usefulness but that text discouraging vanity
parameters or parameters that would be of value to only one
implementation model would be desirable.  That text might be as
simple as a reference to that statement in 5321 -- perhaps
either at the end of Section 1 or as part of new comments in
Section 6 encouraging people to not go overboard.

best,
   john

[1] While that text is carried forward from RFC 1425, I didn't
write or even suggest it (IIR, Marshall Rose did), so I claim no
credit.

---

From: Sam Varshavchik <[email-masked]>
Date: Wed, 17 Mar 2021 21:37:26 -0400

> "LIMITS" extension and EHLO keyword.   However, I think we could

This reminded me of something that I wanted to mention. In whichever form  
the eventual specification of recipient limits takes place: I believe that  
the specification should set a minimum recipient limit. In other words: the  
minimum number of recipients that can be advertised.

I believe that the generally good track record of historical  
interoperability of SMTP implementations goes back to what's in section  
4.5.3 of RFC 821, that gives the minimal limits of various things, like line  
lengths. And, incidentally, the minimum number of recipients that an SMTP  
server should accept is 100 recipients.

I don't remember if this was discussed, but this should be called out in any  
eventual recipient limit specification: the minimum value is 100, as per  
2821 and 821.

---

From: Keith Moore <[email-masked]>
Date: Wed, 17 Mar 2021 22:39:48 -0400

On 3/17/21 9:37 PM, Sam Varshavchik wrote:

> interoperability of SMTP implementations goes back to what's in 
> section 4.5.3 of RFC 821, that gives the minimal limits of various 
> things, like line lengths. And, incidentally, the minimum number of 
> recipients that an SMTP server should accept is 100 recipients. 

It's been a long time but I'm pretty sure I've seen situations in which 
it made sense for the recipient limit to be 1.   For example: a 
special-purpose device (e.g. email to fax, email to printer) or a 
gateway to a dissimilar mail system, or anything for which it makes 
sense to insist that per-recipient errors get immediately reported to 
the client.

Keith

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 17 Mar 2021 22:54:54 -0400

> On Mar 17, 2021, at 10:39 PM, Keith Moore <[email-masked]> wrote:
> 
> It's been a long time but I'm pretty sure I've seen situations in which it made sense for the recipient limit to be 1.   For example: a special-purpose device (e.g. email to fax, email to printer) or a gateway to a dissimilar mail system, or anything for which it makes sense to insist that per-recipient errors get immediately reported to the client.

That's really the realm of LMTP, final delivery, not relaying.

But much as I agree that MTAs should avoid going below the 5321 limits,
I don't know that we can realistically win in the face of the clout of
the my way or the highway crowd of Gmail, Outlook.com and perhaps still
even Yahoo.

So long as they are willing to reply with 452 as many as (100-N) times
when enforcing a limit of N, things mostly work out provided N is not
so small that one ends up deferring some of the message recipients more
than a couple of times in order to deliver the remaining recipients.

The real question is what to do when you have prepared an envelope of
say 100 recipients, and encounter an MTA with a limit of 5?  Do you
then open 20 parallel connections (they'll hate you for that too),
or do serially transmit the same message over the existing connection
(they may also have an unannounced message per connection limit), or
finally serially open new connections, without deferring, but then
this destination ends up hogging multiple delivery agent cycles out
of turn (this matters when a queue manager is trying to manage
some semblance of fairness).

So limits below the expected limit are damaging to sender performance,
and should not be routinely applied.  The problem is that by the usual
suspects many independently operated smaller MTAs are treated with
suspicion...

-- 
-- 
	Viktor.

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Wed, 17 Mar 2021 23:09:20 -0400

> greylisting is going to be difficult. I don't want to make this document

The proposed text doesn't do anything to make that situation worse.  Pretty
much every MTA administrator that didn't just fall out of a tree already knows
that greylisting happens, and they have some already-decided way of
retrying.  The only thing this changes is that the receiving MTA now has a
way to tell the sending MTA "don't bother retrying for NN minutes".

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Wed, 17 Mar 2021 23:10:52 -0400

(Argh.. didn't complete the thought before hitting send.  -ENOCAFFEINE :)

> down, etc. RETRYDELAY would be a much better name for it.

I agree, that covers pretty much all the bases, not just greylisting..

---

From: John C Klensin <[email-masked]>
Date: Thu, 18 Mar 2021 02:05:06 -0400

--On Wednesday, March 17, 2021 21:37 -0400 Sam Varshavchik
<[email-masked]> wrote:

> This reminded me of something that I wanted to mention. In
> whichever form the eventual specification of recipient limits
> takes place: I believe that the specification should set a
> minimum recipient limit. In other words: the minimum number of
> recipients that can be advertised.
> 
> interoperability of SMTP implementations goes back to what's
> in section 4.5.3 of RFC 821, that gives the minimal limits of
> various things, like line lengths. And, incidentally, the
> minimum number of recipients that an SMTP server should accept
> is 100 recipients.
> 
> I don't remember if this was discussed, but this should be
> called out in any eventual recipient limit specification: the
> minimum value is 100, as per 2821 and 821.

Sam,

In principle, I agree with you.  There are two practical
problems and I don't know quite what to do about either.   

First, given the handwaving that I've called "operational
necessity" (and there will be more about the need to address
that in draft-ietf-emailcore-rfc5321bis-03) 5321 already allows
implementations and operators to pick a smaller number.  So,
strictly speaking, if a server returns 250 replies to the first
dozen recipients and then returns 452 for every RCPT command it
gets until the client either sends DATA (or equivalent) or gives
up (presumably with QUIT or RSET), it is conformant to 5321 even
if you, I, and (I hope) many others have to hold our noses while
saying that.  

In part because of that and given comments from others on this
and the emailcore list, if the specification for RCPTLIMIT says
"you MUST NOT provide a number smaller than 100", I'd expect an
implementation or configuration that intends to impose a smaller
limit would either just not advertise the parameter (or LIMITS
itself) or would ignore that requirement and put in a number it
considers appropriate.  Those are "lose either way" options, at
least AFAICT.

Again, I agree about the interoperability aspects of this.  If
nothing else and regardless of what we call it, I think we are
headed down the path in which practical interoperability
depends, not on the standard(s) but on implementations and
configurations selecting compatible profiles of features and
options -- precisely what many people associated with the
ARPANET and early Internet were complaining about in various ITU
protocols when 821 was new and explaining as one of the
important reasons why X.400 was failing or had failed already a
decade later.  Awareness of that problem was at least one of the
reasons for the 1425/5321 text I cited earlier.  But I don't
think the LIMITS extension makes things worse, only more
explicit, and I have no idea how to turn back the tide.

best,
   john

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 18 Mar 2021 11:37:21 +0100

On Wed 17/Mar/2021 18:44:04 +0100 Ned Freed wrote:
> [...]

Good point.  By observing RCPTLIMIT, a client can avoid re-queuing delays.  It 
stops just before reaching the limit and then starts a new transaction right 
away.  However, this won't work if 452 was issued to split recipients according 
to a different criterion than the local max number.

> transient failures, useful.

Applying RETRYDELAY to all transient failures doesn't seem to me to be the 
right thing to do.  However, distinguishing among transient failures conflicts 
with the dog rule.

> My sense is that the utility is marginal given the lengthy list of possible
> causes and the fact that the best retry period for different causes could be
> very different, which I note that the approach taken in the original
> proposal allows.

In this moment I'm unable to think of varying delay lengths.  Some 
considerations, for example how often a server updates some DCC database or how 
long does it take to reboot, may suggest a retry interval common to all 
transient failures that require a delay.  That is, most 4xx, except splitting 
recipients, where immediate retry is better.  So, two categories may suffice.

How about NORETRYDELAY=452?  (Just kidding...)

Best
Ale
--

---

From: Keith Moore <[email-masked]>
Date: Thu, 18 Mar 2021 07:18:21 -0400

> That's really the realm of LMTP, final delivery, not relaying.

No, it's not.   I didn't say anything about this being used for final 
delivery.   The point is that there are legitimately SMTP servers that 
potentially accept traffic from the public Internet, that aren't 
full-blown MTAs.

Keith

---

From: John C Klensin <[email-masked]>
Date: Thu, 18 Mar 2021 09:58:39 -0400

--On Thursday, March 18, 2021 11:37 +0100 Alessandro Vesely
<[email-masked]> wrote:

>...
>> That said, I don't see why this limit necessarily has
>> anything to do  with greylisting. It's about how to long to
>> wait before retying after a transient failure, irrespective
>> of why the delay occurred. If could be greylisting, it could
>> be exceeding a rate or connection limit, it could be one of
>> the sources for validating recipients is down, it could be
>> the spam filtering system is down, etc. RETRYDELAY would be a
>> much better name for it.

> Good point.  By observing RCPTLIMIT, a client can avoid
> re-queuing delays.  It stops just before reaching the limit
> and then starts a new transaction right away.  However, this
> won't work if 452 was issued to split recipients according to
> a different criterion than the local max number.

"Avoid" and "stops"?  Only maybe.  If some variation on
greylisting is applied because the server doesn't approve of,
e.g., the IP address of the sender or the combination of the
reverse-path and one or more of the recipients, a published
maximum on the number of RCPT commands doesn't have anything to
do with it.   And, for a variety of reasons, that might result
in a 4yz code after DATA, not anything but 250 codes after the
RCTP commands.

I see this feature as useful, but we should not over-sell it,
even to each other.

   john

---

From: John C Klensin <[email-masked]>
Date: Thu, 18 Mar 2021 11:35:01 -0400

--On Tuesday, March 16, 2021 07:27 -0700 Ned Freed
<[email-masked]> wrote:

>> >> >>       limit, in a reply, which means that the client
>> >> >>       first needed to issue a command.  The mechanism
>> >> >>       specified here avoids the overhead of that
>> >> >> interactions, by announcing limits prior to any
>> >> >> substantive interaction.
>> >> > 
>> >> > Nice. Added.
>> > 
>> >> fairly clear that SMTP clients should not depend on trying
>> >> to parse or interpret reply text except for VRFY and EXPN
>> >> for which syntax is actually given for replies.
>> > 
>> Except that I thought (although I can't find it right now)
>> that 5321 called  that out as an explicit exception.
> 
> Even if it did, unless it also said "no subsequent standard
> can create similar exceptions", I don't see why it would
> matter. RFC 2034 created the the exception for including
> enhanced status code on top of RFC 821. And  it's been pointed
> out that there's a draft that would create another exception.

No problem as long as we are clear about what we are doing.
And, if the exceptions _require_ enhanced status codes, I have
no concern at all.

> Enhanced status codes provide a pretty powerful signalling

Agreed.  See above.
 
>> >> of announcement you and Dave have in mind, suppose the
>> >> client sends
>> >> would we seriously expect the client to interpret and use
>> >> that information?
>> > 
>> should be addressed in 5321bis, not in your draft... as long
>> as the draft does not increase the confusion.  See below.

> I'm afraid that's actually a bug in RFC 5321, irrespective of
> what emerges from this discussion: It's a bug because any
> server supporting the ENHANCEDSTATUSCODES extension isn't
> going to send that response. And that includes a *lot* of
> servers.

Understood.  However, there appears to be no requirement in RFC
3463 or anywhere else, including RFC 2034, that a server
advertise ENHANCEDSTATUSCODES before sending them.  3463 (and
1893) seem to imply that such codes can simply be added at the
server's discretion and, fwiw, 3463 does not even reference 2034.

While I think the language about the three-digit code structure
needs work (see prior note to Alessandr on the ietf-smtp list
today), I'm not sure how to address the bug you see in 5321bis
without getting into the "scope" territory that has been under
discussion in conjunction with verification of return paths and
removing the SPF and DKIM references.   Perhaps the right
solution would be to put a section into the A/S discussing
extended codes and, in the process, strongly recommending that
servers that intend to send such codes advertise the extension.
Other suggestions welcome.

> As for the "first digit" rule being in tension, I don't think
> it is. The first digit rule doesn't say "never ever attach
> semantics to the rest of the reply". Rather, it's about the
> immediate effect of a reply on the SMTP client's behavior not
> being based on anything more than the first digit.

Right.  See above about trying to clarify that.

> The obvious example of deep analysis of SMTP replies is
> mailing list managers, which have to decide whether or not a
> failure should cause the address to be removed from the list.
> This "hard" or "soft" bounce determination can be quite
> sophisticated - take a look at Sisimai sometime.

Ack.
 
>...
>> The server could equally respond to a MAIL command with "250
>> Is it not a lovely day?".  That would constitute an
>> announcement of sorts, but the expectation that it would be
>> acted on would be the same: in the absence of some spec that
>> goes beyond 5321, both are really comments, not
>> announcements.   Now, if the paragraph had said, e.g.,
>> "...limit, in a reply with an enhanced status code,..." I
>> probably would not have batted an eyelash because, at that
>> point, it would be clear that the server was asking the
>> client to do something different than "get as far as 250 and
>> then move on".    But...
>  

Yes.  That would work.  Does that text need to be in 5321bis?
Perhaps as part of clarifying the first digit rule and code
structure?

>> > understanding the acutal protocol. I think all this cross
>> > that line, whereas Dave's text did not.
> 
>> We are in complete agreement with your first sentence.  All
>> I've suggesting is that, absent some additional words, Dave's
>> text could add confusion and become a hindrance.   I am
>> agnostic as to whether it would be better to add the words
>> (few of them and perhaps using the example above) or to drop
>> the text.  The key design choice, IMO, is to have a facility
>> for announcing these limits that is precisely defined (which
>> you have, IMO, done) and that a client that is aware of and
>> interested in that particular extension keyword will know,
>> rather exactly, now to interpret it.  That is what the
>> extension mechanism is all about.
>...
> The way RFC 2034 made the appearance of ENHANCEDSTATUSCODES
> reliable was to announce it in EHLO. There was a
> counterproposal at the time, from Dan Bernstein, to just
> "sprinkle the codes in replies using a # prefix", but we went
> with the EHLO announcement approach. Which admittedly does
> make this a bit circular...

Right.   And, again, there is no requirement that the extension
be advertised in order to use the codes.  If a server returns an
extended code for anything but a three-digit code with an
established syntax [1], a client that responded by dumping the
message back on wherever it got it from because it had no idea
what those digits were doing there would be in clear violation
of the intent, and probably the letter, of 5321.

> Even so, it seems to me that calls to explain design choices
> over and over are on the rise. This seems to me like one where
> a little text helps more than it hurts.

Agreed.

   john

[1] That may point to another twitch (or bug) or two in 5321.
Suppose we have a command for which a normal reply has a
specific syntax identified in 5321.  Is a server that stuffs an
enhanced reply code after the normal reply code but before the
specified text in conformance?  Good sense and RFCs 2034 and
3463 certainly suggest a "yes" answer, but that doesn't seem to
be the answer in 5321.  That is further complicated by 5321
apparently not containing any ABNF for those defined/structured
responses.   See, for example, the way in which the VRFY
response is defined in Section 3.5.1.   Anyone who sees these as
issues should request that tickets be opened.  ... And think
about the scope issue mentioned above.

---

From: Michael Peddemors <[email-masked]>
Date: Thu, 18 Mar 2021 10:57:12 -0700

While our servers 'might' accept different limits, and the MTA operator 
is free to set those limits, I will give you one example that goes WAY 
below the 100 limit defined as a minimum.

Incremental Weight systems..

A valid RCPT TO address couple be 1:1

Invalid RCPT TO address treated the same as 20:1, eg so that you can't 
use it for email harvesting as easily.. Send to 5 invalid users, and it 
can have the same weight as another MTA sending 100 valid users.

A particular IP space known for spambots might have a weight of 50:1

Or in reverse, the limit could be lower, say 10 RCPT permitted, except 
for known larger email providers based on the PTR records..

Personally, think the IETF should just stay out of recommending any 
limits, or advertisement of limits, we already have mechanisms via the 
4xx and 5xx to tell the remote MTA what to do, and even a reason why we 
did it, but there is no real 'standard' that is evident out there, so 
why are we (IETF) attempting to set standards..

This should come from the industry, and right now, every MTA admin has 
different ideas on this, depending on their usage scenario.

IMHO

-- 
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.

---

From: Ned Freed <[email-masked]>
Date: Thu, 18 Mar 2021 14:58:38 -0700 (PDT)

> potentially accept traffic from the public Internet, that aren't

All true, but nothing prevents this extension from being used with LMTP.
I will adjust the document accordingly.

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Thu, 18 Mar 2021 18:58:16 -0400

> why are we (IETF) attempting to set standards..

IETF standards are there to define interoperable behaviour.  Barring a
change in the SMTP protocol that deprecates multi-recipient envelopes,
an MTA relaying a message needs to know how many recipients it can
reasonably expect to bundle up into a single delivery.

The standard specifies that servers are expected to handle at least 100,
and that way be likely to avoid interoperability issues, since senders
need to be able to handle servers that support only 100 and no more.

A server that supports fewer than 100 recipients per envelope may fail
to interoperate reliably with conformant sending MTAs.  If the sender
is in fact a source of unwanted junk, lossy email service is feature.
But otherwise, if low limits are applied too broadly, the receiving
MTA may face issues receiving multi-recipient messages in a timely
manner or at all.

So I strongly take issue with "the IETF should just stay out...".

> different ideas on this, depending on their usage scenario.

Industry gets to participate in the IETF process to help define
interoperable specifications.

-- 
    Viktor.

---

From: Michael Peddemors <[email-masked]>
Date: Thu, 18 Mar 2021 16:15:07 -0700

> interoperable specifications.

Not trying to start a flame war, just my position on THIS particular 
issue.. As pointed out, we have interoperability already through the use 
of 4xx and 5xx, and it isn't a case where systems are currently 
advertising a limit of recipient counts, where we now need to set a 
common standard.

If you show me MTA's currently advertising limits on how much they 
accept, then it may be something that needs to be addressed.

This seems to be a case of building a cart before the horse.

I think this is a case of sender's who 'hope' that MTA's advertise this 
maybe?

Either way, it seems like the wrong time/place to put in standards for a 
need that is not yet apparent.

But I bow out of this thread then.. I made my opinion known, which I 
also is part of my right to participate in the conversation.

(But I sure would like to see an opinion from an MTA developer that they 
are looking for a standard to advertise how many recipients they 
accept.. I don't think it is on anyone's radar .. oh, we should start 
advertising this.. )

And most sysadmin's still try to hide what MTA they are even running ;)

I now go back to my corner..

---

From: Jeremy Harris <[email-masked]>
Date: Thu, 18 Mar 2021 23:27:28 +0000

On 18/03/2021 23:15, Michael Peddemors wrote:
> If you show me MTA's currently advertising limits on how much they accept, then it may be something that needs to be addressed.

Apart from people building prototypes to demo this proposal, there are no
such MTAs precisely because there is no standard to do so.

> Either way, it seems like the wrong time/place to put in standards for a need that is not yet apparent.

The need is for a more efficient, in roundtrips, way of communicating limits
than getting individual command responses.
-- 
Cheers,
   Jeremy

---

From: Laura Atkins <[email-masked]>
Date: Fri, 19 Mar 2021 08:55:29 +0000

> On 18 Mar 2021, at 23:27, Jeremy Harris <[email-masked]> wrote:
> 

The MTAs aren’t advertising limits, but the companies running the MTAs do set limits on senders. There’s just no programatic way to advertise them.

Yahoo: limits connections, also limits the number of emails per connection and will just close connections after that limit has been reached. 

Gmail: prefers fewer connections and as much mail shoved through it as possible. 

Microsoft: admits there is a limit to connections, but does not make that public. 

Modern bulk MTAs have built in controls so these limits can be manually configured. They also have AI / ML / automatic processes that will modify sending behavior on the fly. These can be per MX, IP, MX cluster, or whatever. 

These things are happening already (and have been for a decade) they’re just not standardized. 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741		

Email Delivery Blog: https://wordtothewise.com/blog

---

From: George Schlossnagle <[email-masked]>
Date: Fri, 19 Mar 2021 08:14:57 -0400

Hi,

We (MessageSystems/SparkPost, makers of the Momentum and PowerMTA mtas)
already support interpreting smtp responses to infer requested behavior in
regards to the number of connections and message velocity limits. As Laura
notes, this is already established behavior with some large providers.
Today we have to parse the whole response string to comply with different
providers bespoke strings. A set of extended codes to make this
standardized and easier to parse would be fantastic.

George

On Fri, Mar 19, 2021 at 04:55 Laura Atkins <[email-masked]> wrote:

> If you show me MTA's currently advertising limits on how much they accept,
> then it may be something that needs to be addressed.
>
>
> The MTAs aren’t advertising limits, but the companies running the MTAs do
> set limits on senders. There’s just no programatic way to advertise them.
>
> Yahoo: limits connections, also limits the number of emails per connection
> and will just close connections after that limit has been reached.
>
> Modern bulk MTAs have built in controls so these limits can be manually
> configured. They also have AI / ML / automatic processes that will modify
> sending behavior on the fly. These can be per MX, IP, MX cluster, or
> whatever.
>
-- 
*GEORGE SCHLOSSNAGLE*
chief evangelist
www.sparkpost.com
(el) [email-masked]

---

From: Ned Freed <[email-masked]>
Date: Fri, 19 Mar 2021 07:28:14 -0700 (PDT)

> advertising a limit of recipient counts, where we now need to set a

Not really. As has been previously pointed out in the discussions of this
matter, there are quite a few subtleties in implementing proper handling of 4YZ
replies when multiple recipients are involved. Fail to get this right and your
mail doesn't get through, and even if it does it may experience sigfniciant
delays.

> accept, then it may be something that needs to be addressed.

But that's precisely the point: They can't currently do this in an
interoperable way because there's no standard for it.

You seem to be arguing that that a standard in this area necessarily has to
evolve from some sort of widely used nonstandard practice. If so, I strongly
disagreee: While this happens fairly often, it's actually a lousy way to do
standardization work.

> This seems to be a case of building a cart before the horse.

Of course it is. That's how standardization is done. We try to anticipate
need whenever possible. And sometimes we get it right, other times
we get it wrong.

When I first did the pipelining SMTP extension - based on work done earlier by
Marshall Rose -  I faced a high degree of pushback in the IETF from people who
didn't think there was or would ever be any interest in optimizing email
transfer to this extent.

In hindsight, do you think that effort was a waste of everyone's time?

Of course in this case we're coming to the party much later, and this makes the
problem that much harder to solve. And we may end up with something nobody
is willing to implement. You pay your money and you take your chances.

> I think this is a case of sender's who 'hope' that MTA's advertise this

I absolutely plan to implement the server side of this and hope that
clients pay attention so they can optimize their behavior accordingly.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 19 Mar 2021 07:43:37 -0700 (PDT)

> in a 4yz code after DATA, not anything but 250 codes after the

Nothing prevents a server from advertising a limit at the
beginning of session based on the sender's reputation at that
point.

And yes, nothing prevents a limit from subsequently being adjusted during the
session, based on whatever, so that the originally advertised limit no longer
applies. This happens all the time with the SIZE extension on limited-resource
servers; it doesn't mean that SIZE isn't useful.

> I see this feature as useful, but we should not over-sell it,

Agreed. There's already a paragraph in the document saying that it doesn't
attempt to deal with in-session limit changes. I think that's sufficient.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 19 Mar 2021 07:52:57 -0700 (PDT)

> sense to insist that per-recipient errors get immediately reported to

Some do this, others accept any number of RCPT TOs and ignore them all. It
depends on whether the the RCPT TO actually contains any information.

This reminds me that this specification needs to cover message submission. Will
add.

				Ned

---

From: Richard Clayton <[email-masked]>
Date: Sun, 21 Mar 2021 17:05:25 +0000

In message <[email-masked]>, Ned Freed
<[email-masked]> writes

>disagreee: While this happens fairly often, it's actually a lousy way to do

I think that you will find that the very large mailbox providers will
advertise <N> as the maximum value that they will accept from good
senders (since it will cause these good senders to gracefully shut
connections when the limit is reached) ... and they will continue to
refuse to accept as many as <N> from bad senders.

Since good/bad is a judgment by the receiver (and may be a view which is
being dynamically changed by complex ML systems during the email
reception process) the result of the new system is to add more
complexity to sender codebases (because they must cope with the world
just as it is at present, alongside the new scheme).

The cost to the receiving system is likely to be small -- since I cannot
imagine that a receiving system is going to freely advertise any more
than 1 bit of information (accept/reject connection) of its current view
of the good/bad state of the sender at EHLO time.

>need whenever possible. And sometimes we get it right, other times

it's also one more step towards constructing a camel (as the DNS folk
call the issue of ever-increasing complexity in their specifications)

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Hector Santos <[email-masked]>
Date: Sun, 21 Mar 2021 21:16:38 -0400

On 3/17/2021 10:39 PM, Keith Moore wrote:
> It's been a long time but I'm pretty sure I've seen situations in 
> which it made sense for the recipient limit to be 1.�� For example: 
> a special-purpose device (e.g. email to fax, email to printer) or a 
> sense to insist that per-recipient errors get immediately reported 
> to the client.
>

Despite any standard, pseudo or otherwise, the ultimate limit is the 
local receiver/system and the minimum for a protocol complete SMTP 
transaction would be 1.

Our system has no limit out of the box and its system wide (global 
registry value). No current out of the box logic per user.  There 
might be a 3rd party RCPT command override p-code script 
(smtpcmd-rcpt.wcx) that may place a limit.  Can't a typical system 
handle 1000, 10K, 100K+ RCPTs?  How does a big list send mail 1 
million subscribers?   When our MLS is going thru a submission 
distribution, it has a transport to SMTP or create UUCP-ready files 
option.  The former method gets to learn from the SMTP receiver RCPT 
responses where a permanent 5yz result could  unsubscribe the user 
after a number of consecutive different message times.  Any permanent 
negative results with the intention of just being a limit could be 
interpreted as a user does no longer exist.

-- 
Hector Santos,
https://secure.santronics.com
https://twitter.com/hectorsantos

---

From: Laura Atkins <[email-masked]>
Date: Mon, 22 Mar 2021 08:45:05 +0000

> On 22 Mar 2021, at 01:16, Hector Santos <hsantos=[email-masked]> wrote:
> 
>>> I believe that the generally good track record of historical interoperability of SMTP implementations goes back to what's in section 4.5.3 of RFC 821, that gives the minimal limits of various things, like line lengths. And, incidentally, the minimum number of recipients that an SMTP server should accept is 100 recipients. 
>> 
>> It's been a long time but I'm pretty sure I've seen situations in which it made sense for the recipient limit to be 1.�� For example: a special-purpose device (e.g. email to fax, email to printer) or a gateway to a dissimilar mail system, or anything for which it makes sense to insist that per-recipient errors get immediately reported to the client.
>> 
> 
> Despite any standard, pseudo or otherwise, the ultimate limit is the local receiver/system and the minimum for a protocol complete SMTP transaction would be 1.
> 
> Our system has no limit out of the box and its system wide (global registry value). No current out of the box logic per user.  There might be a 3rd party RCPT command override p-code script (smtpcmd-rcpt.wcx) that may place a limit.  Can't a typical system handle 1000, 10K, 100K+ RCPTs?  How does a big list send mail 1 million subscribers?  

Most of the very big senders, the ones sending to hundreds or millions of people at a time, use VERP. Depending on what is known about the domain MXs they open a set of connections and send the mail. Most of the bulk mail is sent with one RCPT TO:. This is true for marketing mail but even more true for transactional mail. 

>  When our MLS is going thru a submission distribution, it has a transport to SMTP or create UUCP-ready files option.  The former method gets to learn from the SMTP receiver RCPT responses where a permanent 5yz result could  unsubscribe the user after a number of consecutive different message times.  Any permanent negative results with the intention of just being a limit could be interpreted as a user does no longer exist.

I may have agreed with you a decade ago. But that’s not true for modern bulk senders. Bounces are classified and sorted. We are long past the days where any permanent negative result is interpreted as a user no longer exists. (https://wordtothewise.com/2019/11/theres-something-about-bounces/) 

laura 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741

---

From: Ned Freed <[email-masked]>
Date: Mon, 22 Mar 2021 08:57:26 -0700 (PDT)

> Most of the very big senders, the ones sending to hundreds or millions of
> people at a time, use VERP.

Often combined with private header fields that carry additional information. Of
course you can't rely on getting the headers back in all cases, but you usually
do.

> Depending on what is known about the domain MXs they open a set of connections and send the mail.

Aka MX rollup. Unfortunately MX rollup runs afoul of things like systems that
only allow the specification of a single domain in a session. Caching
RCPTDOMAINMAX values may help with this, and make use of MX rollup more
automatable.

It's somewhat temping to add some sort of "the MXes pointing at this server
can be rolled up" indicator to the limits specification, but since it's not
a limit it doesn't really fit. If done it needs to be a separate EHLO
announcement.

> Most of the bulk mail is sent with one RCPT TO:. This is true for marketing mail
> but even more true for transactional mail.

There's also per-recipient customization in the mix, both for personalization
and display tracking.

> >  When our MLS is going thru a submission distribution, it has a transport to
> > SMTP or create UUCP-ready files option.  The former method gets to learn from
> > the SMTP receiver RCPT responses where a permanent 5yz result could  unsubscribe
> > the user after a number of consecutive different message times.  Any permanent
> > negative results with the intention of just being a limit could be interpreted
> > as a user does no longer exist.

> I may have agreed with you a decade ago. But that’s not true for modern bulk
> senders. Bounces are classified and sorted. We are long past the days where any
> permanent negative result is interpreted as a user no longer exists.
> (https://wordtothewise.com/2019/11/theres-something-about-bounces/)

Packages like Simimai exist for a reason.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Mon, 22 Mar 2021 14:23:44 -0400

--On Monday, March 22, 2021 08:57 -0700 Ned Freed
<[email-masked]> wrote:

>...
> It's somewhat temping to add some sort of "the MXes pointing
> at this server can be rolled up" indicator to the limits
> specification, but since it's not a limit it doesn't really
> fit. If done it needs to be a separate EHLO announcement.
>...

Ned, Laura,

Just a thought, based on little thinking and less information:
Are there any plausible circumstances in which a system might
want to encourage rolling up a few MXs but not very many?  If
so, that would turn it into a limit, rather than requiring a
separate announcement keyword.  I'm having trouble imagining
such a case, so, if the answer is "no", don't bother with a
detailed explanation.   However, because I think there is value
in minimizing the number of announcements, the question seems
worth asking.

    john

---

From: Hector Santos <[email-masked]>
Date: Tue, 23 Mar 2021 11:23:23 -0400

>> million subscribers?
> Most of the very big senders, the ones sending to hundreds or 
> millions of people at a time, use VERP. Depending on what is known 
> about the domain MXs they open a set of connections and send the 
> mail. Most of the bulk mail is sent with one RCPT TO:. This is true 
> for marketing mail but even more true for transactional mail.

Not sure how VERP applies in the Queuing strategies other than to get 
a better feel for tracking, sorting bounces, i.e. how was the user and 
where should the bounce be directed.

Outbound Queuing Strategies (OQS) can be complex.  Long established 
strategies have been developed, fined tuned with variable retry 
tables, per domains, per reply code, including socket vs smtp error 
codes, etc.  Our OQS is pretty stable.  I leave it alone.  It works. 
Just let the engine run and eventually the mail gets out or bounced. :-)

So in lieu of a pre-established authorization, contract for sending, 
as we know receivers have connection load (balancing) limits.  One 
growing concern is with the growth of hosting domains using the same 
MXs.  We see the same MX behavior across different (trade) business 
domains.  It makes the queuing strategies a bit more complex as the 
sorting is more MX-dependent, not domain independent.  For example, I 
can add a rule that says private domain "MyCustomer.com" should get 
priority over customers using free ESP domains, but it is possible, 
the ESP is hosting the private domains.  I don't know if the ESP has a 
different profiles for the hosted domains, but currently, they appear 
to fall under the same umbrella as their regular free ESP mail service 
for the general public.

I do think that the big free ESP domains are very bias towards 
legitimate mail senders with private customer mailing list who happen 
to have members from these bigger ESP domains.   I am not talking 
about the big 1 million or even 1000 list, just maybe 100 and it can 
cause problems.  They place a big burden on the sender to the point I 
have informed one big domain if they continue to slow down my machines 
delaying deliveries with erroneous reply code & response text, despite 
all things checked off to comply with their requirements, I will add 
new logic where these free ESP domains could be restricted from 
mailing list subscriptions, i.e. a new (silly) option to offer mail 
operators. The big free ESP domains make it far too easy for one user 
to stop other users from getting mail and hurt a business reputation. 
Worst, they make it extremely difficult for support to resolve the 
problem.  This is what puts pressure on the small guys (like myself) 
to just totally block the usage of the free ESP domains.

>
>>  When our MLS is going thru a submission distribution, it has a 
>> transport to SMTP or create UUCP-ready files option.  The former 
>> method gets to learn from the SMTP receiver RCPT responses where a 
>> permanent 5yz result could  unsubscribe the user after a number of 
>> consecutive different message times.  Any permanent negative 
>> results with the intention of just being a limit could be 
> I may have agreed with you a decade ago. But that’s not true for 
> modern bulk senders. Bounces are classified and sorted. We are long 
> past the days where any permanent negative result is interpreted as 
> a user no longer exists. 

Nice graph.  In other words, we need to get more into parsing the TEXT 
responses which is now an optional concept with the extended codes?  
To know the reasons for the bounce?

The problem I found is that most of the time, the reply text are 
erroneous and/or vague and there is no standard here.  Among the 
problem, we are seeing more domains claim "DMARC" support, throwing 
out untrue signals it is a standard and now required for mail 
delivery.  For example, you get a 5yz with extended information with a 
link to some policy and the among the list of reasons, it is DMARC, 
SPF and bulk related stuff, maybe even a PTR requirement. Even with 
all those things met, most of the time, it just happen to be some user 
accidentally marked you as spam. Its not hard to do via some of the 
ESP's primitive UI. Even if unintentionally, its very hard to almost 
impossible to undo.   The problem is that one user will affect all the 
other users getting the same message or same mail from the same 
domain.   That's a problem also when Receiver X uses a set of 
verification methods that differs from Receiver Y.  No consistency in 
receiver expectations.

Anyway, regarding outbound mail strategies, there is the draft 
proposal for "retry=" time hint to help with minimizing retry delays 
and this has worked beautifully among conforming servers and clients.

It is obviously doable with updating SMTP software.  It believe we 
should develop a formal language for automated response text 
processing in order to resolve many of these "advanced" issues, like a 
RCPT limit.   Of course, it has to consider backward compatibility. We 
currently would express a RCPT LIMIT idea with a 4yz response.  I 
believe the RCPT LIMIT proposal attempts to reduce the RCPT commands 
when no longer accepted but there is no harm if the client was 
ignorant of this proposal.

Thanks

---

From: Hector Santos <[email-masked]>
Date: Tue, 23 Mar 2021 11:45:54 -0400

> in minimizing the number of announcements, the question seems

If I may ....

If you are referring to situations similar to where private domains 
are being hosting by a 3rd party vendors or even a free ESP network,  
i.e. private-domain.com hosted by Yahoo's MX or  by Microsoft MX's, 
etc, yes, it has created a delivery problem.  The Outbound Queuing 
Strategies (OQS) need to be adjusted/sorted by MX (and thus IP).  A 
rule that may say: "My-Special-Customer.Com gets sending priority over 
gmail.com" no longer applies when the special customer has decided to 
be hosted by gmail.com.

---

From: Laura Atkins <[email-masked]>
Date: Tue, 23 Mar 2021 18:45:33 +0000

> On 22 Mar 2021, at 18:23, John C Klensin <[email-masked]> wrote:
> 
> 
> 
> want to encourage rolling up a few MXs but not very many?

I can’t think of any off the top of my head. I think you’d either want everything rolled up or nothing. But I’m not handling large incoming volumes, so there may be. 

>  If

I would agree with you here. 

laura 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741