Subject: [ietf-smtp] draft-klensin-rfc5321bis-01

From: John C Klensin <[email-masked]>
Date: Wed, 04 Dec 2019 17:14:49 -0500

Getting the -01 version posted took longer than expected,
partially because an XML issue (no nested comments and no "--"
in comments) got in the way.  No complaints and no one's fault,
but I didn't get it up yesterday as promised.

In any event, it is no up.  As explained Monday, no differences
from -00 other than hiding of the comments associated with the
2821->5321 update and transition, a new change log section to
match, and revision of the "Note on Reading..." at the
beginning.  I've checked diffs and don't think anything was
changed inadvertently but, if I missed something, it is a bug
and I'd appreciate a report.  If you find the full history
useful, read -00.

Again, this is not a proposal, a contribution to a WG, or an
assumption that a WG will be formed (or by what mechanism).  It
is just a contribution to give people an idea of what we are up
against, and what at least some of the outstanding issues are,
to consider if there is a proposal to move forward with a
revision.

best,
   john

---

From: Viktor Dukhovni <[email-masked]>
Date: Sat, 14 Dec 2019 16:06:56 -0500

> On Dec 4, 2019, at 5:14 PM, John C Klensin <[email-masked]> wrote:
> 
> to consider if there is a proposal to move forward [...]

One outstanding defect in 5321 is still present in the bis draft:

  https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#section-4.5.3.1.10

   RFC 821 [8] incorrectly listed the error where an SMTP server
   exhausts its implementation limit on the number of RCPT commands
   ("too many recipients") as having reply code 552.  The correct reply
   code for this condition is 452.  Clients SHOULD treat a 552 code in
   this case as a temporary, rather than permanent, failure so the logic
   below works.

while at the same time:

  https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#appendix-G

   o  Revised the reply code sections to add new 521 and 556 codes,
      clarify relationships, and be explicit about the requirement for
      clients to rely on first digits rather than the sequences in
      Section 4.3.2.
      ...
   If an SMTP server has an implementation limit on the number of RCPT
   commands and this limit is exhausted, it MUST use a response code of
   452 (but the client SHOULD also be prepared for a 552, as noted
   above).

Indeed shortly after 5321 appeared Postfix briefly had code to treat
552 as 452, which caused sufficient breakage for that to be reverted
fairly quickly.  The document needs to be update to remove that mistake,
and the onus for any behaviour changes is on any remaining (likely none)
MTAs that still some decades later return the wrong error code for
"too many recipients".

I don't know whether it is worth mentioning that in the same section
we have:

   When a conforming SMTP server encounters this condition, it has at
   least 100 successful RCPT commands in its recipient buffer.  If the
   server is able to accept the message, then at least these 100
   addresses will be removed from the SMTP client's queue.  When the
   client attempts retransmission of those addresses that received 452
   responses, at least 100 of these will be able to fit in the SMTP
   server's recipient buffer.  Each retransmission attempt that is able
   to deliver anything will be able to dispose of at least 100 of these
   recipients.

Which I heartily endorse, but it sadly does not match today's reality
very well.  The "too big to fail" cloud email providers have taken the
liberty to flout the requirement to accept at least 100 valid recipients
per envelope, and senders have had to hand-tune ad-hoc limits.  I don't
know what, if anything can or should be done here to reconcile theory vs.
practice. :-(

-- 
	Viktor.

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 15 Dec 2019 22:30:17 -0500

[ I don't know whether this the right forum in which I should
  look to find community consensus on what looks like a work-around
  in RFC5321 that seems to have long outlived its use-by date. ]

To whit:

 https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#section-4.5.3.1.10 <https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#section-4.5.3.1.10>

while at the same time:

 https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#appendix-G <https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#appendix-G>

This work-around was briefly trialed in Postfix about ten years back and
quickly abandoned.  It led to substantial volumes of mail clogging the
queue until all ~5 days of retries expire.  The reason being that not
all 552 errors at "RCPT TO" are in fact "too many recipients" errors,
and indeed given that servers are required to send "452", the majority
of these were hard mailbox quota errors or similar.  Treating these
as 452 and retrying was harmful to both sending and receiving systems.

I therefore seek community consensus (is there a WG for this, is this
list sufficient, ...?) to drop the work-around from 5321bis.  Servers
have now had plenty of time (over a decade) to switch from 552 to 442
on too many recipients.

-- 
	Viktor.

---

From: John C Klensin <[email-masked]>
Date: Sun, 15 Dec 2019 23:57:13 -0500

--On Sunday, December 15, 2019 22:30 -0500 Viktor Dukhovni
<[email-masked]> wrote:

>   look to find community consensus on what looks like a
> work-around   in RFC5321 that seems to have long outlived its
> use-by date. ]
> 
> To whit:
> 
>  https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#
> section-4.5.3..1.10
> <https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#
> section-4.5.3.1.10>
> 
>   exhausts its implementation limit on the number of RCPT
> commands   ("too many recipients") as having reply code 552.
> The correct reply   code for this condition is 452.  Clients
> SHOULD treat a 552 code in   this case as a temporary, rather
> than permanent, failure so the logic   below works.
> 
> while at the same time:
> 
> <https://tools.ietf.org/html/draft-klensin-rfc5321bis-01.html#
> appendix-G>
> 
>   o  Revised the reply code sections to add new 521 and 556
> codes,      clarify relationships, and be explicit about the
> requirement for      clients to rely on first digits rather
> than the sequences in      Section 4.3.2.
>      ...
>   If an SMTP server has an implementation limit on the number
> of RCPT   commands and this limit is exhausted, it MUST use a
> response code of   452 (but the client SHOULD also be prepared
> for a 552, as noted   above).
> 
> 
> This work-around was briefly trialed in Postfix about ten
> years back and quickly abandoned.  It led to substantial
> volumes of mail clogging the queue until all ~5 days of
> retries expire.  The reason being that not all 552 errors at
> "RCPT TO" are in fact "too many recipients" errors, and indeed
> given that servers are required to send "452", the majority of
> these were hard mailbox quota errors or similar.  Treating
> these as 452 and retrying was harmful to both sending and
> receiving systems.
> 
> I therefore seek community consensus (is there a WG for this,
> is this list sufficient, ...?) to drop the work-around from
> 5321bis.  Servers have now had plenty of time (over a decade)
> to switch from 552 to 442 on too many recipients.

Noting that I am nearly neutral on this question even though I
am in favor in principle in anything that would remove
requirements for unnecessary workarounds from 5321bis, thereby
making it shorter and less convoluted, I have a question that, I
believe, complements Victor's.  At present, RFC 5321 (and its
predecessors) insist that a server should accept up to 100 RCPT
commands.  I know of several, possibly many, systems that have
been configured, based on operational experience, to use much
lower limits.  Is it appropriate for us to lower the limit in
the 5321bis (and, if so, to what?) and/or to insert text
encouraging a somewhat more flexible approach to that limit?

best,
   john

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 16 Dec 2019 01:28:24 -0500

> On Dec 15, 2019, at 11:57 PM, John C Klensin <[email-masked]> wrote:
> 

Yes, I noted the same issue in the final paragraphs of:

  https://mailarchive.ietf.org/arch/msg/ietf-smtp/AhT_hPXCPA_73vfuj9VYzVZP0Jc

  [...] it sadly does not match today's reality

this an area where practice appears to have diverged from the standard.
I not keen to see the limit reduced, but perhaps advice to clients to
default further below 100 than they may doing already may be warranted.
I can only speak to Postfix, which defaults to sending up to 50 and
accepting 1000.

It would be good to know whether there's any sort of new consensus limit
practiced by the likes of Google, Microsoft, ...  Or whether it is all
now highly variable depending on the client's IP reputation and sender
domain and perhaps even list of attempted recipients.

If there is no longer any predictable lower bound, then it becomes
rather difficult to efficiently send multi-recipient mail.  We end
up devolving to Qmail's always one at a time approach.  [ Perhaps
even when DJB appears to be initially "wrong", he has on multiple
occasions been seen to be eventually right. ]

Mailing list email already tends to use techniques like VERP which
result in one delivery per recipient, so the main source of email
to multiple recipients is ad-hoc lists via Cc and Bcc, and legacy
1-to-many aliases that are not instantiated via a list manager.

-- 
	Viktor.

---

From: Tony Finch <[email-masked]>
Date: Mon, 16 Dec 2019 13:01:00 +0000

> I can only speak to Postfix, which defaults to sending up to 50 and

I would like a requirement that servers must not reject the entire message
if there are up to 100 RCPTs even if the server returns 452 for some of
them. Senders can then pipeline safely.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Fitzroy, Sole: Cyclonic, mainly northerly or northwesterly 5 or 6, backing
southerly 6 or 7 later in west. Very rough, becoming rough or very rough.
Thundery showers. Good, occasionally moderate.

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Dec 2019 08:45:07 -0500

> them. Senders can then pipeline safely.

Part of me wonders why there's a minimum number of RCPTs at all if, 
practically speaking, the client has to be able to deal with a 
server-imposed limit no matter where it is.

Seen in this light, the minimum requirement seems like more of a guide 
for optimization by clients than a requirement for servers.

Then again, if existing clients are currently assuming 100 or some 
substantial fraction thereof, I would not want to break them by telling 
servers that they can now impose whatever limit they like.

I am tempted to suggest that there be a MAXRCPTS EHLO response to let 
the server advertise its explicit limit.  But again, this would give 
license to servers to change existing limits.

Other than the rare server that really cannot store 100 recipients, is 
there some operational reason to permit fewer? Did someone empirically 
determine, or arbitrarily decide, that spambots would be more likely to 
send 100 recipients than legitimate senders?

Keith

---

From: Ned Freed <[email-masked]>
Date: Mon, 16 Dec 2019 07:52:57 -0800 (PST)

> them. Senders can then pipeline safely.

First, even if such a requirement was acceptable - and I think there are cases,
such as an address list including a honeypot address - that make it
unacceptable - I don't see how it could be effectively imposed at this point.
The best you could do is have an SMTP extension that announces this as a
policy.

But a new extension seems like a lot of work, especially since you can always
use BDAT to pipeline safely. So perhaps a better approach would be to make BDAT
"more required", whatever that means.

				Ned

---

From: Tony Finch <[email-masked]>
Date: Mon, 16 Dec 2019 16:39:03 +0000

Ned Freed <[email-masked]> wrote:
>
> > I would like a requirement that servers must not reject the entire
> > message if there are up to 100 RCPTs even if the server returns 452
> > for some of them. Senders can then pipeline safely.
>
> First, even if such a requirement was acceptable - and I think there are
> cases, such as an address list including a honeypot address - that make it
> unacceptable -

Sorry, I should have said "must not reject the whole message just because
of the number of recipients". I don't think this needs any signalling:
it's just to clarify the interoperability criteria for the existing 100
recipient rule.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Southeast Dogger, Fisher, German Bight, Humber: Southwesterly 5 to 7, becoming
variable 4, then cyclonic 5 or 6 later. Moderate or rough. Showers then rain.
Good, becoming moderate, occasionally poor.

---

From: Ned Freed <[email-masked]>
Date: Mon, 16 Dec 2019 12:15:31 -0800 (PST)

> it's just to clarify the interoperability criteria for the existing 100

I misunderstood your original point. To me pipelining refers to transport level
pipelining. You're talking about grouping as many recipients as possible into a
transaction, in which case yes, the specific "too many recipients" error would
need to be disallowed.

Of course that wont prevent people from doing it, but it provides justification
for claiming it's their problem to solve.

That said, there are so many capricious and arbitrary limits out there that
have to be respected, to say nothing of things like near-zero information DSNs
effectively mandating VERP, that it's difficult to know where to start.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Mon, 16 Dec 2019 12:29:36 -0800

On Mon, Dec 16, 2019 at 5:45 AM Keith Moore <[email-masked]>
wrote:

> Seen in this light, the minimum requirement seems like more of a guide for
> optimization by clients than a requirement for servers.
>
> I am tempted to suggest that there be a MAXRCPTS EHLO response to let the
> server advertise its explicit limit.  But again, this would give license to
> servers to change existing limits.
>
The limit that Gmail imposes is not based on a specific number of
recipients.  Instead, it's based on available resources and the in-memory
footprint of the rules necessary for handling each recipient.  GSuite
allows for creating rules that can be made complicated by admins who do
things like try to implement their own anti-spam system, or write tools to
convert from their previous system to ours in less than ideal conversions,
as well as having different rules per recipient... and then claim there is
no way to reduce their rules, regardless of how un-useful they are (see the
other thread about ip-literals in EHLO except writ large).  Couple that
with some less than ideal implementation for the rules system itself
(probably fine for small sets of rules, but very painful for large sets)
and you end up with some worst case scenarios in the 50MB per recipient
range.

There's an open bug to remove that particular resource based temp failures
as we've evolved the system to reduce the problem, including properly
4xx'ing recipients that have different rules (before there were complicated
attempts to merge the results or accept messages and generate bounce
messages later if there were mixed results), so we may go back to accepting
the rfc standard minimums most of the time next year.

That said, multiple addresses per transaction has been a pretty rare
feature.  I'm sure our workload is biased because of how rarely we send
multiple transaction mail ourselves (see also enterprise rules engine on
sending with different rules per recipient), but last I checked the average
per transaction was 1.1.  As we've had to move more computation to be at
smtp time, some of which is per-recipient, including too many recipients in
a transaction increases the number of operations that have to complete
successfully, some of which may be cross oceanic (you think these two
addresses are in the same domain, but they actually work for different
parts of an international enterprise which has different data geoloc
requirements for the recipients in different offices), increasing the
chances of a failure which will temp fail the entire transaction.  Keeping
the number of operations small and contained leads to more predictable
performance and better fault isolation.

Ie, its cheap to say "send this to all these people", but much harder to
know whether every one of those people can accept it.

Brandon

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Dec 2019 16:01:12 -0500

On 12/16/19 3:29 PM, Brandon Long wrote:

>
> On Mon, Dec 16, 2019 at 5:45 AM Keith Moore 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Part of me wonders why there's a minimum number of RCPTs at all
>     if, practically speaking, the client has to be able to deal with a
>     Seen in this light, the minimum requirement seems like more of a
>     guide for optimization by clients than a requirement for servers.
>
>     substantial fraction thereof, I would not want to break them by
>     telling servers that they can now impose whatever limit they like.
>
>     I am tempted to suggest that there be a MAXRCPTS EHLO response to
>     let the server advertise its explicit limit.  But again, this
>     would give license to servers to change existing limits.
>
>     Other than the rare server that really cannot store 100
>     recipients, is there some operational reason to permit fewer?  
>     Did someone empirically determine, or arbitrarily decide, that
>     spambots would be more likely to send 100 recipients than
>     legitimate senders?
>
> recipients.  Instead, it's based on available resources and the 
> in-memory footprint of the rules necessary for handling each 
> recipient.  GSuite allows for creating rules that can be made 
> complicated by admins who do things like try to implement their own 
> anti-spam system, or write tools to convert from their previous system 
> to ours in less than ideal conversions, as well as having different 
> rules per recipient... and then claim there is no way to reduce their 
> rules, regardless of how un-useful they are (see the other thread 
> about ip-literals in EHLO except writ large). Couple that with some 
> less than ideal implementation for the rules system itself (probably 
> fine for small sets of rules, but very painful for large sets) and you 
> end up with some worst case scenarios in the 50MB per recipient range.
>
> There's an open bug to remove that particular resource based temp 
> failures as we've evolved the system to reduce the problem, including 
> properly 4xx'ing recipients that have different rules (before there 
> were complicated attempts to merge the results or accept messages and 
> generate bounce messages later if there were mixed results), so we may 
> go back to accepting the rfc standard minimums most of the time next year.
>
> feature.  I'm sure our workload is biased because of how rarely we 
> send multiple transaction mail ourselves (see also enterprise rules 
> engine on sending with different rules per recipient), but last I 
> checked the average per transaction was 1.1.  As we've had to move 
> more computation to be at smtp time, some of which is per-recipient, 
> including too many recipients in a transaction increases the number of 
> operations that have to complete successfully, some of which may be 
> cross oceanic (you think these two addresses are in the same domain, 
> but they actually work for different parts of an international 
> enterprise which has different data geoloc requirements for the 
> recipients in different offices), increasing the chances of a failure 
> which will temp fail the entire transaction. Keeping the number of 
> operations small and contained leads to more predictable performance 
> and better fault isolation.
>
> Ie, its cheap to say "send this to all these people", but much harder 
> to know whether every one of those people can accept it.
>
> Brandon

Reading this, it appears to me that this is basically a tradeoff between 
reporting per-recipient failures during the SMTP session versus 
reporting (some of) them in via one or more NDNs.   Of course NDNs have 
various costs, not only to generate and transmit them, but also in 
accepting a lower probability of the errors actually being reported to 
some party that can address the issue that caused the delivery failure.

Regarding what the standards prescribe, my sense is that the standards 
should give implementors a lot of leeway, but should be constructed as 
to /maximize the reliability of successful delivery of (legitimate) 
email/.    (Granted that "legitimate" is hard to define but I think it 
has to do with two broad factors - one is whether the content of the 
message is appropriate, and the other is whether the message is 
correctly representing its sender and recipients and signal path; and 
NOT anything to do with things unrelated to message content, say, how 
many recipients were in the envelope or whether an IP address literal 
was used in EHLO.)

So does a hard requirement on how many recipients a server should be 
able to accept, improve or degrade the reliability of email delivery?   
I'm not sure, but I immediately see two consequences of not having a 
minimum requirement:

1.  having a minimum recipient requirement appears to shift some of the 
error reporting to NDNs, with lower reliability for such reporting.

2.  not having a minimum requirement may shift some implementation 
burden (and potential for errors) from servers to clients (by requiring 
clients to deal more flexibly with SMTP error codes), perhaps resulting 
in more errors occurring earlier in the signal path with consequently 
less probability of messages being delivered.   Alternatively, the 
client can behave like qmail and only relay one recipient per envelope. 
   This makes the client code simpler and less error-prone, but it also 
consumes more network resources, client resources, and server resources, 
which might also have an adverse effect on reliability if resources are 
limited at any point on the signal path between client and server.

I suspect that all of these considerations pale in comparison with the 
adverse effect on reliability caused by bogus spam filters (which seems 
to be part of the reason for not supporting a minimum number of 
recipients per envelope).   So I might be inclined to suggest that 
anything that encourages more message filtering via dubious criteria 
should be discouraged by future revisions of the standards and/or by any 
operational recommendations we might someday publish.

Keith

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 16 Dec 2019 22:30:19 -0500

[ Reposting, since the original message turned into a thread about
  recipient count limits.  Perhaps this attempt might elicit some
  discussion on the original question. 

  I don't know whether this the right forum in which I should
  look to find community consensus on what looks like a work-around
  in RFC5321 that seems to have long outlived its use-by date. ]

To whit:

while at the same time:

-- 
	Viktor.

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Dec 2019 22:57:26 -0500

On 12/16/19 10:30 PM, Viktor Dukhovni wrote:

> list sufficient, ....?) to drop the work-around from 5321bis.  Servers

I think it would make sense for 5321bis to

a) restate that a genuine "too many recipients" error (i.e. too many 
RCPT TO commands issued, or perhaps too many RCPT TO commands with 
distinct recipient addresses issued) MUST be reported via a 452 error code.

b) remove the recommendation that 552 in response to RCPT TO be treated 
as a per-recipient temporary error

c) [maybe] state that 552 has in past versions of the SMTP specification 
been used to indicate "too many recipients", but has also unfortunately 
been used to indicate other errors which should not be treated as 
temporary per-recipient errors.   So if a client implementation wishes 
to treat 552 as a per-recipient temporary error, it should perhaps limit 
the number and/or frequency of attempts to resend that message to that 
recipient.

(IMO, even "mailbox quota exceeded" should be treated as a temporary 
error, but the retry frequency should be reduced to something like once 
per day.   Some quota exceeded errors really are brief in nature, but 
some users simply stop reading their mail for an extended period of time.)

Keith

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 16 Dec 2019 23:36:37 -0500

> On Dec 16, 2019, at 10:57 PM, Keith Moore <[email-masked]> wrote:
> 
> a) restate that a genuine "too many recipients" error (i.e. too many RCPT TO commands issued, or perhaps too many RCPT TO commands with distinct recipient addresses issued) MUST be reported via a 452 error code.
> 
> b) remove the recommendation that 552 in response to RCPT TO be treated as a per-recipient temporary error
> 
> c) [maybe] state that 552 has in past versions of the SMTP specification been used to indicate "too many recipients", but has also unfortunately been used to indicate other errors which should not be treated as temporary per-recipient errors.   So if a client implementation wishes to treat 552 as a per-recipient temporary error, it should perhaps limit the number and/or frequency of attempts to resend that message to that recipient.
> 
> (IMO, even "mailbox quota exceeded" should be treated as a temporary error, but the retry frequency should be reduced to something like once per day.   Some quota exceeded errors really are brief in nature, but some users simply stop reading their mail for an extended period of time.)

One comment on the quota issue, I think that whether to return 552 or 452 for
over-quota mailboxes is best left to the *receiving* system, that system has
better knowledge of its policies and/or user habits, rather than second-guessed
by the sending system.  That is, if the receiving system sends 552, then no
retries, and if sends 452 then retries, I don't think the sender has enough
information to make a better choice.

-- 
	Viktor.

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Dec 2019 23:48:40 -0500

On 12/16/19 11:36 PM, Viktor Dukhovni wrote:

> over-quota mailboxes is best left to the*receiving*  system, that system has

Sorry, I miswrote in my previous reply - "too many recipients" should be 
451 of course.

I think you make a good point re: 452 vs. 552.   A server might do well 
to return 452 to mean "mailbox full" for some recipient, and 552 for 
"mailbox full" for another recipient, based on each individual 
recipient's usage history.

Keith

---

From: Tony Finch <[email-masked]>
Date: Tue, 17 Dec 2019 13:50:44 +0000

> transaction, in which case yes, the specific "too many recipients" error would

I did mean transport-level pipelining.

The thing I want to avoid is having the entire transaction rejected
because the server considers it to have too many recipients. If the client
pipelines 100 RCPT commands, and the server incorrectly considers that too
many, then the client will have serious problems getting the message
delivered without operator intervention and/or bug-compatibility
workarounds. Operator intervention might be to disable pipelining for that
destination (assuming the sending software stops trying RCPTs at the first
452) which might be enough to avoid upsetting small-minded servers.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Whitby to Gibraltar Point: Variable 3 or less, becoming south or southeast 3
to 5 later. Smooth or slight. Rain at first in south. Good, occasionally
moderate at first in south.

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 17 Dec 2019 11:04:06 -0500

> 452) which might be enough to avoid upsetting small-minded servers.

It is not clear that "the first 452" is sufficiently unambiguous to
presume 452 for the remaining recipients even in the absence of
pipelining.  [ Perhaps "452 4.5.3 ..." would be sufficiently clear. ]

The most important text on this in the *21 RFCs is the requirement to
wait for QUIT before closing the connection even after some (reasonable
number) of error replies.

[ Note that when recipients are rejected with a 5XX error, the sender
  generally makes progress even if the message transmission tempfails,
  because at least the rejected recipients are now resolved. ]

If at least one recipient has been accepted, the receiving MTA SHOULD
allow the transaction to proceed to DATA (or BDAT), and accept the
message for the valid recipients.

Because pipelining can result in an "overshoot" of the receiver's
recipient limit, Postfix allows (default) 1000 additional recipients
over the configured recipient limit (also default 1000) before the
connection error count is incremented (too many errors can trigger
connection close):

    http://www.postfix.org/postconf.5.html#smtpd_recipient_overshoot_limit

    smtpd_recipient_overshoot_limit (default: 1000)
        The number of recipients that a remote SMTP client can send in
        excess of the limit specified with $smtpd_recipient_limit,
        before the Postfix SMTP server increments the per-session error
        count for each excess recipient.

    http://www.postfix.org/postconf.5.html#smtpd_soft_error_limit
    http://www.postfix.org/postconf.5.html#smtpd_hard_error_limit
    http://www.postfix.org/postconf.5.html#smtpd_error_sleep_time

If the receivers recipient limit is too low, and there is not sufficient
tolerance for "overshoot" then indeed a sender's multi-recipient message
might never make it to any of the recipients.

Furthermore, even with tolerance for overshoot, if the receivers limit
is say just 1 recipient per envelope, and the sender is RFC compliant
and sending 100 (Postfix defaults to 50, to stay on the safe side), then
it will take 100 retries to deliver the message to all recipients.

If there's only one receiving MX host, that means the message would need
to be deferred and retried 100 times.  Given a 5 day retry limit, and a
retry interval of 1 to 2 hours, the message can easily time out.

So I don't believe that removing the floor is a good idea.  In
particular, it is essential for receiving systems that to decide to
support less than 100 recipients, to be willing to tempfail 99
recipients, and still accept the message for the 1st recipient.

On sending side, absent local policy for particular destinations based
on reasonable knowledge of the receiving systems willingness to accept
more than 100 recipients per envelope, sending systems SHOULD NOT
attempt to stuff more than 100 recipients in a single envelope.

-- 
    Viktor.

---

From: "Peter J. Holzer" <[email-masked]>
Date: Thu, 19 Dec 2019 10:46:18 +0100

On 2019-12-16 22:57:26 -0500, Keith Moore wrote:
> a) restate that a genuine "too many recipients" error (i.e. too many RCPT TO
> commands issued, or perhaps too many RCPT TO commands with distinct recipient
> addresses issued) MUST be reported via a 452 error code.

Agree.

> per-recipient temporary error

Agree.

> c) [maybe] state that 552 has in past versions of the SMTP specification been
> used to indicate "too many recipients", but has also unfortunately been used to
> indicate other errors which should not be treated as temporary per-recipient
> errors.

I don't think that the use for other - permanent - errors was
unfortunate. That was correct. What was unfortunate was that an error
code from the wrong category ("5" instead of "4") was used in the first
place. I thought that maybe the author of RFC 821 intended the error to
be permanent, but the "Too Many Recipients Scenario" on page 63 shows
that this is not the case. Here the 552 code is clearly treated as a
transient error. I don't know why. Maybe it was just a typo. I don't
think it matters much at this point.

>   So if a client implementation wishes to treat 552 as a per-recipient
> temporary error, it should perhaps limit the number and/or frequency of
> attempts to resend that message to that recipient.

I would rather state that some servers might still implement the error
from RFC 821 (do we have any data whether such servers exist at all and
how common they are) and that client may choose to accomodate them, but
should not generally treat 552 as a temporary errors. This might be
accomplished for example with a per-destination configuration option or
maybe some heuristics (e.g. based on the number of RCPT commands or the
text of the error message).

> (IMO, even "mailbox quota exceeded" should be treated as a temporary error, but

Yup, but that should be decided by the server, not the client. The
client should not decide that a 5xx return code was really meant to be a
4xx or vice versa.

> the retry frequency should be reduced to something like once per day.

Yes, but again, how can the client know what retry frequency is
appropriate? 

I vaguely remember that there was a draft for an extension where the
server could communicate  a hold time to the client. Something like "try
again in 5 seconds" for too many recpients or "try again in 24 hours"
for a full mailbox.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | [email-masked]         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

---

From: Jeremy Harris <[email-masked]>
Date: Thu, 19 Dec 2019 10:58:04 +0000

> again in 5 seconds" for too many recpients or "try again in 24 hours"

There was https://tools.ietf.org/html/draft-santos-smtpgrey-02
but I don't think it gained traction.
-- 
Cheers,
  Jeremy

---

From: "Peter J. Holzer" <[email-masked]>
Date: Thu, 19 Dec 2019 12:08:11 +0100

> but I don't think it gained traction.

Yes, that was it.

        hp

---

From: Paul Smith <[email-masked]>
Date: Thu, 19 Dec 2019 11:16:21 +0000

> but I don't think it gained traction.

That is actually mildly widely used. There are MTA clients which look 
for hints in a 4xx response to decide when to retry and the 
'retry=hh:mm:ss' text is one of the hints they look for.

It's a trivial-effort thing for a server to implement, and 
not-much-more-effort thing for a client to implement if it can already 
handle different per-message retry delays.

But it was designed to 'optimise' greylisting, so may not be useful for 
this scenario (but it may be - after all, the client can't always work 
out WHY the message was temp rejected).

-- 

Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

---

From: Sam Varshavchik <[email-masked]>
Date: Thu, 19 Dec 2019 07:07:08 -0500

Peter J. Holzer writes:

> should not generally treat 552 as a temporary errors.

I never did, and ~25 years ago it was pretty clear that all 4xx are  
temporary, and all 5xx are permanent. It was always clear to me that the  
remaining two digits are to be ignored, for the purposes of determining how  
to handle the response.

I do not remember anyone ever raising the issue that I should've deferred a  
552 error instead of bouncing it.

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 Dec 2019 11:38:28 -0500

> 452) which might be enough to avoid upsetting small-minded servers.

A 452 would seem to be appropriate per RFC5321 section 4.5.3.1.10

In our Mail List Server's MTA, it handles the codes:

452 does not deactivate user, increment error count.
552 deactivate user

Technically correct I believe, this is probably a candidate for 
RFC5321bis clarification. I may need make them both the same, not 
deactivate the user. It wasn't the user's problem the server ran out 
of space!! lol

But in all cases, the MTA does not quit the session. It continues with 
RCPT TO and eventually DATA.  By default, our server does not 
implement a limit and the MTA is authorized to relay mail to remote 
addresses.  But if there were a server limit, then completing a 
distribution might be lost with either 452 or 552 reply codes.  These 
need to trigger a distribution retry with those marked 452.

Note, this all works well when the List Server creates all the 
separate mail files per target (UUCP/UUCICO mode) and the eyeballing 
SMTP mail server picks them up and sends them out, one by one.   This 
is only a problem when pipeling or list server's MTA is going direct 
to an authorizing SMTP server.

-- 
HLS