Subject: Re: new issue: non-2606 domains
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Tue, 1 May 2007 08:49:19 +0200
List-Id: <ietf-smtp.imc.org>

[email-masked] writes:
> example.edu is not covered by RFC 2606.  It is however assigned to IANA.

I saw that, and assumed it was overlooked.

Arnt

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap Issues 0b/21/25
Date:  Tue, 01 May 2007 05:12:08 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 69
Sender: [email-masked]

SM wrote:

> We should also consider the legacy systems out there. A complete
> reversal to reject, no bounces, makes the model unreliable.

+1  We're not going to copy the SpamCop FAQ wholesale into 2821bis.
( Nevertheless I've invited the SpamCop newsgroup readers to chime
  in, some have rather radical ideas compared with what we say. :-)

> If we push too strongly for a "no bounce" approach, we'll be
> creating blackholes.

Yes, there's a similar situation in NetNews, if you go too far in
attempts to avoid "dupes" (articles arriving more than once) you
get "nopes" (articles arriving less than once), and for obvious
reasons I always considered "nopes" as worse than "dupes".

> Sometimes the "border" is not clear-cut, i.e. the mail may go
> through several hops before reaching a MTA which can decide
> whether delivery will be successful or not.

The border is defined by query=mx, and among the mailers acting
as MX the first getting mail from another mailer is "the border".

That's what I think how "border" could be defined, ignoring some
corner cases like "no MX" or all definitions of "local mail" if
it touches an MX.

> I read:
>     "[add REFERENCES here]
[...]
> as a push towards SPF and CBV.

I think CBV isn't specified in any RFC, and so we can't add a
reference to it.  Similarly there's no RFC for grey-listing,
SRS, SES, CSV, or BATV.  Referencing RFC 4409 chapter 6.1 should
be okay.  For 4408 I'm not "neutral" (4409-1=4408 is nice :-)
Maybe one example 4409 6.1 is enough.

> Quoting two paragraphs from Section 6.2.

> "delivered.  Reliably determining that a return address is invalid can
>  be a difficult and time-consuming process, especially if the putative
>  sending system is not directly accessible or doesn't fully and
>  accurately support VRFY and, even if a "drop messages with invalid
>  return addresses" policy is adopted, it SHOULD be applied only when
>  there is near-certainty that the return addresses are, in fact,
>  invalid."

That wording doesn't consider that this determination can be also
as simple as one DNS lookup (or rather between 1 and 11 lookups).
I'm not yet happy with the statement before your quote:

   address, although the history of the network is that users are
   typically better served by delivering any message that can be
   delivered.  Reliably determining that a return address is invalid can

This IS history.  It should say "the history of the network was
that users were typically better served by delivering any message
that could be delivered."  Past tense.

While I wrote that I've started my REXX excuse for procmail, and
it will now delete the next about 950 of 1200 pending mails that
arrived in the last hour.  Today users are typically better served
by rejecting suspicious messages at the border, otherwise they end
up in obscure blackholes.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 34:  non-2606 domains
Date:  Tue, 01 May 2007 04:16:31 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 7
Sender: [email-masked]

John C Klensin wrote:

> the issue now is, IMO, about a cosmetic change.

Okay, I promise to be silent if this particular
nit runs into any IESG [Discuss] roadblocks... :-)

Date: Mon, 30 Apr 2007 20:57:03 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Robert A. Rosenberg wrote:

> Query - Why not in an "Unable to Report to the Return Path" situation,
> just route the Notification Message to a Role Account address on the
> Server instead of (or in addition to) Logging?

That's fine.  It's the default for Sendmail.  It's also one of the
first things I change when I install Sendmail for anyone who does any
reasonable amount of mail.  Otherwise, the role account simply gets
inundated.

I think any mechanism that makes it easy for an operator to quickly figure
out what happened after the fact is acceptable, whether that's by reading
the archived role account mail or running a script on a log file.

The point is, I don't think we should lightly abandon NDNs, and
certainly not without at least mentioning steps that should be taken
by a responsible SMTP implementation to mitigate the damage.

Regards,

David.

Date: Mon, 30 Apr 2007 19:55:43 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-03 Issue 34:  non-2606 domains
Sender: [email-masked]

--On Tuesday, 01 May, 2007 00:27 +0200 Frank Ellermann
<[email-masked]> wrote:

> My consideration is that I don't know the owner of foo.com and
> can't tell if he likes that addresses in his domain are used in
> 2821.  And if he likes it I'd wonder if 2821 is a good place
> for foo.com search engine optimization.

Sure.  And, as I indicated in another note (possibly not to the
list), I have no problem at all with the IESG, or someone else,
insisting on the use of 2060 names in new documents.  But, for
those that have been using other names for a decade or two, and
the names they have been using,

(i) if the holder of foo.com doesn't like the impact of having
that domain used in examples, they presumably have gotten used
to it a long time ago... and the first of those examples
preceded their registration.   In particular, I suggest that
having such a domain for email may constitute holding up a
"spammers, especially dumb spammers, please hit me" sign,
especially if the mailboxes used in those examples actually
exist.

(ii) anyone who picks up those examples and uses them unchanged
to run or test something is sufficiently stupid that they
probably deserve to encounter what you politely describe as
search engine optimization.

(iii) the "search engine optimization" problem exists for http
connections.   An SMTP connection won't cause any of those
issues to be triggered although it might cause the sender's
putative backward-pointing address and IP address to be logged
and perhaps used in an attack or intelligence test.

I just don't know how far it is worth going to try to protect
stupid people from themselves.

> Another consideration are users making up what they think are
> "random" addresses / IPs / etc. hurting real addresses.  The 
> address harvesters and spiders are everywhere.  It's just bad
> practice to make up example addreses if there are reserved IPs,
> domains, and phone numbers for such purposes.

Again, if these addresses were being made up, new, at this
point, I'd certainly agree with you.  But they are quite old (in
Internet time), so the issue now is, IMO, about a cosmetic
change.

    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Date:  Tue, 01 May 2007 01:37:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 79
Sender: [email-masked]

John Leslie wrote:

> There's actually some rather good work towards that, called BATV:
> http://www.mipassoc.org/batv/

> which enables the use of a MailFrom which cannot be delivered to
> the "sender" unless it passes some checks by the "originating"
> mailserver.

BATV is rather tricky if you intend to send via unrelated routes
(different MONs in Keith's terminology) directing error reports
to one "originating" MRN.  If that's no issue BATV should reject
most unsolicited auto-responses (anything mentioned in RFC 3834 +
DSNs + MDNs) at the alleged originator.

For that point I'd say SPF can aggregate unrelated MONs on demand,
but it's difficult with BATV.  On the other hand BATV works for
almost all unsolicited auto-responses no matter what the bad guys
or the receivers do, while SPF depends on at least some receivers
supporting it, and that the bad guys not being sure who checks SPF
stay away from FAIL-protected addresses.

Both techniques can be combined as desired.  One disadvantage of
BATV is that it doesn't catch all forged Return-Paths (as SPF
FAIL does it when used), it only eliminates backscatter arriving
at the alleged sender in the form of bounces and auto-responses.

Before the spammers decided to stay away from my FAIL-protected
addresses (that took them some months in 2004, maybe the release
of SA 3 convinced them) most bounces I got where "user unknown"
and similar crap, rarely "over quota", a few hundreds of those
"spamarrest challenges", a wannabe-antispam pyramid scheme, and
a few hundreds other C/R-systems (earthlink etc.)

These "user unknown" bounces tell me that many spams forging my
address did make it to mailboxes of "known" primary victims.  The
backscatter is only a part of what really happened, in several
cases I could reconstruct the patterns of dictionary attacks.

BATV doesn't help with this aspect of the problem, forged mails
to existing users will end up in their mailbox (or spamfilter).
BATV also doesn't identify "good bounces welcome" like SPF PASS.

Its biggest advantage, complete independence of the good will of
receivers, let alone spammers, is also its biggest disadvantage.

Apart from that I recommend to combine BATV with SPF for users
who can limit their outbound routes (per domain) to one MON/MRN.

In a certain sense BATV is a hardcore version of SPF, it's more
restrictive wrt "where can I use which Return-Path".  OTOH it
has obviously no problem with alias-forwarding to third parties,
SPF insists on "check mails at your border or better forget it".

> we can set the stage for solutions of that ilk -- by allowing
> delivery servers to refuse to send NDNs without "satisfactory"
> assurance that the MailFrom address is useful to reach the
> sender.

+1

> That would _not_ be a fundamental change from RFC821 -- the
> "satisfactory" assurance was a given when 821 was written.

And implied by RFC 2821 if you care to read it until "originator
as indicated by the reverse path" finally reveals its secret
"if this indication is wrong you already did screw up" nature.

> And we're only fooling ourselves if we pretend that MUSTard in
> 2821bis will keep NDNs from being dropped in practice.

+1  Both alleged senders and receivers have to contribute to the
solution(s), and of course all affected parties try to say "it's
not our problem" following the 2821 lead.

Frank
-- 
Compare <http://www.openspf.org/Frank_Ellermann/Auto-responders>

Date: Mon, 30 Apr 2007 19:42:21 -0400
From: John C Klensin <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, [email-masked]
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

--On Monday, 30 April, 2007 18:05 -0400 "Robert A. Rosenberg"
<[email-masked]> wrote:

> 
> At 09:45 -0700 on 04/30/2007, [email-masked] wrote
> about Re: "for" clause on Received: header field:
> 
>> The problem I see with the matching approach, aside from the
>> obious overhead, is while you can assume that a successful
>> match to, say, a To: field means it is OK to expose that
>> recipient address in a for clause, not finding  any matches
>> doesn't mean the address has bcc semantics. So you end up
>> presenting  the subset
>> of the active recipient list that happen not to have been
>> transformed by forwarding/routing activites in ways that
>> prevent matching. The result makes for clause disppear in
>> prcecisely the cases where it is most useful for debugging
>> purposes: When transformation of the recipient addresses by
>> forwarding/routing has led to a mismatch between what's in
>> the header and what's in the envelope.

> The solution to the matching issue is simple. If you are the
> ultimate receiver domain (ie: Example.com for mail addressed
> to [email-masked]), just clone the message so there is a
> separate copy for each Rcpt-To address and place the
> appropriate address into the for clause of each cloned message
> - IOW: Act as a SMTP Server that is designed to not group
> multiple Rcpt-To commands into an outgoing email message even
> though the Mail-To's point at the same MX Server.

Robert,

Sure.  But, at the delivery server and in delivery to the
mailbox or mail store, messing with the "for" field this way is
certainly feasible, but misses some of the point because (i) the
user usually knows that it is his or her mailbox to which
delivery occurs and (ii) Received fields are hard for an
arbitrary MUA to parse.  

There may be information in the delivery address -- especially
for those of us who use many addresses pointing to a much
smaller number of mailboxes or folders-- but that is precisely
why many MTAs support and insert a local header equivalent to
"X-Delivered-To:" that contains only the relevant address (no
need to parse it out of the last Received header line).

     john

Date: Mon, 30 Apr 2007 16:34:14 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: new issue: non-2606 domains
Sender: [email-masked]

At 06:27 30-04-2007, Arnt Gulbrandsen wrote:

>Some examples use non-2606 domains.
>
>I suggest the following general replacements, which seem to make all 
>the examples and text conflict-free:

[snip]

>physics.foo-u.edut -> physics.example.edu

example.edu is not covered by RFC 2606.  It is however assigned to IANA.

>There's also an occurence of isi.edu, but I cannot bring myself to 
>suggest removing Jon Postel's email address from 821ter ;)

Me neither. :)

Regards,
-sm 

Date: Mon, 30 Apr 2007 16:29:22 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

At 23:09 29-04-2007, Frank Ellermann wrote:
>Whether those references are added or not, I still think the general
>SMTP model has to be "tuned" for this millennium.  All the "MUST NOT
>reject" should be toned down (or removed if the problem they tried to

We should also consider the legacy systems out there.  A complete 
reversal to reject, no bounces, makes the model 
unreliable.  Currently, the sender assumes that the mail has been 
delivered if they didn't get a bounce.  If we push too strongly for a 
"no bounce" approach, we'll be creating blackholes.

>But there's only one place for the receiving side to decide this, at
>the border MTA.  Later is too late, it can cause bounces to innocent
>bystanders, or - if the mail is dropped - it loses legit mail (in
>the case of a "false positive").

Sometimes the "border" is not clear-cut, i.e. the mail may go through 
several hops before reaching a MTA which can decide whether delivery 
will be successful or not.

I read:

    "[add REFERENCES here], has been done in providing ways to
     ascertain that an address is valid or belongs to the person who
     actually sent the message. A server MAY attempt to verify the return
     path before using its address for delivery notifications, but"

as a push towards SPF and CBV.  CBV looks like a way to get around 
the fact that most mail serves have VRFY disabled.  Under certain 
circumstances, CBV can cause a denial of service.

I suggested a rewording of Section 6.2. as that section deals with 
the bounces people have been complaining about.  It favors rejects 
without ruling out bounces.

Quoting two paragraphs from Section 6.2.

   "delivered.  Reliably determining that a return address is invalid can
    be a difficult and time-consuming process, especially if the putative
    sending system is not directly accessible or doesn't fully and
    accurately support VRFY and, even if a "drop messages with invalid
    return addresses" policy is adopted, it SHOULD be applied only when
    there is near-certainty that the return addresses are, in fact,
    invalid."

   "Conversely, if a message is rejected because it is found to contain
    hostile content (a decision that is outside the scope of an SMTP
    server as defined in this document), rejection ("bounce") messages
    SHOULD NOT be sent unless the receiving site is confident that those
    messages will be usefully delivered."

The wording above does bring the "SMTP model" in tune without making 
email unreliable as it outlines the pros and the cons.  The "SHOULD" 
strikes the right balance while a "MUST" may be read as prohibitive.

Regards,
-sm

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Date:  Tue, 01 May 2007 00:48:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

John Leslie wrote:
 
> "Reject at the border" is already becoming a normal practice: be
> patient...

Cando if 2821bis helps to encourage it... :-)

> We cannot presume universal use of SPF.

Yes, it would be also against the spirit of voluntary deployment.
For a "worldwide upgrade FUSSP" it's too complex, if that would
be possible (it's not) it could be done with a simpler technique.

> OTOH, I could live with your wording...

Which of the three proposed qualifiers does the trick for you ?

>> For the "normal" case (in my parallel universe that's an SPF PASS
>> or a similar case :-) I'd like to keep the "MUST bounce" as it was.
 
> Don't hang on too tight to that horse -- it's pining for the
> fjords... :^(

Yes.  We've to find a course between Scylla "accept + bounce" and
Charybdis "accept + silently drop".  And there's a common factor
in both undesirable outcomes. <eg>

Frank

Date: Mon, 30 Apr 2007 18:05:11 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

At 09:45 -0700 on 04/30/2007, [email-masked] wrote about 
Re: "for" clause on Received: header field:

>The problem I see with the matching approach, aside from the obious overhead,
>is while you can assume that a successful match to, say, a To: field means it
>is OK to expose that recipient address in a for clause, not finding 
>any matches
>doesn't mean the address has bcc semantics. So you end up presenting 
>the subset
>of the active recipient list that happen not to have been transformed by
>forwarding/routing activites in ways that prevent matching. The result makes
>for clause disppear in prcecisely the cases where it is most useful for
>debugging purposes: When transformation of the recipient addresses by
>forwarding/routing has led to a mismatch between what's in the header and
>what's in the envelope.

The solution to the matching issue is simple. If you are the ultimate 
receiver domain (ie: Example.com for mail addressed to 
[email-masked]), just clone the message so there is a separate copy 
for each Rcpt-To address and place the appropriate address into the 
for clause of each cloned message - IOW: Act as a SMTP Server that is 
designed to not group multiple Rcpt-To commands into an outgoing 
email message even though the Mail-To's point at the same MX Server.

Date: Mon, 30 Apr 2007 18:25:51 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

At 11:10 -0400 on 04/30/2007, David F. Skoll wrote about Re: Recap 
Issues 0b/21/25:

>A proper failure report SHOULD consist of a notification sent
>	to the Return-Path of the original message.  Where this is not
>	possible (because of a blank Return-Path) or not desirable
>	(because the Return-Path is untrusted for local policy
>	reasons), the server MUST record enough information for a
>	human operator to reconstruct the nature of the delivery
>	failure, the original Return-Path and the addresses of the
>	original recipients.

Query - Why not in an "Unable to Report to the Return Path" 
situation, just route the Notification Message to a Role Account 
address on the Server instead of (or in addition to) Logging? That 
way you have an ALREADY parsed list of the errors instead of needing 
to scan the Log to locate them.

Date: Mon, 30 Apr 2007 18:04:25 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

At 10:23 +0200 on 04/30/2007, Frank Ellermann wrote about Re: "for" 
clause on Received: header field:

>  > Previous standards (RFC 821, 822) did not allowed several
>>  addresses, so is these multiple mailboxes on "for" clause
>>  never implemented?
>
>If it's implemented it's not better than Apparently-To, as
>noted in 2821 4.4, but the Apparently-To got a "SHOULD NOT".
>Why allow an in essence identical damage in the for-clause ?

While I agree that multiple addresses in the for clause for 
relay-server created Receive headers is a bad idea (and a privacy 
issue like the above mentioned Apparently to), this should NOT 
preclude the creation of a single address for clause if the message 
gets cloned for final delivery once it gets to the destination domain.

IOW: If a message arrives addressed to [email-masked] and 
[email-masked], then none of the Received headers inserted on its way 
to example.com should have a for clause. OTOH, the example.com server 
should be allowed, if it wants, to clone the message (so a separate 
copy exists for each addressee as if the original relaying server was 
set to NOT group copies to the same domain into a single Multi 
Rcpt-To transmission) in which case inserting the single addressee 
for is a non-issue (ie: is in the same class as if this was the only 
[email-masked] address in a multi-address message). If the message is 
not-cloned but just delivered to each mailbox with no changes, there 
should be no for clause in the exanple.com inserted headers.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 34:  non-2606 domains
Date:  Tue, 01 May 2007 00:27:35 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 35
Sender: [email-masked]

John C Klensin wrote:
 
> Issue 34 assigned.

Thanks.  

Please use "edu.example" where Arnt proposed "example.edu", the
latter is not covered by 2606, and there's no RFC published by
TLD edu about their practice to reserve example.edu for examples.

> If the main consideration here is that people believe that 
> 2821bis would be improved, or reader confusion reduced, by 
> switching to names specifically provided for in 2606, then we
> should do it.

My consideration is that I don't know the owner of foo.com and
can't tell if he likes that addresses in his domain are used in
2821.  And if he likes it I'd wonder if 2821 is a good place for
foo.com search engine optimization.

Another consideration are users making up what they think are
"random" addresses / IPs / etc. hurting real addresses.  The 
address harvesters and spiders are everywhere.  It's just bad
practice to make up example addreses if there are reserved IPs,
domains, and phone numbers for such purposes.

> YMMD.

At some point in time we'll neeed a 2606bis, the first attempt 
in this direction wasn't convincing, we discussed this in 2005:

http://news.gmane.org/group/gmane.ietf.general/thread=16473/focus=17105

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Date:  Tue, 01 May 2007 00:00:53 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 72
Sender: [email-masked]

John C Klensin wrote:

> I believe that we cannot "make it permissible" to silently drop
> messages

Yes, that part makes me also very nervous without going into some
details under which conditions it's permissible.  One thing that's
apparently common practice is to delete mails identified as worms
by some piece of software.  The justification is that it's not
acceptable to put them into the mailboxes of clueless users, and
that it's of course also not acceptable to forward them to the
alleged senders claiming that it's a case of "MUST bounce".  But
my ISPs at least all asked me to permit this practice for worms
sent to me.  Or one ISP didn't ask me but "informs" me when it
deleted mail for me, where it thought that it might be a worm or
phish - resulting in hundreds of notifications per day :-(

Any strategy where users (or some years ago postmasters) are
supposed to wade through tons of spam, and try to identify the
false positives is a hopeless fig-leaf for "silently drop".

> there are all sorts of perfectly good reasons why one would not
> be able to validate the forward and reverse paths at the same
> time.

There are also all sorts of perfectly stupid reasons to avoid
this trouble, based on "2821 allows accept and bounce", or it
actually propagates "accept and bounce".  And that's wrong, it
should propagate "reject a.s.a.p. or else you're in deep sh*t".

> I believe that people who are advocating such things need to
> generate documents and push them through on the standards track,
> creating new requirements and new mechanisms, but paying
> attention to the whole set of issues associated with keeping
> mail delivery predictable and reliable,

Most SPF folks did propose standards track _and_ IETF Last Call,
they also did *not* support to close MARID precisely at the time
when the WG concluded that MS Sender-ID is technically flawed.
<http://purl.net/xyzzy/home/test/MARID.appeal.txt> (unconfirmed)

If you ever wonder why I'm so interested in all these procedural
questions, which is kind of overkill for two I-Ds.  Here we're not
discussing the merits of SPF, or if it's the most baroque solution
for an in essence simple 2821 problem, here we need to mitigate
the 2821 problems for folks deciding to use SMTP without SPF.

For starters 2821bis has to admit that there are serious problems
in its post-821 design, and that those 2821 design problems force
receivers to prefer "reject" instead of "accept" at their border.

Because if they don't do this and drop accepted mails silently
the whole design renders SMTP unreliable.  And continuing to send
bounces to innocent bystanders also doesn't cut it when the error
reports are automatically trashed as noise.  The design has to
encourage that "good" bounces are sent to the originator.  But I
don't see how 2821-SMTP can _enforce_ this without getting into a
"trash as noise" trap, with a similar effect as "drop silently".

> if we destroy whatever system is being attacked in order to
> fight off the attacks, the bad guys win.

If we ignore the attack and stick to "accept and bounce" the bad
guys also win.  We've to move the defenses to the border, where
they belong.

> I think that is the tip of a much larger iceberg.

Yes, and the iceberg has a number, 2821.

Frank

Date: Mon, 30 Apr 2007 17:48:11 -0400
From: John Leslie <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Frank Ellermann <[email-masked]> wrote:
> David F. Skoll wrote:
> 
> <snip language about "proper failure report">
> 
> SHOULD, should, recommended, or RECOMMENDED, what's going to happen
> with those huge log files, apart from deleting them by a cron-job ?

1) disk space is cheap;
2) they presumably will be rotated, and only deleted after a week or
   more;
3) parsing them by a script looking for an incident will be simple.

>> If we avoid defining what a "proper failure report" is and leave it
>> up to SMTP implementors, I fear that e-mail will become completely
>> unreliable (instead of somewhat unreliable as it is today.)
> 
> Yes, but it's your proposal to say "SHOULD bounce" instead of the
> "MUST bounce" in 2821.  I fear that this is another a loophole to
> avoid real (and painful) solutions, based on "reject at the border".

   "Reject at the border" is already becoming a normal practice: be
patient...

> I wouldn't be happy if an SPF PASS ends up in a log file deleted by
> a cron job.  Maybe your SHOULD needs a qualifier:  "If the receiver
> cannot validate the return-path then" etc.  

   We cannot presume universal use of SPF.

   OTOH, I could live with your wording...

> For the "normal" case (in my parallel universe that's an SPF PASS
> or a similar case :-) I'd like to keep the "MUST bounce" as it was.

   Don't hang on too tight to that horse -- it's pining for the
fjords... :^(

--
John Leslie <[email-masked]>

Date: Mon, 30 Apr 2007 17:24:15 -0400
From: John Leslie <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

John C Klensin <[email-masked]> wrote:
> --On Monday, 30 April, 2007 13:46 -0400 "David F. Skoll"
> <[email-masked]> wrote:
>> John C Klensin wrote:
>>> [David Skoll proposed:]
>>>
>>>> 	A proper failure report SHOULD consist of a notification
>>>> 	sent to the Return-Path of the original message.  Where
>>>> 	this is not possible (because of a blank Return-Path) or
>>>> 	not desirable (because the Return-Path is untrusted for
>>>> 	local policy reasons), the server MUST record enough
>>>> 	information for a human operator to reconstruct the nature
>>>> 	of the delivery failure, the original Return-Path and the
>>>> 	addresses of the original recipients.
>>> 
>>> ... one could probably replace the strong clause starting with
>>> ", the server MUST..." with some words that non-normatively
>>> advised that behavior...
>> 
>> OK; fair enough.  I think, though, that if we're going to make
>> it permissible to silently drop undeliverable messages (which
>> is what people want to avoid backscatter but is IMO totally
>> against the spirit of the original RFC), then we must put
>> something in writing that says what a well-behaved server is
>> expected to do.  I could live with a SHOULD.
> 
> Speaking personally, I believe that we cannot "make it
> permissible" to silently drop messages, absent policy
> considerations about the individual message, without requiring
> support for delivery notifications.

   Let's step back a pace or two.

   The "spirit of the original RFC" was that no-one would be trying
to game the system; thus MailFrom was quite likely to be correct,
and barring that, was unlikely to annoy anyone. This "spirit" is
so far expired that it's not even a good joke anymore.

   We need to let go of the pretense that NDNs-as-we-know-them-today
implement this "spirit".

   If you've followed me this far, the question should be, "What
is the best way available to notify someone able to fix a small
problem that the message will not be read?"

   There's actually some rather good work towards that, called BATV:

http://www.mipassoc.org/batv/

which enables the use of a MailFrom which cannot be delivered to
the "sender" unless it passes some checks by the "originating"
mailserver. This sort of mechanism can automate delivery of NDNs
to a server with good information about whether they match
something sent through that server.

   We certainly cannot mandate BATV in 2821bis, but we can set the
stage for solutions of that ilk -- by allowing delivery servers
to refuse to send NDNs without "satisfactory" assurance that the
MailFrom address is useful to reach the sender.

   That would _not_ be a fundamental change from RFC821 -- the
"satisfactory" assurance was a given when 821 was written.

   And we're only fooling ourselves if we pretend that MUSTard in
2821bis will keep NDNs from being dropped in practice.

> The alternatives essentially make email completely unpredictable,

   Not _all_ alternatives...

> since there are all sorts of perfectly good reasons why one
> would not be able to validate the forward and reverse paths at
> the same time.

   (What does that have to do with it?)

> And adding such a requirement would be as much out of scope for 2821
> as requiring some particular model of sender validation.

   Requiring delivery notifications is at least as unlikely to be
obeyed in practice as requiring NDNs.

   We should not expect to "jawbone" our way to our visions of how
email "should" work: we should be content to document the protocol,
leaving it to the folks who manage email systems to balance the
tradeoffs.

   Predictability of delivery is already dead. Users have stopped
expecting it. Honestly, we _have_ reached the stage where NDNs
are optional: if we want them to work "better", we should be
designing systems to give assurance of their utility.

--
John Leslie <[email-masked]>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Date:  Mon, 30 Apr 2007 23:08:06 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 50
Sender: [email-masked]

David F. Skoll wrote:

>>>| A proper failure report SHOULD consist of a notification sent
>>>| to the Return-Path of the original message.  Where this is not
>>>| possible (because of a blank Return-Path) or not desirable
>>>| (because the Return-Path is untrusted for local policy
>>>| reasons), the server MUST record enough information for a
>>>| human operator to reconstruct the nature of the delivery
>>>| failure, the original Return-Path and the addresses of the
>>>| original recipients.
 
>> Without commenting on whether or not this is the best approach,
>> or even a good idea, one could probably replace the strong
>> clause starting with ", the server MUST..." with some words that
>> non-normatively advised that behavior.
[...] 

+1
 
> OK; fair enough.  I think, though, that if we're going to make it
> permissible to silently drop undeliverable messages (which is what
> people want to avoid backscatter but is IMO totally against the 
> spirit of the original RFC), then we must put something in writing
> that says what a well-behaved server is expected to do.  I could
> live with a SHOULD.

SHOULD, should, recommended, or RECOMMENDED, what's going to happen
with those huge log files, apart from deleting them by a cron-job ?

> If we avoid defining what a "proper failure report" is and leave it
> up to SMTP implementors, I fear that e-mail will become completely
> unreliable (instead of somewhat unreliable as it is today.)

Yes, but it's your proposal to say "SHOULD bounce" instead of the
"MUST bounce" in 2821.  I fear that this is another a loophole to
avoid real (and painful) solutions, based on "reject at the border".

I wouldn't be happy if an SPF PASS ends up in a log file deleted by
a cron job.  Maybe your SHOULD needs a qualifier:  "If the receiver
cannot validate the return-path then" etc.  

Or "if the receiver cannot assess the validity of the return-path"
etc.  This needs some wordsmithing, it could be also "if the 
receiver isn't in a position where it can assess the plausibility".

For the "normal" case (in my parallel universe that's an SPF PASS
or a similar case :-) I'd like to keep the "MUST bounce" as it was.

Frank

Date: Mon, 30 Apr 2007 16:57:05 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

John C Klensin wrote:

> Speaking personally, I believe that we cannot "make it
> permissible" to silently drop messages, absent policy
> considerations about the individual message, without requiring
> support for delivery notifications.

I agree.

So my question remains:  Do we want to spell out explicitly that:

   Where there is a non-null Return-Path, a "proper failure report"
   MUST consist of an notification sent to the Return-Path unless
   policy considerations about the individual message dictate
   otherwise.  In any event, a proper failure report SHOULD permit an
   operator to reconstruct the nature of the delivery failure, the
   original sender, and the failed recipients.

Regards,

David.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: General-Address-Literal
Date:  Mon, 30 Apr 2007 22:31:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 80
Sender: [email-masked]

John C Klensin wrote:

  "[" ESC "\[" ETX NAK CAN SUB "]"
>> That's a General-Address-Literal as specified in 2821bis-03
>> if I didn't screw up.  The tested mail server didn't like it.

John convinced me that I screwed up, it has no tag.  Repeating
the EHLO literal test with "[IPv9:" ESC "\[" ETX NAK CAN SUB "]"

Santronics.com: 501.  Psg.com: 501.   De.clara.net: 501.

> You are assuming that, because something is permitted by the
> syntax, it is valid.

Almost.  I assume that implementors tackling <Address-Literal>
will implement it as generally as possible because it's used in
various places in the spec., everywhere where a <domain> is
allowed a domain-literal in square brackets is also allowed.

That's not limited to the EHLO.  Of course receivers could say
"your sending from an IPv4, I don't like your IPv9 EHLO, no
matter what IPv9 is".  I could repeat the test with an "IPv9"
MAIL FROM instead of the EHLO if that's your objection.

> This might, possibly, be a topic under Issue 24 (general
> ABNF cleanup), but it isn't inherently a problem with the
> spec because, as far as you know, the server is checking
> that the string isn't a valid IPv4 address literal nor a
> valid IPv6 literal and then returning a 550 code.

However they say 501 syntax error.  Yes, it's related to the
ABNF cleanup, but NO-WS-CTL is something more fundamental.

It violates a SHOULD in your net-UTF8 draft, and as far as I
followed your references it also violates a similar rule in
the net-ASCII RFCs:  Thou shalt not (ab)use control codes.

> first, your test string doesn't start with a _standardized_
> tag, so it is invalid syntax.

That's not how I'd implement it reading the spec.  Maybe I
should allow for a configurable list of "standardized tags",
populated with IPv6.  But I would wish to check IPv6 more
strictly than any "IPvFuture".  And I'd wish to get the
"IPvFuture" of RFC 3986 because it has no control codes :-)

> even a full ABNF rewrite can't fix this because the
> specification is intended to be extensible.

"Extending" the syntax to allow control codes is a bad plan:

| The "defined but not required" codes -- BEL, BS, HT, VT, FF --
| and the undefined control codes ("C0") SHOULD NOT be used
| unless required by exceptional circumstances.
 [I-D.klensin-net-utf8-03 section 2.2 bullet 1]

And it's no "extension", you already have <NO-WS-CTL>s as a
side-effect of using 2822 <dcontent>, and I like to kill it.

>> I'd also love to adopt the IPv6 syntax from RFC 3986, with
>> or without tag, but that would be a separate issue, and of
>> course we wouldn't break what works only for "aesthetical"
>> reasons.

> I believe it would also be out of scope and this is not the
> only place where the syntax permitted or required by SMTP is
> different from that of the URI spec.

An IPv6 is an IPv6, and I think the 3986 folks got this right.

It's potentially harmful for interoperability if 2821bis and
3986 have different ideas about the syntax of an IPv6 literal.

It's no surprise if 3986 is "better" wrt IPv6, 3986-2821=1165,
almost four years.  The USEFOR RFC uses 3986 syntax, SPF uses
3513 "syntax", and as far as I can tell it 3986 is the formal
ABNF of what 3513 says in prose by example.

Frank

Date: Mon, 30 Apr 2007 16:21:33 -0400
From: John C Klensin <[email-masked]>
To: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

--On Monday, 30 April, 2007 13:46 -0400 "David F. Skoll"
<[email-masked]> wrote:

> 
> John C Klensin wrote:
> 
> [David Skoll]
>>> 	A proper failure report SHOULD consist of a notification
>>> 	sent to the Return-Path of the original message.  Where
>>> 	this is not possible (because of a blank Return-Path) or
>>> 	not desirable (because the Return-Path is untrusted for
>>> 	local policy reasons), the server MUST record enough
>>> 	information for a human operator to reconstruct the nature
>>> 	of the delivery failure, the original Return-Path and the
>>> 	addresses of the original recipients.
> 
>> Without commenting on whether or not this is the best
>> approach, or even a good idea, one could probably replace the
>> strong clause starting with ", the server MUST..." with some
>> words that non-normatively advised that behavior.  There are
>> two problems with the strong version: one is that it would be
>> a new requirement, as you suggest and the second is that we
>> have carefully avoided imposing requirements on post-delivery
>> (or post-acceptance) behavior such as logging because such
>> behavior is not observable on the wire.
> 
> OK; fair enough.  I think, though, that if we're going to make
> it permissible to silently drop undeliverable messages (which
> is what people want to avoid backscatter but is IMO totally
> against the spirit of the original RFC), then we must put
> something in writing that says what a well-behaved server is
> expected to do.  I could live with a SHOULD.

Speaking personally, I believe that we cannot "make it
permissible" to silently drop messages, absent policy
considerations about the individual message, without requiring
support for delivery notifications.   The alternatives
essentially make email completely unpredictable, since there are
all sorts of perfectly good reasons why one would not be able to
validate the forward and reverse paths at the same time.   And
adding such a requirement would be as much out of scope for 2821
as requiring some particular model of sender validation.

Again speaking personally I believe that people who are
advocating such things need to generate documents and push them
through on the standards track, creating new requirements and
new mechanisms, but paying attention to the whole set of issues
associated with keeping mail delivery predictable and reliable,
no matter how much noise there is out there.  Put differently,
as with measures taken to fight other sorts of attacks, if we
destroy whatever system is being attacked in order to fight off
the attacks, the bad guys win.

> If we avoid defining what a "proper failure report" is and
> leave it up to SMTP implementors, I fear that e-mail will
> become completely unreliable (instead of somewhat unreliable
> as it is today.)

I think that is the tip of a much larger iceberg.

     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: "for" clause on Received: header field
Date:  Mon, 30 Apr 2007 21:33:59 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

John C Klensin wrote:

> Whether one has a single address in a "for" clause or three,
> the information disclosure risk is identical.

ACK.  My proposal to remove the Apparently-To section was a bad
plan.  You need something for the forward pointer to 7.2 in the
"for" section.

Rewriting 7.2 to address "for" issues instead of Apparently-To
could make sense, if it's not only me who missed that the "for"
can be a very similar problem.

IMO the concern that a "for" used only for one recipient could,
when absent, disclose the existence of one or more blind carbon
copies to receivers behind the same MX, is negligible compared
to a "for" disclosing directly one or all affected addresses.

And I'd still support Kari's proposal that at most one "for" as
in 82x is interesting enough, more than one can't be a good idea.

Frank

Date: Mon, 30 Apr 2007 13:46:14 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

John C Klensin wrote:

[David Skoll]
>> 	A proper failure report SHOULD consist of a notification sent
>> 	to the Return-Path of the original message.  Where this is not
>> 	possible (because of a blank Return-Path) or not desirable
>> 	(because the Return-Path is untrusted for local policy
>> 	reasons), the server MUST record enough information for a
>> 	human operator to reconstruct the nature of the delivery
>> 	failure, the original Return-Path and the addresses of the
>> 	original recipients.

> Without commenting on whether or not this is the best approach,
> or even a good idea, one could probably replace the strong
> clause starting with ", the server MUST..." with some words that
> non-normatively advised that behavior.  There are two problems
> with the strong version: one is that it would be a new
> requirement, as you suggest and the second is that we have
> carefully avoided imposing requirements on post-delivery (or
> post-acceptance) behavior such as logging because such behavior
> is not observable on the wire.

OK; fair enough.  I think, though, that if we're going to make it
permissible to silently drop undeliverable messages (which is what
people want to avoid backscatter but is IMO totally against the spirit
of the original RFC), then we must put something in writing that says
what a well-behaved server is expected to do.  I could live with a
SHOULD.

If we avoid defining what a "proper failure report" is and leave it up
to SMTP implementors, I fear that e-mail will become completely
unreliable (instead of somewhat unreliable as it is today.)

Regards,

David.

Date: Mon, 30 Apr 2007 13:30:24 -0400
From: John C Klensin <[email-masked]>
To: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: rfc2821bis-03 Issue 25: Re: Recap Issues 0b/21/25
Sender: [email-masked]

--On Monday, 30 April, 2007 11:10 -0400 "David F. Skoll"
<[email-masked]> wrote:

> I think it should read something like this:
> 
> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.
> 
> 	A proper failure report SHOULD consist of a notification sent
> 	to the Return-Path of the original message.  Where this is not
> 	possible (because of a blank Return-Path) or not desirable
> 	(because the Return-Path is untrusted for local policy
> 	reasons), the server MUST record enough information for a
> 	human operator to reconstruct the nature of the delivery
> 	failure, the original Return-Path and the addresses of the
> 	original recipients.
> 
> I fear, though, that may exceed the charter as you say.

Without commenting on whether or not this is the best approach,
or even a good idea, one could probably replace the strong
clause starting with ", the server MUST..." with some words that
non-normatively advised that behavior.  There are two problems
with the strong version: one is that it would be a new
requirement, as you suggest and the second is that we have
carefully avoided imposing requirements on post-delivery (or
post-acceptance) behavior such as logging because such behavior
is not observable on the wire.

         john

Date: Mon, 30 Apr 2007 13:24:50 -0400
From: John C Klensin <[email-masked]>
To: John Leslie <[email-masked]>, "David F. Skoll" <[email-masked]>
cc: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

--On Monday, 30 April, 2007 12:10 -0400 John Leslie
<[email-masked]> wrote:

>    Sending an NDN in the case where MailFrom is blank is
> already prohibited (isn't it?); so that clause is not
> necessary. Other than that, I could live with this.

Yes, it is now prohibited.

    john

From: [email-masked]
Cc: [email-masked], Kari Hurtta <[email-masked]>
Date: Mon, 30 Apr 2007 09:45:41 -0700 (PDT)
Subject: Re: "for" clause on Received: header field
To: Kari Hurtta <[email-masked]>
Sender: [email-masked]

> John C Klensin <[email-masked]> writes in gmane.ietf.smtp:

> > --On Monday, 30 April, 2007 10:23 +0200 Frank Ellermann
> > <[email-masked]> wrote:
> >
> > >...
> > >> Previous standards (RFC 821, 822) did not allowed several
> > >> addresses, so is these multiple mailboxes on "for" clause
> > >> never implemented?
> > >
> > > If it's implemented it's not better than Apparently-To, as
> > > noted in 2821 4.4, but the Apparently-To got a "SHOULD NOT".
> > > Why allow an in essence identical damage in the for-clause ?
> >
> > Whether one has a single address in a "for" clause or three, the
> > information disclosure risk is identical.   If the address(es)
> > in "for" are identical to, or a subset of, the forward-pointing
> > addresses in the headers, then there is no information
> > disclosure and no problem whether there is one address or more
> > than that.  If anything in the "for" is not in the
> > forward-pointing address set in the headers, then there is a
> > disclosure that could be problematic.

> Looking from another angle:

>         If mail have just one envelope recipient and
>         that is copied to "for" clause, that does not disclose
>         possible Bcc: recipients (only possible BCC recipient
>         for that copy of mail is just that recipient of mail.)

Not quite. The exposure occurs not when the for clause is present but when it
is absent. If the recipient knows that his mail system inserts a for clause
whenever there's only one recipient but does not do so when there are more than
one, they can use the lack of a for clause as an indicator that there was
another recipient. And if they are the local user listed in the headers that's
close to a confirmation that a bcc operation was done. They can't tell who the
bcc recipient was but they can tell that there was one.

Mind you, I think this is enough of a concern compared with the value of having
for clause for debugging purposes that we ship with settings that insert a for
clause when there's only one recipient, but we do provide a way to turn it off
for sites that are sufficiently concerned about bcc exposures.

It's even possible for the presence of a legitimately generated for clause to
reveal a bcc recipient, but it's such an obscure and bizarre case that it isn't
worth worrying about. Here's how it would work: Suppose you have a setup where
a user agent creates an alias on the server that expands to a normal recipient
and to a bcc recipient, then sends the mail to the alias while listing on the
normal recipient in the headers it creates. But only only does this not make
sense from a practical perspective - why on earth would anyone do something so
bizarre - but it also has "layering violation" and "other nasty security
implications" written all over it.

>         If there is several envelope recipients on mail and
>         these are copied to "for" clause, possible BCC recipients
>         are disclosed. Avoiding of that requires that
>         envelope recipients are matched to addresses on header
>         section.

The problem I see with the matching approach, aside from the obious overhead,
is while you can assume that a successful match to, say, a To: field means it
is OK to expose that recipient address in a for clause, not finding any matches
doesn't mean the address has bcc semantics. So you end up presenting the subset
of the active recipient list that happen not to have been transformed by
forwarding/routing activites in ways that prevent matching. The result makes
for clause disppear in prcecisely the cases where it is most useful for
debugging purposes: When transformation of the recipient addresses by
forwarding/routing has led to a mismatch between what's in the header and
what's in the envelope.

> > At least judging from the DRUMS discussion, the problem with
> > Apparently-to is that it was supplied on precisely those
> > occasions on which the forward-pointing envelope address did not
> > match any of the forward-pointing header addresses, so, almost
> > by definition, it was a disclosure problem.
> >
> >      john

> Apparently-To: was generated when mail was no address headers.

Right, which never made some to me in that it didn't actually correct the
syntax error of not having at least one recipient field. (Yes, I know this is
no longer an error in 2822, but it was with 822.)

				Ned

Date: Mon, 30 Apr 2007 18:37:02 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Cc: John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> ...
> Is that more or less what you have in mind?

Yes.

> If so, two questions for the group:
>
> (1) Is this worth doing (I think yes -- at least the
>        C: MAIL FROM:<@foo.com:[email-masked]>
> needs to be fixed for consistency with other changes).

That's exactly what made me aware of this, so chalk up one more yes.

> (2) Note that a side-effect of the famous issue 25 (about NDN 
> "bounces") is that, if the relay (foo.com) cannot verify either the 
> forward-pointing address ([email-masked]) or the backward-pointing one 
> ([email-masked]) prior to sending back the 250 response to the RCPT 
> command, it is in big trouble.   If we
> apply this fix (or even if we don't), does it immediately go into the 
> Issue 25 list?

I'd try to skirt the issue by saying "foo.com, having received the 
message, verifies delivery to [email-masked] is permissible and 
possible. foo.com now does a DNS lookup on xyz.com."

Arnt

Date: Mon, 30 Apr 2007 12:10:08 -0400
From: John Leslie <[email-masked]>
To: "David F. Skoll" <[email-masked]>
Cc: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

David F. Skoll <[email-masked]> wrote:
> 
> I think it should read something like this:
> 
> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.
> 
> 	A proper failure report SHOULD consist of a notification sent
> 	to the Return-Path of the original message.  Where this is not
> 	possible (because of a blank Return-Path) or not desirable
> 	(because the Return-Path is untrusted for local policy
> 	reasons), the server MUST record enough information for a
> 	human operator to reconstruct the nature of the delivery
> 	failure, the original Return-Path and the addresses of the
> 	original recipients.

   Sending an NDN in the case where MailFrom is blank is already
prohibited (isn't it?); so that clause is not necessary. Other than
that, I could live with this.

--
John Leslie <[email-masked]>

Date: Mon, 30 Apr 2007 11:43:51 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

Kari Hurtta wrote:

>         If mail have just one envelope recipient and
>         that is copied to "for" clause, that does not disclose
>         possible Bcc: recipients (only possible BCC recipient
>         for that copy of mail is just that recipient of mail.)

This is what Sendmail does by default, I believe.  It only includes
the for clause if there's one envelope recipient.

However, it could still be an information leak if the single envelope
recipient is a mailing list.  For example:

> Received: ... for <[email-masked]>

Contrived, I know. :-)

Regards,

David.

To: [email-masked]
From: Kari Hurtta <[email-masked]>
Subject:  Re: "for" clause on Received: header field
Date:  30 Apr 2007 18:20:07 +0300
Lines: 47
Cc: Kari Hurtta <[email-masked]>
Sender: [email-masked]

John C Klensin <[email-masked]> writes in gmane.ietf.smtp:

> --On Monday, 30 April, 2007 10:23 +0200 Frank Ellermann
> <[email-masked]> wrote:
> 
> >...
> >> Previous standards (RFC 821, 822) did not allowed several
> >> addresses, so is these multiple mailboxes on "for" clause
> >> never implemented?
> > 
> > If it's implemented it's not better than Apparently-To, as
> > noted in 2821 4.4, but the Apparently-To got a "SHOULD NOT".
> > Why allow an in essence identical damage in the for-clause ?
> 
> Whether one has a single address in a "for" clause or three, the
> information disclosure risk is identical.   If the address(es)
> in "for" are identical to, or a subset of, the forward-pointing
> addresses in the headers, then there is no information
> disclosure and no problem whether there is one address or more
> than that.  If anything in the "for" is not in the
> forward-pointing address set in the headers, then there is a
> disclosure that could be problematic.

Looking from another angle:

        If mail have just one envelope recipient and
        that is copied to "for" clause, that does not disclose
        possible Bcc: recipients (only possible BCC recipient
        for that copy of mail is just that recipient of mail.)

        If there is several envelope recipients on mail and
        these are copied to "for" clause, possible BCC recipients
        are disclosed. Avoiding of that requires that 
        envelope recipients are matched to addresses on header
        section.

> At least judging from the DRUMS discussion, the problem with
> Apparently-to is that it was supplied on precisely those
> occasions on which the forward-pointing envelope address did not
> match any of the forward-pointing header addresses, so, almost
> by definition, it was a disclosure problem.
> 
>      john

Apparently-To: was generated when mail was no address headers.

/ Kari Hurtta

Date: Mon, 30 Apr 2007 11:10:31 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

John Leslie wrote:

>    I claim this data supports my contention that folks don't _use_
> the NDNs that they get.

I don't agree.  I do agree that almost nobody uses or understands
the content of the NDNs they get.  However, I would guess that anywhere
from 2% to 10% of people use the *fact* that they received an NDN to take
further action.  We deal with many unsophisticated users, and although they
have no clue what an NDN means, the receipt of one prompts them to contact
their helpdesk person or the intended recipient (if they can figure out
that much...)

>    Getting back to the question, "properly report" involves, at the
> minimum, logging the incident so that it's _possible_ to reconstruct
> what happened. I'd like to suggest forwarding those log entries to
> the client you received the message from, but I suspect that exceeds
> our charter.

I think it should read something like this:

	In either case, a formal handoff of responsibility for
	the message occurs: the protocol requires that a server
	MUST accept responsibility for either delivering a
	message or properly reporting the failure to do so.

	A proper failure report SHOULD consist of a notification sent
	to the Return-Path of the original message.  Where this is not
	possible (because of a blank Return-Path) or not desirable
	(because the Return-Path is untrusted for local policy
	reasons), the server MUST record enough information for a
	human operator to reconstruct the nature of the delivery
	failure, the original Return-Path and the addresses of the
	original recipients.

I fear, though, that may exceed the charter as you say.

Regards,

David.

Date: Mon, 30 Apr 2007 10:55:22 -0400
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

Issue 35 assigned, note below.

--On Monday, 30 April, 2007 15:31 +0200 Arnt Gulbrandsen
<[email-masked]> wrote:

> I like the text about source routes, however I think example
> D.3 is too much of a good thing. I would like to have it
> replaced by a simpler "Secondary MX" example.

Just to clarify what you are looking for, you would have me
change D.3 so it reads something like (ignoring the 2606 issue)

  Step 1 -- Source Host to Relay Host

	The source host performs a DNS lookup on XYZ.COM (the
	destination address) and finds DNS MX records specifying
	xyz.com as the best preference and foo.com as a lower
	preference.  It attempts to open a connection to xyz.com
	and fails.  It then opens a connection to foo.com, with
	the following dialogue:

    S: 220 foo.com Simple Mail Transfer Service Ready
    C: EHLO bar.com
        [...]
    C: RCPT TO:<[email-masked]>
    S: 250 OK
        [...]
    C: .
    S: 250 OK
    C: QUIT
    S: 221 foo.com Service closing transmission channel

 Step 2 -- Relay Host to Destination Host

	foo.com, having received the message, now does a DNS
	lookup on xyz.com.  It finds the same set of MX records,
	but cannot use the one that points to it (or to any
	other host as a worse preference).  It tries to open a
	connection to xyz.com itself and succeeds. Then we have:

    S: 220 xyz.com Simple Mail Transfer Service Ready
    C: EHLO foo.com
    S: 250 xyz.com is on the air
    C: MAIL FROM:<[email-masked]>
    S: 250 OK
    C: RCPT TO:<[email-masked]>
          [...]
  
Is that more or less what you have in mind?   If so, two
questions for the group:

(1) Is this worth doing (I think yes -- at least the 
       C: MAIL FROM:<@foo.com:[email-masked]>
needs to be fixed for consistency with other changes).

(2) Note that a side-effect of the famous issue 25 (about NDN
"bounces") is that, if the relay (foo.com) cannot verify either
the forward-pointing address ([email-masked]) or the
backward-pointing one ([email-masked]) prior to sending back the
250 response to the RCPT command, it is in big trouble.   If we
apply this fix (or even if we don't), does it immediately go
into the Issue 25 list?

   john

From: "Mark E. Mallett" <[email-masked]>
Date: Mon, 30 Apr 2007 10:50:38 -0400
To: Tony Hansen <[email-masked]>
Cc: [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

On Sun, Apr 29, 2007 at 11:04:30PM -0400, Tony Hansen wrote:
> 
> The consensus on this text
> 
>   It is worth noting that the file transfer protocol (FTP [35]) uses a
>   very similar code architecture and that the SMTP codes are based on
>   the FTP model.  However, SMTP uses a one-command, one-response model,
>   while FTP is asynchronous and the 1yz codes are not part of the SMTP
>   model.
> 
> is almost dead even. I'll leave this one open for two more days, then
> give a consensus call. If it remains even, it will become "editor's choice".

Seems reasonable to me; it does no harm and adds a small bit of
information.  Not being able to resist bit-fiddling, I'd replace "the
1yz codes" with "1yz codes" or "FTP's 1yz codes", though.

mm

Date: Mon, 30 Apr 2007 10:46:36 -0400
From: John Leslie <[email-masked]>
To: Jeff Macdonald <[email-masked]>
Cc: [email-masked]
Subject: Re: 2821bis-03
Sender: [email-masked]

Jeff Macdonald <[email-masked]> wrote:
> On Fri, Apr 27, 2007 at 01:59:41PM -0400, John Leslie wrote:
> <snip> 
>> Today the vast majority of NDNs merely indicate that some spammer
>> forged your email address into a MailFrom -- often without any
>> particular evil intent except to bypass Verizon's dumb CBV. We need
>> to admit that spammers aren't going away anytime soon, nor is
>> Verizon's practice of callback verification.
> 
> Verizon recently (a month or so ago) stopped doing CBV. At least that's
> what the original implementer told me. :)

   Indeed, I find these have disappeared from my logs. :^)

   I stand corrected.

--
John Leslie <[email-masked]>

Date: Mon, 30 Apr 2007 10:39:35 -0400
From: John Leslie <[email-masked]>
To: "David F. Skoll" <[email-masked]>
Cc: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

David F. Skoll <[email-masked]> wrote:
> 
> Tony Hansen wrote [suggested text]:
> 
>> A server MAY attempt to verify the return
>> path before using its address for delivery notifications, but
>> methods of doing so are not defined here nor is any particular
>> method recommended at this time.
> 
> How do we reconcile that with:
> 
>> ... the protocol requires that a server
>> MUST accept responsibility for either delivering a
>> message or properly reporting the failure to do so.
> 
> If a server verifies the return path and decides not to use it, what
> constitutes "...properly reporting the failure" to deliver a message?

   A good question -- well worth our time to discuss a bit, though I
doubt we'll come up with any text to include.

   First, I'd like to violate one of the rules here, by including some
actual data ;^(

   We run a number of mail-servers here, several of which apply the
same anti-spam rule and return a 5xy error including a reference to
a jlc.net webpage. I collected one week's data for that one rule:

    5xy responses sent: 368,074
     webpage accesses:        2

   (Actually, those two were from the same IP address, less than
five minutes apart.)

   I claim this data supports my contention that folks don't _use_
the NDNs that they get. (Yes, I know, the clients we gave that 5xy
to may not have delivered the actual text in the NDN they generated.)

   I believe any data we could collect involving "typical" users would
show the same trend: 99.9% of NDNs are not read. (What folks _do_,
in my experience, is call the intended recipient and complain, "your
spam filter is broken" -- if they do anything at all, that is...)

   This is why, BTW, I'd like to retreat from SHOULD to MAY on sending
NDNs: they accomplish almost nothing, and they annoy too many folks..

   Getting back to the question, "properly report" involves, at the
minimum, logging the incident so that it's _possible_ to reconstruct
what happened. I'd like to suggest forwarding those log entries to
the client you received the message from, but I suspect that exceeds
our charter. 

   Sending an NDN which has only one chance in a million of conveying
the right information shouldn't be touted as "properly reporting".
Log files with better persistence are far more useful.

   IMHO, we're going to have to settle for vague language on what to
do in the case of an error not detected during the SMTP session.

--
John Leslie <[email-masked]>

Date: Mon, 30 Apr 2007 16:35:37 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 34: non-2606 domains
Cc: John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> Issue 34 assigned.
>
> <editor hat on>
> ok.  Reluctant as I am to open that text for cosmetic reasons, this 
> seems plausible.

Hm. I suggested the issue _only_ because of an impression that it'll 
have to be done either now or during IESG evaluation.

> If no one seriously objects, I am willing to make the changes for -04 
> iff at least one will step forward and promise the group to 
> _carefully_ check the edits.
> ...

Not me - I write the first list and would suffer from tunnel vision.

Arnt

Date: Mon, 30 Apr 2007 10:22:11 -0400
From: Jeff Macdonald <[email-masked]>
To: John Leslie <[email-masked]>
Cc: John C Klensin <[email-masked]>, Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: 2821bis-03
Sender: [email-masked]

On Fri, Apr 27, 2007 at 01:59:41PM -0400, John Leslie wrote:
<snip> 
>    Today the vast majority of NDNs merely indicate that some spammer
> forged your email address into a MailFrom -- often without any
> particular evil intent except to bypass Verizon's dumb CBV. We need
> to admit that spammers aren't going away anytime soon, nor is
> Verizon's practice of callback verification.

Verizon recently (a month or so ago) stopped doing CBV. At least that's
what the original implementer told me. :)

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Mon, 30 Apr 2007 10:20:16 -0400
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: rfc2821bis-03 Issue 34:  non-2606 domains 
Sender: [email-masked]

Issue 34 assigned.

<editor hat on>
ok.  Reluctant as I am to open that text for cosmetic reasons,
this seems plausible.  If no one seriously objects, I am willing
to make the changes for -04 iff at least one will step forward
and promise the group to _carefully_ check the edits.   I expect
everyone to read -04, of course, but this is the sort of thing
that can easily slip into the "my eyes glaze over and I assume
someone else has looked at it" category, resulting in the
introduction of bugs.
</editor hat off>

Personal observation: Regardless of the good or bad experiences
others have had (or heard rumors of) with IESG nit-picking that
puts procedures ahead of getting work done, I have gotten very
aggressive about that stuff for reasons unrelated to this
specific effort.  If the main consideration here is that people
believe that 2821bis would be improved, or reader confusion
reduced, by switching to names specifically provided for in
2606, then we should do it.  If, by contrast, the reason for
making this change is because of a perceived requirement to
"conform" to 2606, I would rather spend my time writing appeals
than changing and checking this text.   I note, upon rereading
2606, that it makes some names available for use in examples,
but does not, in any way, require the use of those names or even
(unless one reads the Security Considerations section though an
"anything not permitted is forbidden" lens) suggest that they
should be required.   If there is now such a requirement, it is
one imposed without clear evidence of community consensus.  Even
if such consensus existed and were demonstrated, it would be
more reasonable as a requirement on new documents than as a
basis for forcing additional work and changes to documents being
updated.   Just my opinion and issue, of course, YMMD.

      john

--On Monday, 30 April, 2007 15:27 +0200 Arnt Gulbrandsen
<[email-masked]> wrote:

> 
> Some examples use non-2606 domains.
> 
> I suggest the following general replacements, which seem to
> make all the examples and text conflict-free:
> 
> foo.org -> foo.example.org
> bar.org -> bar.example.org
> d.bar.org -> d.example.org
> xyz.com -> v.example.org
> ijk.org -> example.com
> foo.com -> example.com
> physics.foo-u.edut -> physics.example.edu
> specific.generic.com -> example.com
> ISIF.USC.EDU -> ISIF.EXAMPLE.EDU
> bar.com -> example.net
> 
> There's also an occurence of isi.edu, but I cannot bring
> myself to suggest removing Jon Postel's email address from
> 821ter ;)
> 
> Arnt
> 

Date: Mon, 30 Apr 2007 15:31:47 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: new issue: remove source routes from example D.3
Sender: [email-masked]

I like the text about source routes, however I think example D.3 is too 
much of a good thing. I would like to have it replaced by a simpler 
"Secondary MX" example.

Arnt

Date: Mon, 30 Apr 2007 15:27:35 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: new issue: non-2606 domains
Sender: [email-masked]

Some examples use non-2606 domains.

I suggest the following general replacements, which seem to make all the 
examples and text conflict-free:

foo.org -> foo.example.org
bar.org -> bar.example.org
d.bar.org -> d.example.org
xyz.com -> v.example.org
ijk.org -> example.com
foo.com -> example.com
physics.foo-u.edut -> physics.example.edu
specific.generic.com -> example.com
ISIF.USC.EDU -> ISIF.EXAMPLE.EDU
bar.com -> example.net

There's also an occurence of isi.edu, but I cannot bring myself to 
suggest removing Jon Postel's email address from 821ter ;)

Arnt

Date: Mon, 30 Apr 2007 09:04:41 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

--On Monday, 30 April, 2007 10:23 +0200 Frank Ellermann
<[email-masked]> wrote:

>...
>> Previous standards (RFC 821, 822) did not allowed several
>> addresses, so is these multiple mailboxes on "for" clause
>> never implemented?
> 
> If it's implemented it's not better than Apparently-To, as
> noted in 2821 4.4, but the Apparently-To got a "SHOULD NOT".
> Why allow an in essence identical damage in the for-clause ?

Whether one has a single address in a "for" clause or three, the
information disclosure risk is identical.   If the address(es)
in "for" are identical to, or a subset of, the forward-pointing
addresses in the headers, then there is no information
disclosure and no problem whether there is one address or more
than that.  If anything in the "for" is not in the
forward-pointing address set in the headers, then there is a
disclosure that could be problematic.

At least judging from the DRUMS discussion, the problem with
Apparently-to is that it was supplied on precisely those
occasions on which the forward-pointing envelope address did not
match any of the forward-pointing header addresses, so, almost
by definition, it was a disclosure problem.

     john

Date: Mon, 30 Apr 2007 08:53:34 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Tony Hansen <[email-masked]>
cc: SMTP Interest Group <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Two questions/ comments about this note...

--On Sunday, 29 April, 2007 23:19 -0400 Hector Santos
<[email-masked]> wrote:

>...
>> Thoughts on an addition like this? I know it doesn't go far
>> enough for some people, and probably goes too far for some
>> other people. So I'm offering it as a possible way forward.
> 
> +1 and -1
> 
> Why +1?
>...
> Accepting a local forward-path without authentication is the
> industry BCP.  For a remote forward-path, requiring
> authentication is the industry BCP, otherwise you may have
> open-relay entry points and accept/bounce attack threat entry
> points.

It is a "best practice" or even a "general current practice"
(the latter has, IMO, much more impact on 2821bis under the
rules than discussions of "best") iff "authentication" is taken
very broadly.  The reality, as I have observed it, is that it is
general, current, and best practice to prevent open relay
behavior.  But that is as often done by validating the sender in
some fashion rather than validating / authenticating the return
path.    As far as I can tell, we have:

	(i) Techniques for validating the authorization of a
	particular sender to send mail using a given submission
	server.  The return paths are not validated and use by
	an sender who is authorized to use a particular
	submission server of an inappropriate backward-pointed
	address is dealt with administratively (including, but
	not restricted, to legal proceedings) if at all.  A
	special subset of these techniques does not depend on
	mail processing at all but on the mechanisms, such as an
	authenticated tunnel, to reach a submission server that
	is not otherwise accessible.
	
	(ii) Techniques for validating the relationship between
	a backward-pointed address and a particular domain.
	Those techniques may or may not involve any element of
	sender authorization.  And some of them are better
	suited to evaluation of the likelihood that the message
	is bogus by the receiver, rather than for rejection of
	the mail by a relay MTA.
	
	(iii) Techniques for validating the domain of the
	backward-pointing address, but not specific addresses or
	the binding between address and sender or user (note
	that, as RFC 822 and its successors point out, these can
	be different).   Most of these are more useful for
	receiver assessment than than are for accepting/
	rejecting mail, at least IMO. 
	
	(iv) In theory, (i) could be combined with validation of
	a specific list of addresses which a particular sender
	is permitted to use in backward-pointed addresses in
	both envelope and headers.  In practice, other than
	intra-enterprise in a few cases, I don't believe this is
	regularly done in a sophisticated way.  I have seen it
	done by imposing a rule that one is only permitted to
	send mail from a specific ISP-allocated address, but
	that mechanism --while it reduces spam and spoofing-- is
	not good for the notion that people should be able to
	link email addresses to identities, independent of the
	particular ISP to which they are connected, if they want
	to do so.

All of this goes far beyond today's 2821 and 2821bis: as far as
they are concerned, these are administrative procedures, not
parts of the protocol.  More important, I think it would be
unwise for us to change 2821 in a way that seemed to favor one
of these approaches over another or to pretend that one
dominated the others in terms of current practice.

> In any case, I think it is very important, IMO, that the
> strong point that the reverse-path MUST be valid before it can
> serve any usefulness is stated, and that is how this is done
> is out of scope.

I think it is hard to argue that the "MUST" in your sentence
above is a 2821 problem or that it represents current practice,
rather than a preference (however important), and that there are
important questions about what is meant by validity.  As soon as
"validity" gets over into "authorization to use", one may be
dealing with issues that cannot possibly be validated by a
protocol.
 
> Why -1?
> 
> The problem with your 3463-like suggestion is that it creates
> the dilemma and question of "WHEN" should the reverse-path be
> considered valid?
> 
> The specification mandates a MUST for reverse-path validity.
> CAN-SPAM also mandates it as well.

IANL and your mileage/ interpretation may differ, but I don't
believe that CAN-SPAM mandates any such thing.   What CAN-SPAM
mandates is that the sender address must be available and
actually reach the sender (or message originator -- it is
ambiguous about that too if we accept our understanding of those
terms).  I think all of its requirements could be met with a
valid header "From:" field and a completely bogus return-path
and maybe vice versa.

> So the question is?
> 
> When is the VALIDITY of the reverse-path important?
> 
>      At the moment it is presented?  or
>      At the moment it is required to be used - a BOUNCE action?
> 
> The latter is what 3463 may imply and create a design model
> using this concept - which in my view is not quite safe in
> today's world.

Of course.  If someone is trying to subvert the system, there
are race conditions involved that are easily manipulated.   But
I don't see what 2821bis could say about this even if we were so
inclined.

    john

Date: Mon, 30 Apr 2007 08:44:17 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Tony Hansen wrote:

>     A server MAY attempt to verify the return
>     path before using its address for delivery notifications, but
>     methods of doing so are not defined here nor is any particular
>     method recommended at this time.

How do we reconcile that with:

>       ... the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.

If a server verifies the return path and decides not to use it, what
constitutes "...properly reporting the failure" to deliver a message?

Regards,

David.

Date: Mon, 30 Apr 2007 05:31:52 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Lisa Dusseault <[email-masked]>, [email-masked]
CC: John C Klensin <[email-masked]>, Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>, Chris Newman <[email-masked]>, Ted Hardie <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: RFC2821 advancement -- implementation reports
Sender: [email-masked]

Folks,

One of the requirements for advancement is implementation reports.  That 
raises the question of what to document, given the massive deployment we 
already know exists?

I suggest soliciting a collection of reports, from vendors and operators 
with their own code.  The reports should declare:

    1. That they have implemented RFC2821.

    2. For how long it has been deployed.

    3. What features they know they have *not* implemented

    4. What features they know to be problematic

    5. Any other comments they might wish to share.

The collection of these documents is certain to show that RFC2821 has
gained widespread adoption. Whether it will show any pattern of 
interesting problems ought to be quickly discovered.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 30 Apr 2007 08:06:10 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

--On Sunday, 29 April, 2007 23:38 -0400 Tony Hansen
<[email-masked]> wrote:

> 
> I think someone suggested moving the comment to Appendix F
> Deprecated Features. I can't find the suggestion right this
> minute though.

I don't recall seeing such a suggestion.  It would be odd, since
I think out consensus --implicit in removing the 1yz explanation
itself-- is that 1yz codes were never there.

     john

> John C Klensin wrote:
>> 
>> --On Sunday, 29 April, 2007 23:04 -0400 Tony Hansen
>> <[email-masked]> wrote:
>> 
>>> The consensus on this text
>>> 
>>>   It is worth noting that the file transfer protocol (FTP
>>> [35]) uses a   very similar code architecture and that the
>>> SMTP codes are based on   the FTP model.  However, SMTP uses
>>> a one-command, one-response model,   while FTP is
>>> asynchronous and the 1yz codes are not part of the SMTP
>>> model.
>>> 
>>> is almost dead even. I'll leave this one open for two more
>>> days, then give a consensus call. If it remains even, it will
>>> become "editor's choice".
>> 
>> Speaking personally (not as editor), I'm slightly inclined to
>> leave it, simply because dropping of the 1yz text would seem
>> odd without _some_ comment.  Very slightly, however.
>> 
>>      john
> 

Date: Mon, 30 Apr 2007 07:52:12 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: General-Address-Literal (was: Issue list)
Sender: [email-masked]

--On Monday, 30 April, 2007 07:22 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> John C Klensin wrote:
> 
>>> Kari  (2007-04-20): "for" clause on Received: header field
>>> http://permalink.gmane.org/gmane.ietf.smtp/5708
>  
>> As far as Pete and I are concerned, this has been resolved 
>> by further stripping the Received line syntax from 2822.
> 
> Okay, I had Kari's article still on "read again". and didn't
> look if 2821upd already did something with this issue.  
>  
>>> http://permalink.gmane.org/gmane.ietf.smtp/5810
>  
>> This note is a confusion of issues.  If you want an issue
>> number assigned to one or more of the issues in it, post
>> separate notes with meaningful subject lines.
> 
> The subject line was General-Address-Literal, I try it again.
> 
> Here's my "implementation and interoperability test" result:
> 
> using SPAMVIR.DE.CLARA.NET for xyzzy.claranet.de
> 20070430 06:50:34 TCP connection with SPAMVIR.DE.CLARA.NET:25
> 50:34.14 220 Claranet Germany Mail Service
> 50:34.14 ehlo x2c( '5B1B5C5B0315181A5D' )
> 50:34.26 501 Syntactically invalid EHLO argument(s)
> 50:34.26 quit
> 50:34.39 221 spamvir07.de.clara.net closing connection
> 
> I've edited the log because it's otherwise hard to send in a
> mail, the x2c( '5B1B5C5B0315181A5D' ) stands for what I sent: 
> 
> "[" ESC "\[" ETX NAK CAN SUB "]"
> 
> That's a General-Address-Literal as specified in 2821bis-03
> if I didn't screw up.  The tested mail server didn't like it.
> 
> Your backup MX also doesn't like it (501).  Hector's MX does
> not like it (501).  How many do we need to declare NO-WS-CTL
> as neither implemented nor a good idea ?

Assume for a moment that your apparent assumption that anything
is permitted in a general-address-literal is correct.  You are
assuming that, because something is permitted by the syntax, it
is valid.  If that criterion is applied, lots of what is in 821
and 2821 will fail.  This might, possibly, be a topic under
Issue 24 (general ABNF cleanup), but it isn't inherently a
problem with the spec because, as far as you know, the server is
checking that the string isn't a valid IPv4 address literal nor
a valid IPv6 literal and then returning a 550 code.  From that
point of view, the most serious thing you have demonstrated here
is that a client needs to act on the basis of the response code
and that associated text cannot be taken too seriously.  But
text that says exactly that is already in the spec... and was
already in 821.

In reality, it isn't even that bad.    The text of 2821 says:

      General-address-literal = Standardized-tag ":" 1*dcontent
      Standardized-tag = Ldh-str
            ; MUST be specified in a standards-track RFC
            ; and registered with IANA

So, first, your test string doesn't start with a _standardized_
tag, so it is invalid syntax.  If it did, that standardized tag
would need to be followed by a colon.  No colon appears, so the
syntax is invalid.  And the only tags that can be standardized
and be consistent with the syntax must be conformant to Ldh-str,
and Ldh-str is defined by 

      Let-dig = ALPHA / DIGIT
      Ldh-str = *( ALPHA / DIGIT / "-" ) Let-dig

So, unless we have screwed up ALPHA or DIGIT, no string of
control or escape characters are permitted in it.

Sounds like a syntax error to me.

Worse, even a full ABNF rewrite can't fix this because the
specification is intended to be extensible.  One could try to
have a server distinguish between 

   550 the syntax is clearly bogus

and

   550-I do not understand the prefix you used, possibly
   550-because it isn't standardized, and hence cannot
   550 figure out whether the putative address is valid

but 2821 and 821 says that, whether I do that or not, the result
is the same, since the client isn't supposed to pay attention to
anything but the code.

Now, one could construct a case that, if the extended reply code
extension is being used, then these two cases ought to generate
different codes.  But that is not a matter for 2821.

>> Beyond that, a decision on this is up to Tony
> 
> Tony's MX at messagelabs accepts the EHLO, but doesn't like
> RCPT TO:<postmaster> - I got a "544 No @ in address (#5.1.3)"

That is a bug: acceptance of "postmaster" without a domain was a
very clear DRUMS decision based on consensus that was required
by 821.

>> if interoperability reports (which are not part of _my_
>> issue list) indicate that the syntax for IPv6 literals is
>> not supported by at least two independent implementations,
>> that syntax can be removed.
> 
> We're not talking about exactly the same thing, the "general
> address literal" isn't for IPv6, it's for a 3986 "IPvFuture".

Yes.  I unfortunately answered from memory and thought we had
defined the construction, more or less (and deliberately using
different production names), as 
   address-literal = ipv4-literal / other-literal
   other-literal = literal-prefix ":" stuff
   literal-prefix = "IPv6" / ldh-string
It wasn't done that pay, presumably so that one could assign a
specific syntax to the IPv6 literal string.

> I'd also love to adopt the IPv6 syntax from RFC 3986, with or
> without tag, but that would be a separate issue, and of course
> we wouldn't break what works only for "aesthetical" reasons.

I believe it would also be out of scope and this is not the only
place where the syntax permitted or required by SMTP is
different from that of the URI spec.

      john

Date: Mon, 30 Apr 2007 07:16:39 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Retries - may be related to closed issue 12  4yz behavior
Sender: [email-masked]

--On Monday, 30 April, 2007 04:34 -0400 Hector Santos
<[email-masked]> wrote:

> 
> I would like to hear comments on the idea of retries using the
> same ENV information, in particular the same reverse-path?
> 
> In regards to the 4yz response,
> 
>    4yz  Transient Negative Completion reply
>       The command was not accepted, and the requested action
> did not
>       occur.  However, the error condition is temporary and
> the action
>       may be requested again.  The sender should return to the
> beginning
>       of the command sequence (if any).  It is difficult to
> assign a
>       meaning to "transient" when two different sites
> (receiver- and
>       sender-SMTP agents) must agree on the interpretation.
> Each reply
>       in this category might have a different time value, but
> the SMTP
>       client SHOULD try again.  A rule of thumb to determine
> whether a
>       reply fits into the 4yz or the 5yz category (see below)
> is that
>       replies are 4yz if they can be successful if repeated
> without any
>       change in command form or in properties of the sender or
> receiver
>       (that is, the command is repeated identically and the
> receiver
>       does not put up a new implementation.)
> 
> I think the above has an implied consideration that suggest
> when the server issues the 4yz response, the error is only
> temporary and thus the client may issue the request again with
> the idea the parameters are the same.

Yes.

> Why do I bring this up?
> 
> Well, I seen a SMTP client from Google that is getting
> greylisted with a 451 error but it is retrying (in new
> sessions) with a different MAIL FROM (reverse-path).

>From the standpoint of 2821 (and 821) this is a different
message.  Not that makes any difference, since neither
specification has any model of keeping server state between mail
sessions.

> Now, I don't wish to debate the greylist idea that many
> systems are using, but in general, the idea of what is
> expected when an 4yz is issued.
> 
> I am not sure if this should be part of the spec but the above
> text is written with semantics that the CLIENT is expected to
> try again with the same information at some later point.

yes
 
> I am wondering if there is any good reason to provide insight
> as such.
> 
> In the case of Google, today, I was helping my wife set up a
> blog at Google's blogger.com and it was doing an email
> verification.  It wasn't coming in so I checked the logs.
> 
> It has tried 3 times in the last 24 hours.
> 
> I know for sure it was the same transaction because our
> greylist system is implemented at the DATA stage which allows
> us to save the temporary rejected mail body for operator
> perusal.  I was able to compare the various email bodies it
> tried, and it was all the same email verification attempt.
> 
> I believe many Greylist implementations simply reject at the
> RCPT TO, and do not collect the body.

As far as 821/ 2821 are concerned, there is no vocabulary for
"collecting the body" (or any signature or approximation to the
body).  At the time a 4yz or 5yz (or a post-DATA 2yz) reply is
delivered, the server is finished.  2821 doesn't even specify
logging.

> Greylisted or not, if a server is temporarily rejecting an
> email transaction for typical normal (4yz) reasons as
> indicated in the example reply codes, I believer server
> implementations is basing this idea that the retry attempts
> will be made again with the same information.

Why?   Why (other than greylisting and other tactics) should the
server even care?  All it has said is "not now, try again
later.".

> Of course, IMO that is not a requirement, but I am wondering
> if there is any need to mention that in the above text.

Almost certainly not a "need" and, as the person who will be
blamed for the length of this document, I'm reluctant to add
text about the general character of mail practices for which
there is no RFC-documented theoretical or architectural
foundation.

       john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Date:  Mon, 30 Apr 2007 11:48:43 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

Hector Santos wrote:

> We can view that as a responsibility, but I see that more
> like a technical requirement for the protocol itself.

As long as they don't drop mail for frivolous reasons - IIRC
a system crash is cited as an example for a frivolous reason -
I don't care if that's technical or political or both.

> CBV works on the premise that SMTP mandates that the Return-
> Path MUST be valid.

CBV is an odd heuristics, if both sides go to the permitted
timeout I'm not sure that it can work, in theory.  A "valid"
Return-Path doesn't imply that you can talk to the sender
while it's sending.  About 12 years ago single tasking O/S
still existed.

I'd agree that CBV is better than "odd IPs only on Tuesdays",
and it's also better than accept - spamcheck - bounce, but
it's not good enough to be mentioned in 2821bis.

The "MUST take responsibility" IMO means "I must report an
error if there's a good chance" (1 permille is a rather good
chance) "that this is legit mail".  And for obvious reasons
you cannot bounce everything, if 80% is spam with forged
return paths.

IMO the "MUST take responsibility" includes "SHOULD refuse
to take responsibility" (= reject) if something's dubious,
otherwise the receiver is trapped.  Bounces hit innocent
bystanders, dropping mail kills legit mail.  Even simple
recipes like "let's at least drop SPF FAIL silently" will
delete legit mails (e.g. from a forwarder not using SRS).

The only realistic chance is reject at the border.  What
you do later is hopeless, whatever you do, you lose.  If
you lose anyway, bounce please, maybe use a separate IP -
"MUST take responsibility" might require two IPs, one for
the bounces.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-01 Issue 7: Use of 2119 definitions for conformity language
Date:  Mon, 30 Apr 2007 11:18:34 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Tony Hansen wrote:
 
> In -03, there was a bunch of MUSTard (MUST/SHOULD/MAYs) added under
> issue 16 (changing informal statements about requirements to more
> formal, 2119-like, language). This was an alternative to issue 7 to
> switch fully to 2119. Since John posted this, there have been no
> comments whatsoever.

Issue 16 and issue 7 are AFAIK mostly unrelated, and I've mentioned
the question of 2119 keywords in other threads.  If you think that
16 and 7 are "alternatives" this idea should get its own issue number,
it's the first time that I hear that.  And I think it's wrong.  

Assuming that we're we talking about the same thing, getting rid of
the idiosyncratic terminology in 2.3 and using a 2119 reference like
numerous other RFCs.  

Often John assigned an issue number after some discussion.  I have
considered that as "it's now noted, so let's look for other bugs".

I haven't considered these issue numbers as an invitation to post
anything I wrote before they were assigned again, that would make
no sense.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: Re: RFC2821bis-02 Issue 30: Deprecate use of source routes completely and remove text
Date:  Mon, 30 Apr 2007 11:03:17 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Tony Hansen wrote:

> Since John formulated this summary and suggestion almost a week ago,
> there has been no comment. Nix, nada, zilch.

Hu ?  The question of return routes was discussed elsewhere, and IIRC
some folks supported to kill them.  For a value of "some" not smaller
than one.

They muddy the water because they can't work as expected when they're
generally ignored and/or not generated.  Nobody supported to revive
return routes (in a minimalistic fashion at the border or otherwise),
it won't work.  It's just syntactical junk that has to be accepted
and ignored when it comes from 821-SMTPs, if such systems still exist.

John proposed to put the cruft into F.2 also in another message, and
that got several reactions, or do I start to dream about 2821bis ?

Frank

P.S. to Arnt:  ACK, we can't go to the IESG with "foo.com" addresses,
they'd kill us with a reason, no matter who originally wrote the text.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and "command"
Date:  Mon, 30 Apr 2007 10:36:52 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Tony Hansen wrote:

>> It has been pointed out that Section 2.3 contains a definition
>> for "Reply" (in 2.3.11) but no parallel definition for
>> "Command".  Is this worth fixing?
 
> The consensus on this issue appears to be yes

If that was trigered by my first reply here:  I later came to the
conclusion that it's nonsense.  A useful defintion would copy text
from a later section, and one of those "later" sections is 2.4,
immediately below 2.3.11.

> Whether this is accomplished via ABNF changes or strictly via
> textual definitions is subject to other issues.

The command = hello / mail / rcpt / data / rset / ... proposal is
unrelated to this beast, that's needed for the ABNF cleanup. 

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: Re: RFC2821bis-02 Issue 22: Requiring removal of extra return-path headers in 4.4
Date:  Mon, 30 Apr 2007 10:43:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Tony Hansen wrote:

> I am considering this issue to be closed with no change.

IMO that still needs an informative reference to RFC 3834,
otherwise implementors might think that MUST means "maybe,
less important if it's for calendar software used as MUA."

Frank

Date: Mon, 30 Apr 2007 04:34:51 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Retries - may be related to closed issue 12  4yz behavior
Sender: [email-masked]

I would like to hear comments on the idea of retries using the same ENV 
information, in particular the same reverse-path?

In regards to the 4yz response,

   4yz  Transient Negative Completion reply
      The command was not accepted, and the requested action did not
      occur.  However, the error condition is temporary and the action
      may be requested again.  The sender should return to the beginning
      of the command sequence (if any).  It is difficult to assign a
      meaning to "transient" when two different sites (receiver- and
      sender-SMTP agents) must agree on the interpretation.  Each reply
      in this category might have a different time value, but the SMTP
      client SHOULD try again.  A rule of thumb to determine whether a
      reply fits into the 4yz or the 5yz category (see below) is that
      replies are 4yz if they can be successful if repeated without any
      change in command form or in properties of the sender or receiver
      (that is, the command is repeated identically and the receiver
      does not put up a new implementation.)

I think the above has an implied consideration that suggest when the 
server issues the 4yz response, the error is only temporary and thus the 
client may issue the request again with the idea the parameters are the 
same.

Why do I bring this up?

Well, I seen a SMTP client from Google that is getting greylisted with a 
451 error but it is retrying (in new sessions) with a different MAIL 
FROM (reverse-path).

Now, I don't wish to debate the greylist idea that many systems are 
using, but in general, the idea of what is expected when an 4yz is issued.

I am not sure if this should be part of the spec but the above text is 
written with semantics that the CLIENT is expected to try again with the 
same information at some later point.

I am wondering if there is any good reason to provide insight as such.

In the case of Google, today, I was helping my wife set up a blog at 
Google's blogger.com and it was doing an email verification.  It wasn't 
coming in so I checked the logs.

It has tried 3 times in the last 24 hours.

I know for sure it was the same transaction because our greylist system 
is implemented at the DATA stage which allows us to save the temporary 
rejected mail body for operator perusal.  I was able to compare the 
various email bodies it tried, and it was all the same email 
verification attempt.

I believe many Greylist implementations simply reject at the RCPT TO, 
and do not collect the body.

Greylisted or not, if a server is temporarily rejecting an email 
transaction for typical normal (4yz) reasons as indicated in the example 
reply codes, I believer server implementations is basing this idea that 
the retry attempts will be made again with the same information.

Of course, IMO that is not a requirement, but I am wondering if there is 
any need to mention that in the above text.

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: "for" clause on Received: header field 
Date:  Mon, 30 Apr 2007 10:23:22 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 32
Sender: [email-masked]

Kari Hurtta wrote:

>| Is multiple mailboxes ever used on "for" -clause ?
>| Syntax for it is quite inconsistent between standards.
 
> But perhaps implementation and interoperability report
> tells?

More "implementation" than "interoperabiity" if I understood
section 3.8.2 correctly, servers aren't supposed to look 
"into" the timestamp lines they get.  A big point for DKIM,
and a small point against EAI's "for8".

> Previous standards (RFC 821, 822) did not allowed several
> addresses, so is these multiple mailboxes on "for" clause
> never implemented?

If it's implemented it's not better than Apparently-To, as
noted in 2821 4.4, but the Apparently-To got a "SHOULD NOT".
Why allow an in essence identical damage in the for-clause ?

> Just wondering :-)

Me too.  I've just tested to send a mail from GMail to two
local parts of @xyzzy, and _apparently_ both Google and my 
ISP noted only the first RCPT TO in the for-clause.  Maybe
that's too naive as test (?)  I've no clue what Google uses
as their mailout (something that uses DKIM and Domainkeys),
at my ISP it's Exim (two versions).

Frank

Date: Mon, 30 Apr 2007 10:16:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Recap: Re: RFC2821bis-02 Issue 30: Deprecate use of source routes completely and remove text
Cc: Tony Hansen <[email-masked]>
Sender: [email-masked]

Tony Hansen writes:
> Since John formulated this summary and suggestion almost a week ago, 
> there has been no comment. Nix, nada, zilch.

I like the text as it is in -03. It reflects common wisdon/best common 
practice. I'm somewhat aftraid that if the text is removed, someone 
will jump to the wrong conclusion and for 2821ter we'll have another 
thread like this round's 1xy thread.

There is some use of foo.org and bar.org, which aren't blessed by RFC 2606.

Arnt

To: [email-masked]
From: Kari Hurtta <[email-masked]>
Subject:  "for" clause on Received: header field (Re: General-Address-Literal (was: Issue list))
Date:  30 Apr 2007 09:21:55 +0300
Lines: 48
Cc: Kari Hurtta <[email-masked]>
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes in gmane.ietf.smtp:

> John C Klensin wrote:
> 
> >> Kari  (2007-04-20): "for" clause on Received: header field
> >> http://permalink.gmane.org/gmane.ietf.smtp/5708
>  
> > As far as Pete and I are concerned, this has been resolved 
> > by further stripping the Received line syntax from 2822.
> 
> Okay, I had Kari's article still on "read again". and didn't
> look if 2821upd already did something with this issue.  
>  

Yes. I asked:

| Is multiple mailboxes ever used on "for" -clause ?
| Syntax for it is quite inconsistent between standards.

But perhaps implementation and interoperability report
tells?

Previous standards (RFC 821, 822) did not allowed several 
addresses, so is these multiple mailboxes on "for" clause 
never implemented?

Multiple mailboxes was new feature on RFC 2821, 2822.

Just wondering :-)

Syntax error what I noted on 
subject "draft-klensin-rfc2821bis-03.txt: For -rule"
Klensin said that it is now fixed.

draft-resnick-2822upd-01.txt does now

   received        =       "Received:" *received-token ";" date-time CRLF

   received-token  =       word / angle-addr / addr-spec / domain

So there is no conflict between draft-klensin-rfc2821bis
and draft-resnick-2822upd.  (EAI WG probably needs (and
probaly not need) catch up this, but that does not belong 
to here. )

/ Kari Hurtta

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap Issues 0b/21/25
Date:  Mon, 30 Apr 2007 08:09:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

Tony Hansen wrote:

 [proposed text]
> This specification does not deal with the verification of return
> paths for use in delivery notifications. Recent work, such as
> [add REFERENCES here], has been done in providing ways to
> ascertain that an address is valid or belongs to the person who
> actually sent the message. A server MAY attempt to verify the return
> path before using its address for delivery notifications, but
> methods of doing so are not defined here nor is any particular
> method recommended at this time.

> Thoughts on an addition like this?

Looks good, and one of the references (RFC 4409 "enforced submission
rights") is even not exactly "recent work".  The other 4408 reference
is strictly based on a "voluntary opt-in" mode, i.e. the domain owner
of the alleged sender and the postmaster of the receiver volunteer to
opt-in their users, or they don't.  (No sarcasm, IMO that's how it is)

Whether those references are added or not, I still think the general
SMTP model has to be "tuned" for this millennium.  All the "MUST NOT
reject" should be toned down (or removed if the problem they tried to
solve is meanwhile solved).  Any "SHOULD bounce" has to be put under
a magnifying glass, there's a clear "MUST bounce" somewhere, so any
additional "SHOULD bounce" rules might be a bad idea.  Especially if
they try to emulate "selective reject" by bouncing... :-(

And the "MUST accept responsibility" + "MUST bounce to originator"
have to state clearly that this can't work if the originator is in
fact unknown or at least dubious.  There are situations where the
originator should be known (on the sending side from MSA to mailout),
by implementing 4409 6.1 or similar magic, otherwise the MSA is an
open relay and/or needs to be fixed.

And there are many other situations where the originator is known or
at least plausible (mail from a known "good" source, SPF PASS, etc.)

But there's only one place for the receiving side to decide this, at
the border MTA.  Later is too late, it can cause bounces to innocent
bystanders, or - if the mail is dropped - it loses legit mail (in
the case of a "false positive").

While that's "obvious" after reading 2821 again and again and between
the lines the actual prose hides this flaw, it pretends that delayed
bounces are no problem and therefore accept is always the right thing
if there's a minimal chance that it could work out.

But that'a not true.  Everybody HERE knows that it's not true.  Many
readers out there missed that carefully hidden hole.  If it's not
possible to fix a bug immediately the minimum that must be done is
to document this bug as clear and good as possible, and to mitigate
the undesirable effects.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  General-Address-Literal (was: Issue list)
Date:  Mon, 30 Apr 2007 07:22:23 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 60
Sender: [email-masked]

John C Klensin wrote:

>> Kari  (2007-04-20): "for" clause on Received: header field
>> http://permalink.gmane.org/gmane.ietf.smtp/5708
 
> As far as Pete and I are concerned, this has been resolved 
> by further stripping the Received line syntax from 2822.

Okay, I had Kari's article still on "read again". and didn't
look if 2821upd already did something with this issue.  
 
>> http://permalink.gmane.org/gmane.ietf.smtp/5810
 
> This note is a confusion of issues.  If you want an issue number
> assigned to one or more of the issues in it, post separate notes
> with meaningful subject lines.

The subject line was General-Address-Literal, I try it again.

Here's my "implementation and interoperability test" result:

using SPAMVIR.DE.CLARA.NET for xyzzy.claranet.de
20070430 06:50:34 TCP connection with SPAMVIR.DE.CLARA.NET:25
50:34.14 220 Claranet Germany Mail Service
50:34.14 ehlo x2c( '5B1B5C5B0315181A5D' )
50:34.26 501 Syntactically invalid EHLO argument(s)
50:34.26 quit
50:34.39 221 spamvir07.de.clara.net closing connection

I've edited the log because it's otherwise hard to send in a
mail, the x2c( '5B1B5C5B0315181A5D' ) stands for what I sent: 

"[" ESC "\[" ETX NAK CAN SUB "]"

That's a General-Address-Literal as specified in 2821bis-03
if I didn't screw up.  The tested mail server didn't like it.

Your backup MX also doesn't like it (501).  Hector's MX does
not like it (501).  How many do we need to declare NO-WS-CTL
as neither implemented nor a good idea ?

> Beyond that, a decision on this is up to Tony

Tony's MX at messagelabs accepts the EHLO, but doesn't like
RCPT TO:<postmaster> - I got a "544 No @ in address (#5.1.3)"

> if interoperability reports (which are not part of _my_
> issue list) indicate that the syntax for IPv6 literals is
> not supported by at least two independent implementations,
> that syntax can be removed.

We're not talking about exactly the same thing, the "general
address literal" isn't for IPv6, it's for a 3986 "IPvFuture".

I'd also love to adopt the IPv6 syntax from RFC 3986, with or
without tag, but that would be a separate issue, and of course
we wouldn't break what works only for "aesthetical" reasons.

Frank

Date: Mon, 30 Apr 2007 00:54:01 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Issue list
Sender: [email-masked]

Inventing new approaches is out of scope. John's comments are correct
with respect to how to move this forward to Draft Standard.

	Tony

John C Klensin wrote:
> Beyond that, a decision on this is up to Tony, but I believe
> that a request to _change_ the fundamental syntax used to
> express different versions of IP and the associated literals is
> out of scope.  I believe that we can make syntax changes at this
> point to clarify or correct what is there, but not to invent new
> (to 2821/2822) approaches.
> 
> Of course, if interoperability reports (which are not part of
> _my_ issue list) indicate that the syntax for IPv6 literals is
> not supported by at least two independent implementations, that
> syntax can be removed.  But that would give us no standard
> support for IPv6 literals at all, not an opportunity to invent a
> new syntax at this time.  Just IMO, of course.

Date: Mon, 30 Apr 2007 00:27:57 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-01 Issue 7: Use of 2119 definitions for conformity language
Sender: [email-masked]

I'm working through the issues here.

In -03, there was a bunch of MUSTard (MUST/SHOULD/MAYs) added under
issue 16 (changing informal statements about requirements to more
formal, 2119-like, language). This was an alternative to issue 7 to
switch fully to 2119. Since John posted this, there have been no
comments whatsoever.

I'm closing issue 7 in favor of issue 16.

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> 
> 
> --On Sunday, 25 March, 2007 03:20 +0200 Frank Ellermann
> <[email-masked]> wrote:
> 
>> Tony Finch wrote (2005-09-05)
>>
>>> On Sat, 3 Sep 2005, John C Klensin wrote:
>>>> The specific difficulty arises in numbered paragraph 6 of RFC
>>>> 2119, which, to save chasing references, reads: [...]
>>  
>>>> RFC 2821 imposes a number of requirements that, by some
>>>> interpretations, are "not actually required for
>>>> interoperation".
>>  
>>> I think the other part of 2119's guidance - "to limit behavior
>>> which has potential for causing harm" - allows the stricter
>>> requirements of 2821.
>>> In any case, paragraph 6 of 2119 is a meta-requirement about
>>> IETF documentation standards, rather than about the contents
>>> of those documents - it makes recommendations for writers not
>>> readers. Stripping off the rationale doesn't change the
>>> definition of the keywords.
>> In that thread Tony, Wayne, John Leslie, and me argued to use
>> the 2119 keywords instead of 2181's idiosyncratic definitions,
>> while John Klensin argued that 2119 keywords can only be used
>> "where it is actually required for interoperation".
>> ...
> 
> Ok, I don't feel nearly as strongly about this as I did during
> the DRUMs period, so will do what the group wishes.  However,
> I'd encourage everyone to remember that the choice made during
> DRUMs, leading to the current arrangements, was motivated by two
> things:
> 
> (1) Consistency with 821 and 1123 (the local definitions are
> very nearly a copy of those in 1123 if I recall).
> 
> (2) Some rather loud and intense claims (accompanied by some
> name-calling) by one participant in the WG.   My interpretation
> of those claims was that, if under the 2119 definitions, some of
> the restrictions and specific language in what became 2821 would
> be impossible because his implementation didn't do what was
> expected and the Internet had not completely melted down as a
> result, or at least email still worked.  The non-2119 language
> was used to avoid that particular battle, or at least some of it.
> 
> I'm assigning issue #7 to this one.
> 
>      john
> 

Date: Mon, 30 Apr 2007 00:00:31 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Recap: Re: RFC2821bis-02 Issue 30: Deprecate use of source routes completely and remove text
Sender: [email-masked]

Since John formulated this summary and suggestion almost a week ago,
there has been no comment. Nix, nada, zilch.

It does reflect suggestions made in passing in several other messages,
but I feel that it is important enough that needs to be an explicit
decision made on the suggestion.

If there are no further messages on this issue, the *default* decision
will be "no change".

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> Hi.
> 
> This has been discussed, but just to give it a number and insure
> that it doesn't fall through the cracks...
> 
> 2821 sort of danced around the issue of source routes, going a
> little further than 1123 did in discouraging their use but not
> getting rid of them entirely.  The method used to do this in
> 2821 is a little fuzzy, since it left the text in place
> instructing relays to copy their own domains into the
> backward-pointing address.  That was discussed as issue 26 and
> the working draft for -03 now has that issue straightened out
> (assuming I've gotten it right).
> 
> However, that discussion raised another possibility, which is
> that now is the time to deprecate source routes entirely.  More
> precisely that would:
> 
> 	(i) Leave the syntax in place and continue to require
> 	that systems be able to accept and parse it.
> 	
> 	(ii) Forbid systems from using source routes, either
> 	changing the "they should be ignored but may be used"
> 	language that now appears in Section 3.3 (and more or
> 	less repeated in 3.6) to forbidding their use for
> 	routing or leaving those two sections alone (I consider
> 	eliminating the redundancy to be an editorial matter if
> 	I can figure out how to do it).
> 	
> 	(iii) Continue with (ii) by changing the "SHOULD not
> 	generate" in the second paragraph of 4.1.1.3 to MUST NOT.
> 	
> 	(iv) Get rid of Appendix C, replacing it with some
> 	handwaving to the effect that, if people really need to
> 	know how this obsolete feature worked, they should read
> 	821.
> 	
> 	(v) Strengthen the prohibitions in Appendix F.2.
> 
> So far, we've heard from a couple of people on-list who feel
> that this is a much better idea than the patching contemplated
> by issue 26 and no one who has argued for keeping them.  Whether
> it is realistic to further deprecate them or not probably
> depends on interoperability testing although we could arguably
> prohibit the things as a bad idea that is not widely supported.
> 
>      john
> 
>     
> 

Date: Sun, 29 Apr 2007 23:50:35 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

Tony Hansen wrote:
> The consensus on this text
> 
>   It is worth noting that the file transfer protocol (FTP [35]) uses a
>   very similar code architecture and that the SMTP codes are based on
>   the FTP model.  However, SMTP uses a one-command, one-response model,
>   while FTP is asynchronous and the 1yz codes are not part of the SMTP
>   model.
> 
> is almost dead even. I'll leave this one open for two more days, then
> give a consensus call. If it remains even, it will become "editor's choice".

This is all minor to me so I can go either way, but........

I voted for the removal on the basis that the 1yz is no longer part of 
the spec and that throwing in 1yz may raise more questions than not for 
new developers, especially those who only interest is SMTP and have not 
other protocol perspective to lean on.

I believe, if I read John right, it was stated as such to explain the 
removal of 1yx - "where did it go?"  In my view, that is more logical if 
the 1yz description was not removed. In this case, as it is stated, it 
makes more sense to explain why it should only be used for extensions.

Alternatively, IMO, I think it is may be more acceptable if a rewording 
was stated as a theory basis in section 4.2.1 as the initial paragraph.

Example:

   4.2.1.  Reply Code Severities and Theory

   SMTP inherits the reply code model and theory from the file
   transfer protocol (FTP [35]).  However, as described in this
   section, only specific reply-code ranges are acceptable.

or

   4.2.1.  Reply Code Severities and Theory

   The SMTP reply code model uses a similar C/S (client/server)
   command response architecture that is based on early
   C/S protocol designs such as file transfer protocol (FTP [35]).

I like the latter because it makes no assertion that the SMTP reply 
codes are based on FTP, but based on similar frameworks such as FTP.

I think the overall issue is that no assertion or implication should be 
made to the reader that he/she should research FTP to understand SMTP 
better.

My opinion, of course.

--
HLS

Date: Sun, 29 Apr 2007 23:42:10 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Recap: Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and "command"
Sender: [email-masked]

John C Klensin wrote:
> It has been pointed out that Section 2.3 contains a definition
> for "Reply" (in 2.3.11) but no parallel definition for
> "Command".  Is this worth fixing?

The consensus on this issue appears to be yes. Whether this is
accomplished via ABNF changes or strictly via textual definitions is
subject to other issues.

	Tony Hansen
	[email-masked]

Date: Sun, 29 Apr 2007 23:38:07 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

I think someone suggested moving the comment to Appendix F Deprecated
Features. I can't find the suggestion right this minute though.

	Tony

John C Klensin wrote:
> 
> --On Sunday, 29 April, 2007 23:04 -0400 Tony Hansen
> <[email-masked]> wrote:
> 
>> The consensus on this text
>>
>>   It is worth noting that the file transfer protocol (FTP
>> [35]) uses a   very similar code architecture and that the
>> SMTP codes are based on   the FTP model.  However, SMTP uses a
>> one-command, one-response model,   while FTP is asynchronous
>> and the 1yz codes are not part of the SMTP   model.
>>
>> is almost dead even. I'll leave this one open for two more
>> days, then give a consensus call. If it remains even, it will
>> become "editor's choice".
> 
> Speaking personally (not as editor), I'm slightly inclined to
> leave it, simply because dropping of the 1yz text would seem odd
> without _some_ comment.  Very slightly, however.
> 
>      john

Date: Sun, 29 Apr 2007 23:33:38 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Recap: Re: RFC2821bis-02 Issue 22: Requiring removal of extra return-path headers in 4.4
Sender: [email-masked]

On this issue, I find the argument against changing running code to be
compelling. And since there have been no further arguments for or
against this issue, I am considering this issue to be closed with no change.

	Tony Hansen
	[email-masked]

Date: Sun, 29 Apr 2007 23:32:22 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

--On Sunday, 29 April, 2007 23:24 -0400 Tony Hansen
<[email-masked]> wrote:

> The consensus on the basic issue of changing the wording of
> this paragraph to include the word MUST:
> 
> 	The responsibility of an SMTP client is to transfer mail
> 	messages to one or more SMTP servers, or report its
> 	failure to do so.
> 
> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.
> 
> is positive.
> 
> However, several people raised issues with some ambiguities in
> the text. I found Dave Crocker's description of the
> ambiguities to be the most succinct. These ambiguities do
> exist whether the MUST is there or not, so they pass my "chair
> filter", so should probably be dealt with.

Text would be welcome.
    john

Date: Sun, 29 Apr 2007 23:30:00 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

--On Sunday, 29 April, 2007 23:04 -0400 Tony Hansen
<[email-masked]> wrote:

> 
> The consensus on this text
> 
>   It is worth noting that the file transfer protocol (FTP
> [35]) uses a   very similar code architecture and that the
> SMTP codes are based on   the FTP model.  However, SMTP uses a
> one-command, one-response model,   while FTP is asynchronous
> and the 1yz codes are not part of the SMTP   model.
> 
> is almost dead even. I'll leave this one open for two more
> days, then give a consensus call. If it remains even, it will
> become "editor's choice".

Speaking personally (not as editor), I'm slightly inclined to
leave it, simply because dropping of the 1yz text would seem odd
without _some_ comment.  Very slightly, however.

     john

Date: Sun, 29 Apr 2007 23:24:15 -0400
From: Tony Hansen <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Recap: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

The consensus on the basic issue of changing the wording of this
paragraph to include the word MUST:

	The responsibility of an SMTP client is to transfer mail
	messages to one or more SMTP servers, or report its
	failure to do so.

	In either case, a formal handoff of responsibility for
	the message occurs: the protocol requires that a server
	MUST accept responsibility for either delivering a
	message or properly reporting the failure to do so.

is positive.

However, several people raised issues with some ambiguities in the text.
I found Dave Crocker's description of the ambiguities to be the most
succinct. These ambiguities do exist whether the MUST is there or not,
so they pass my "chair filter", so should probably be dealt with.

	Tony Hansen
	[email-masked]

Date: Sun, 29 Apr 2007 23:19:46 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Tony Hansen wrote:

> However, I'd like to follow the lead of the 3463 reference found in
> section 2.3.11, and suggest that some wording be added, probably to
> section 3.6, along these lines:
> 
>     This specification does not deal with the verification of return
>     paths for use in delivery notifications. Recent work, such as
>     [add REFERENCES here], has been done in providing ways to
>     ascertain that an address is valid or belongs to the person who
>     actually sent the message. A server MAY attempt to verify the return
>     path before using its address for delivery notifications, but
>     methods of doing so are not defined here nor is any particular
>     method recommended at this time.
> 
> Thoughts on an addition like this? I know it doesn't go far enough for
> some people, and probably goes too far for some other people. So I'm
> offering it as a possible way forward.

+1 and -1

Why +1?

I suggested this type of insight for 28211bis in my very first input on 
2821bis.  I referred to the existing similar insight regarding the 
reverse-path and forwarding path in section 3.3

    3.3.  Mail Transactions

    ..... Despite the apparent
    scope of this requirement, there are circumstances in which the
    acceptability of the reverse-path may not be determined until one or
    more forward-paths (in RCPT commands) can be examined.  In those
    cases, the server MAY reasonably accept the reverse-path (with a 250
    reply) and then report problems after the forward-paths are received
    and examined.  Normally, failures produce 550 or 553 replies.

Here we have two insights for validation; one for the forward-path and 
one regarding the acceptability of the reverse-path.

Note, this does not imply validating the forward-path first should be 
the only basis for accepting the reverse-path.  It can also imply that 
you may not wish to perform any overhead on checking the reverse-path 
until you know there is going to be a payoff with a valid forward-path.

The latter is how we implement the concept to reduce return path 
validation by 60% and I believe, others like Scott Kitterman who is 
considering recommending this optimization concept for SPF as well.

The proof of concept is simple:

Accepting a local forward-path without authentication is the industry 
BCP.  For a remote forward-path, requiring authentication is the 
industry BCP, otherwise you may have open-relay entry points and 
accept/bounce attack threat entry points.

In any case, I think it is very important, IMO, that the strong point 
that the reverse-path MUST be valid before it can serve any usefulness 
is stated, and that is how this is done is out of scope.

Why -1?

The problem with your 3463-like suggestion is that it creates the 
dilemma and question of "WHEN" should the reverse-path be considered valid?

The specification mandates a MUST for reverse-path validity.  CAN-SPAM 
also mandates it as well.

So the question is?

When is the VALIDITY of the reverse-path important?

     At the moment it is presented?  or
     At the moment it is required to be used - a BOUNCE action?

The latter is what 3463 may imply and create a design model using this 
concept - which in my view is not quite safe in today's world.

--
HLS

Date: Sun, 29 Apr 2007 23:04:30 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

The consensus on this text

  It is worth noting that the file transfer protocol (FTP [35]) uses a
  very similar code architecture and that the SMTP codes are based on
  the FTP model.  However, SMTP uses a one-command, one-response model,
  while FTP is asynchronous and the 1yz codes are not part of the SMTP
  model.

is almost dead even. I'll leave this one open for two more days, then
give a consensus call. If it remains even, it will become "editor's choice".

	Tony Hansen
	[email-masked]

Date: Sun, 29 Apr 2007 22:28:10 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap Issues 0b/21/25
Sender: [email-masked]

I've been rereading all of the discussions around issues 0b/21/25. Some
of the discussion wants to require verification of the sender before
ever sending an non-delivery notification.

While this does reflect what some implementations are doing and is the
effort of some recent standards and non-standards efforts, it is not
common deployment. So I'm going to make a ruling that requiring such
validation goes too far in the direction of a "new behavior" requirement.

However, I'd like to follow the lead of the 3463 reference found in
section 2.3.11, and suggest that some wording be added, probably to
section 3.6, along these lines:

    This specification does not deal with the verification of return
    paths for use in delivery notifications. Recent work, such as
    [add REFERENCES here], has been done in providing ways to
    ascertain that an address is valid or belongs to the person who
    actually sent the message. A server MAY attempt to verify the return
    path before using its address for delivery notifications, but
    methods of doing so are not defined here nor is any particular
    method recommended at this time.

Thoughts on an addition like this? I know it doesn't go far enough for
some people, and probably goes too far for some other people. So I'm
offering it as a possible way forward.

	Tony Hansen
	[email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Editorial ABNF oddities (was: 2821bis-03:   Receive line From-domain Address-literal usage)
Date:  Sun, 29 Apr 2007 23:13:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

John C Klensin wrote:

 [blank line in ABNF]
> fixing this sort of thing can also be left for the RFC Editor

I think syntax errors in the ABNF have to be fixed before the 
IESG review, otherwise it would get a [DISCUSS].  At least that
was how it was handled for RFC 4646.  And for RFC 4647 even an
"ABNF-like" comment in the prose got a [DISCUSS], because it
wasn't consistent with two similar ABNF literals in this memo
(bad %2A vs. good %x2A). 

Could you publish the xml source for the drafts ?  It's simple
to catch ABNF typos if you use <figure><artwork type="abnf">
and a global <?rfc strict="yes" ?> directive.

There's a subtle LWSP issue in the core ABNF:  If the apparently
empty line in your <address-literal> is really empty, then it's
an error, Bill's parser says:

| 2:0: error: state 68, token CRLF: parse error, unexpected CRLF

If the apparently empty line contains trailing white space it
is okay, and Bill's parser says "No errors during parsing." 

Some folks thought that promoting 2821 to DS is a short trip,
and "somebody else" thought that promoting 4234 to STD is a mere
formality.  What really happened is that another person had to
write a new 4234 interop report (admittedly the old 2234 report
was a bit odd :-), the assumption that there are no 4234 errata
was wrong (the huge "pending errata mbox" contained something),
and now some months later there's a fresh 4234bis-00.

At least it's an opportunity to add a warning about LSWP to the
ABNF security considerations, this already hurt DKIM and 2831bis:
<http://permalink.gmane.org/gmane.ietf.general/24801>

Frank

Date: Sat, 28 Apr 2007 22:43:55 -0400
From: John C Klensin <[email-masked]>
To: Kari Hurtta <[email-masked]>, [email-masked]
Subject: Re: draft-klensin-rfc2821bis-03.txt: For -rule
Sender: [email-masked]

--On Saturday, 28 April, 2007 21:27 +0300 Kari Hurtta
<[email-masked]> wrote:

> 
> 
> draft-klensin-rfc2821bis-03.txt:
> 
>   For            = CFWS "FOR" 1*( FWS Path / Mailbox ))
> 
> should this be:
> 
>   For            = CFWS "FOR" 1*( FWS ( Path / Mailbox ))
> 
> 
> There is clearly one '(' missing.

Yep.  Fixed.  Thanks.
    john

Date: Sat, 28 Apr 2007 22:26:29 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: 2821bis-03:   Receive line From-domain Address-literal usage
Sender: [email-masked]

John C Klensin wrote:
> 
>> Quite possibly, in summary, it may be deemed a new issue,
>> possible two:
>>
>>  - Clarifying Bracketed address-literals MUST requirement, and
>>  - discouraging alterations of the address-literal for
>>        the trace line.
>>
>> The two may be related and possible be folded into 1 issue.
> 
> Or it may be no issue at all... I'll leave that up to Tony and
> general discussion.

Right, that was implied above.

>> The MUA client actually sent:
>>
>>      EHLO [192.169.1.101]
> 
> First of all, note that most MTAs who use a 
>     ... from domain-or-literal ( literal ) ...
> format are announcing that the actual IP address of the
> sender-SMTP was different from the IP address specified (or
> associated with the domain name given).  They are prohibited (in
> relays) and discouraged (in submission servers) from rejecting
> mail on that basis, but warnings have long been considered
> appropriate.
> 
> You, and others, have questioned that rule, both wrt NAT-based
> addresses and about whether it has become a proper basis for
> rejection.

Right, it is a separate issue (IP literal matching) but it was the 
genesis of finding this behavior with Google.

As you may recall, Thunderbird now offers a way to allow users to define 
the EHLO/HELO argument to address the above private ip literal issue 
with clients residing on private NAT networks.  That is how I was able 
to perform the test to define the different EHLO/HELO arguments.

But the configuration is on a per server basis.  When I added the Google 
  gmail.com account to my Thunderbird 2.0 MUA, it used the default 
behavior, thus the private IP literal for the EHLO command above.

> Now, my personal preference would have been to document this a
> little differently, partially because I don't like adding syntax
> violations to the complex of problems that are there already.
> But it is a matter of taste and a territory into which I don't
> think 2821bis needs to go or should go.  I think I would prefer
> that a server deal with situations like this one by inserting
> something more like
> 
>   Received: (from bogus-NAT-address 192.168.1.101
>       [68.215.50.125])
>       by mx.google.com with ESMTP id
>       h17sm2595511wxd.2007.04.27.19.07.38;
>       Fri, 27 Apr 2007 19:07:38 -0700 (PDT)
> 
> or maybe, taking liberties with a different rule,
> 
>   Received: from [68.215.50.125] 
>       (EHLO bogus NAT address 192.168.1.101)
>       by mx.google.com with ESMTP id
>       h17sm2595511wxd.2007.04.27.19.07.38;
>       Fri, 27 Apr 2007 19:07:38 -0700 (PDT)

and this is partly why I brought it.  The Received: line has been an 
issue that many have been trying to stabilize to increase the automated 
aspects of reading it.  It seems to me that many packages simply did not 
establish any reliable faith on using it for any automated concept.

> Looking at this example only for the moment and ignoring the
> other things you found out about Google's behavior, a
> receiver-SMTP that does this could be doing something creative
> with the syntax to note the fact that the smtp-Sender has
> violated 2821 by issuing an invalid EHLO command (because it
> uses a private address on the public Internet, not because of
> syntax) and it has decided to accept and transport the mail
> anyway.   I don't particularly like the syntax of their
> announcement that they have done so, but the standards violation
> occurs earlier and I have to approve of their attempt to
> document the violation --and the situation as it appears from
> their end-- carefully.

I see your point, but it seems to me that it might be better to have 
some "standard" way of documented that failure or not documenting it and 
just record "as is" what was provided and allow the post processing MFA 
(Mail filtering Agents) to react to it, if applicable.

Case in point:  SIEVE,  SPAM ASSASIN.  I though they expect to see 
questions marks.   So if it was an upfront error, should the receiver 
record this error or just record the information for others systems to 
read as it expect it to read it?   Remember, in this case, the error was 
not the syntax itself, but that it didn't match the connecting IP 
address.  But Google didn't record it as a IP mis-match error, but 
rather that it didn't understand the brackets.

>> 1) SMTP compliant bracketed address-address, IP matching
>>
>>    EHLO [68.215.50.125]
>>
>> 	--> Received: from ?68.215.50.125?
> 
> Bug, IMO.

Right, so does it make sense to add any instead that it should either 
record it "as is" or do we come up with other method to record the error?

Again, keep in mind that there are MFAs (Mail Filtering Agents) that may 
not expect the questions marks for address-literals as part of 
from-domain field.

>> 2) No bracketed address-address, IP matching
>>
>>    EHLO 68.215.50.125
>>
>> 	--> Received: from 68.215.50.125
> 
> Worse bug, IMO.  But, of course, this is also a client violation.

Right.

>> 3) Simulate what Outlook would do (sent computer name)
>>
>>    EHLO hdev1
>>
>> 	--> Received: from hdev1
> 
> Different bug, but also a client violation.

For which unfortunatly, servers has to shallow if they don't wish to 
alienate one of the largest pool of end users.

>> 4) Use the real/correct ptr domain
>>
>>    EHLO adsl-215-50-125.mia.bellsouth.net
>>
>> 	-->  Received: from adsl-215-50-125.mia.bellsouth.net
> 
> behaving as expected.
> 
>> 5) Use the incorrect ptr domain (125 changed to 126)
>>
>>    EHLO adsl-215-50-126.mia.bellsouth.net
>>
>> 	-->  Received: from adsl-215-50-125.mia.bellsouth.net

Sorry, on the received: output, it should of been:

     Received: from adsl-215-50-126.mia.bellsouth.net  	

 > With no IP address in brackets?   Clearly permitted by 2821 (and
 > 821), but probably not in particularly good taste today.   Still
 > not a 2821bis issue, IMO.

The client behavior SHOULD be:

   - Obtain the IP address on the machine,
   - Obtain the domain for this IP (PTR lookup),
   - If none, use bracketed IP literal.

For test #4, I simulated the idea that the NAT machine was sending the 
transaction by using the proper domain for the public IP of this machine.

For test #5, to check to see if Google was performing an domain/IP 
match, like many ISP servers do, I simulated an altered domain name 
(changing 125 to 126) to see if Google was going to do domain/ip check. 
  It did not.

> Or might reflect the more general possibility that someone at
> gmail is asleep at the switch.

I wonder if an gmail.com engineer is lurking here. I'll see if I can 
find an appropriate contact address to send the note.

>> Why or how could any of this constitute [an issue] for 2821bis?
> 
>> Well, in our quest for clarifying and codifying existing and
>> possible dominant behavior,  for new implementors what I can
>> see occurring is they begin may use systems like GOOGLE
>> gmail.com as a design baseline for practical and good
>> 2821/2822 protocol behavior. They may follow their lead,
>> including not expecting brackets and also converting them to
>> questions marks for the trace line.
> 
>> Comments anyone?
> 
> This is the same problem that we had when some people were
> assuming that whatever sendmail was doing was the spec, without
> looking at the spec, 20 years ago.  It was a problem then, it is
> a problem now... only the previous bad actor has cleaned up its
> act and the new bad actor has.  It is not a problem with the
> standard, IMO, unless someone can go find the relevant gmail
> people and get them to tell us that their trace field
> implementation ended up this way because they got confused by
> the spec.  Otherwise, this is interesting, and sad, but not a
> problem.

I agree.  Its the same issue as with the reply codes.  The difference 
being is that the BCP is not to reject or more appropriate fail the 
transaction on this basis.  Hence its hasn't been a big issue.

But thats changing. In the name of spam/security, that is clearly changing.

I just figured maybe there would be a need to embedded, "fit" a few 
highlighted technical MUST about the protocol expects.

I'm just winging it now, but maybe something like:

   Please note, SMTP-clients MUST enclose address-literals with
   brackets.

and,

   SMTP-Receivers MUST|SHOULD use the address-literals unmodified when
   recoding the From-Domain field in the Received: header.

Something simple, something to clarify what is already "implied" in the 
specs.

Thanks

--
HLS

To: [email-masked]
From: Kari Hurtta <[email-masked]>
Subject:  draft-klensin-rfc2821bis-03.txt: For -rule
Date:  28 Apr 2007 21:27:53 +0300
Lines: 15
Cc: Kari Hurtta <[email-masked]>
Sender: [email-masked]

draft-klensin-rfc2821bis-03.txt:

  For            = CFWS "FOR" 1*( FWS Path / Mailbox ))

should this be:

  For            = CFWS "FOR" 1*( FWS ( Path / Mailbox ))

There is clearly one '(' missing.

/ Kari Hurtta

Date: Sat, 28 Apr 2007 13:51:07 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis status report
Sender: [email-masked]

--==========7259A80B544293DE6E57==========
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi.

I've just done a review of the state of the document compared
with what needs to be done, the original "charter" note, etc.  

These comments are only the personal opinion of the editor, but
I think we are at a fork and need to decide about what we want
to do and why.

We are, IMO, down to one major, substantive, issue, which is the
matter of how to handle NDNs in 2821bis in the light of various
models of sender authentication and what is going on in the rest
of the world.  Note that is a question about what should be done
with 2821bis, not a question about the general subject matter.
That distinction is very important, in part because of
constraints on the "2821bis to Draft" plan: independent of
debates about their merit, many of the ideas that have been
proposed would take 2821bis back to Proposed or earlier.  As
just one example, if one adopts some flavor of "no NDNs except
to authenticated senders" rule, then I think one must discuss,
probably normatively, how one does the authentication and what
is adequate or not.  One may also need to address techniques
(such as the one Hector proposed) for keeping an incoming
sender->relay connection alive while getting information about
what the relay->NextHop transaction would do.  Any such changes,
other than the vaguest of hand-waving, would almost certainly
constitute new work (of the "return to Proposed" variety) as far
as 2821bis is concerned.  I don't believe that inserting vague
hand-waving just to evade the document advancement rules would
benefit anyone (others may disagree, of course).  This subject
turns up on the issues list (never version attached, in PDF
form) as Issue 25 but has spillover effects on several others.

As I read the rules for advancement, any changes to the document
at this point must represent either (i) removing material,
especially material that has not be implemented, (ii)
clarifications to existing text that don't change the protocol
as generally understood when 2821 was published, (iii) changes
that reflect current existing practices _and_ that are
consistent with what is already in the text.   Those are _very_
strong constraints.

The other issues listed are, IMO, mostly trivial or cosmetic, at
least in the sense that I've heard no arguments that the
problems they are trying to fix are severe enough that they have
led to massive confusion and/or difficulty in getting
implementations that interoperate.   

Issues 2 and 10 are in sort of a gray area: they do get close to
interoperability issues.  I'm waiting for text from Ned but, if
we take the second path below, I probably understand the issues
well enough to make something up if we conclude that text won't
show up soon enough.

Anyway, I see two possible paths from here onward.

	(1) We decide that we really need to get this document
	as nearly perfect as possible, changing the ABNF to some
	elegant and normative form rather than using it as
	explanation of the syntax of commands, fixing even small
	typographic and formatting errors wherever they are
	found, adjusting terminology for complete consistency
	and minimizing of possible confusion with other
	documents, and continuing to find areas in which the
	text isn't clear enough to be proof against bizarre (or
	at least a little strange) interpretations.  In each
	case, we get to argue about the best way to address the
	issues that are identified and then apply the new text.
	That probably implies a year or three of work (at a
	somewhat more leisurely pace).  It is up to the ADs, but
	I think anything with that sort of object and timeframe
	probably requires a formal WG, with all that implies.
	Some people, notably Frank, have made strong arguments
	that such an effort is worthwhile and the timing is to
	be expected.
	
	(2) We decide that our goal is simply to get something
	out that cleans up the obvious and serious problems in
	2821.   The main goal of that effort is to provide a
	better referencing base for a number of documents that
	are in progress now.  That list presumably includes
	pieces of the EAI and Lemonade work.   Perhaps it also
	cleans up issues needed to put work on OPES-like
	activities, the family of things that include SPF and
	DKIM (and other heavy and lightweight approaches to
	header, sender domain, or sender validation and/or
	authentication or authorization), and perhaps some
	future extensions, onto a better foundation than 2821
	provides. In that case, we are ready, I believe, to
	resolve that key problem in some way, quickly clean up
	the rest of the issues list, and adopt a "showstoppers
	only" model to introduction of any new issues.

At some level, I don't have much of a preference.  But I think
we need to decide RSN about what we are trying to do here.

      john

--==========7259A80B544293DE6E57==========
Content-Type: application/pdf; name="RFC2821bis-status.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="RFC2821bis-status.pdf";
 size=12609

--==========7259A80B544293DE6E57==========--

Date: Sat, 28 Apr 2007 12:05:23 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: 2821bis-03:   Receive line From-domain Address-literal usage
Sender: [email-masked]

--On Saturday, 28 April, 2007 00:26 -0400 Hector Santos
<[email-masked]> wrote:

> Hi John,
> 
> I came across something today that may be of item of interest
> or may need some clarification.  I wasn't sure if this comment
> should or not be part of Issue 27 since it is labeled as close
> in your issues list).

More of this below, but, as a general observation, 2821 attempts
to be fairly careful to avoid specifying the expected behavior
by the server (or client) when the client (or server) violates
the protocol.   I hope we don't need to re-open that
meta-decision.

> Quite possibly, in summary, it may be deemed a new issue,
> possible two:
> 
>      - Clarifying Bracketed address-literals MUST requirement,
> and
>      - discouraging alterations of the address-literal for
>        the trace line.
> 
> The two may be related and possible be folded into 1 issue.

Or it may be no issue at all... I'll leave that up to Tony and
general discussion.

The rest of my comments below are just personal opinion except
where noted as "Editor:".

> So what prompted me to post a comment about a potential new
> issue?
> 
> First, the Received: ABNF includes the From-domain entity:
> 
>     From-domain    = "FROM" FWS Extended-Domain
>     Extended-Domain  = Domain /
>                    ( Domain FWS "(" TCP-info ")" ) /
>                    ( Address-literal FWS "(" TCP-info ")" )
> 
> For Address-literal, we have:
> 
>     address-literal  = "[" ( IPv4-address-literal /
> 
>                       IPv6-address-literal /
>                       General-address-literal ) "]"
>                       ; See Section 4.1.3
> 
> (Side Nit: Note the blank line)

Editor: Those blank lines (there are others) are an artifact of
the mechanism used in the ABNF to get those deep indentations
and a switch I made in the directives about spacing of list
items.  I'm gradually getting rid of them, but fixing this sort
of thing can also be left for the RFC Editor.

> Of course, putting aside the issues of mixed non-compliant
> practices in the field, per ABNF specification, the
> address-literal MUST be enclosed within brackets.
>...
 
> I sent a test message which I quickly received at gmail.com.
> I looked at the header lines and noticed something odd in the
> gmail.com server adding of its Received: trace line:
> 
>   Received: from ?192.168.1.101? ( [68.215.50.125])
>      by mx.google.com with ESMTP id
> h17sm2595511wxd.2007.04.27.19.07.38;
>      Fri, 27 Apr 2007 19:07:38 -0700 (PDT)

> Note the from address-literal with the question marks -
> ?192.169.1.101?
> 
> The MUA client actually sent:
> 
>      EHLO [192.169.1.101]

First of all, note that most MTAs who use a 
    ... from domain-or-literal ( literal ) ...
format are announcing that the actual IP address of the
sender-SMTP was different from the IP address specified (or
associated with the domain name given).  They are prohibited (in
relays) and discouraged (in submission servers) from rejecting
mail on that basis, but warnings have long been considered
appropriate.

You, and others, have questioned that rule, both wrt NAT-based
addresses and about whether it has become a proper basis for
rejection.

Second, note that there are strong prohibitions against ever
having an RFC 1918 private address on the public Internet.   It
would be a plausible (albeit well outside of 2821)
interpretation that
    HELO [192.168.1.101]
to a server on the public Internet is complete nonsense and
that, even if the mail is accepted for further transmission,
"[192.161.1.101]" is simply invalid, regardless of its syntax
properties.

If one accepts even a fraction of that, then a system that
behaves this way is operating outside the spec by tolerating
such an EHLO command at all and is trying to do something
creative to note that rather messy state.   

Now, my personal preference would have been to document this a
little differently, partially because I don't like adding syntax
violations to the complex of problems that are there already.
But it is a matter of taste and a territory into which I don't
think 2821bis needs to go or should go.  I think I would prefer
that a server deal with situations like this one by inserting
something more like

  Received: (from bogus-NAT-address 192.168.1.101
      [68.215.50.125])
      by mx.google.com with ESMTP id
      h17sm2595511wxd.2007.04.27.19.07.38;
      Fri, 27 Apr 2007 19:07:38 -0700 (PDT)

or maybe, taking liberties with a different rule,

  Received: from [68.215.50.125] 
      (EHLO bogus NAT address 192.168.1.101)
      by mx.google.com with ESMTP id
      h17sm2595511wxd.2007.04.27.19.07.38;
      Fri, 27 Apr 2007 19:07:38 -0700 (PDT)

> So one of the largest ISPs in the world, is altering the
> Received line by using replacing the brackets for question
> marks.

No, they aren't _altering_ the received line.    It is their
received line.  

Looking at this example only for the moment and ignoring the
other things you found out about Google's behavior, a
receiver-SMTP that does this could be doing something creative
with the syntax to note the fact that the smtp-Sender has
violated 2821 by issuing an invalid EHLO command (because it
uses a private address on the public Internet, not because of
syntax) and it has decided to accept and transport the mail
anyway.   I don't particularly like the syntax of their
announcement that they have done so, but the standards violation
occurs earlier and I have to approve of their attempt to
document the violation --and the situation as it appears from
their end-- carefully.

> Rhetorically, why would it do this?  Why not just use the
> bracketed address literal provided?

For some hypothetical system behaving that way, that could be a
reasonable explanation of an almost-reasonable behavior.  For
gmail the explanation above is obviously not the correct one,
but it is useful to separate the questions.  What they are
actually doing --given your tests -- makes them non-compliant.
But that isn't a reason to change 2821 either and hence not,
IMO, an issue for us.

> Again, Rhetorically, I ask, does the google server even
> support the reading of bracketed IP address-literal for
> EHLO/HELO or any other address-literal provided during the
> transaction?
> 
> If there are any google gmail.com lurking this list, they
> might want to chime in about the above.
> 
> To help explore these question, I went ahead and performed
> more test to see how it handled the EHLO/HELO.   I sent five
> (5) new messages with each using different EHLO with the
> following Received line results:
> 
> 1) SMTP compliant bracketed address-address, IP matching
> 
>    EHLO [68.215.50.125]
> 
> 	--> Received: from ?68.215.50.125?

Bug, IMO.

> 2) No bracketed address-address, IP matching
> 
>    EHLO 68.215.50.125
> 
> 	--> Received: from 68.215.50.125

Worse bug, IMO.  But, of course, this is also a client violation.

> 3) Simulate what Outlook would do (sent computer name)
> 
>    EHLO hdev1
> 
> 	--> Received: from hdev1

Different bug, but also a client violation.

> 4) Use the real/correct ptr domain
> 
>    EHLO adsl-215-50-125.mia.bellsouth.net
> 
> 	-->  Received: from adsl-215-50-125.mia.bellsouth.net

behaving as expected.

> 5) Use the incorrect ptr domain (125 changed to 126)
> 
>    EHLO adsl-215-50-126.mia.bellsouth.net
> 
> 	-->  Received: from adsl-215-50-125.mia.bellsouth.net

With no IP address in brackets?   Clearly permitted by 2821 (and
821), but probably not in particularly good taste today.   Still
not a 2821bis issue, IMO.

> Based on all this I can take a SWAG that
> 
> a) Google is not supporting bracketing address-literals,

Looks like it.

> b) Google replaces unexpected characters with '?' for
>     the Received line address-literal field.

Yep.

> c) Even though the transaction requires the strong SUBMIT 587
>     protocol, it it relaxed on not enforcing correct
>     address-literal syntax IP matching nor rDNS check
> authentication.

Yes, but not an issue for 2821bis.  Possibly an issue for
4409bis, but I doubt it.

> The latter (c) is a separate issue, but it might reflect the
> importance or unimportance of the bracketed address-literals.

Or might reflect the more general possibility that someone at
gmail is asleep at the switch.

> Why or how could any of this constitute for 2821bis?

> Well, in our quest for clarifying and codifying existing and
> possible dominant behavior,  for new implementors what I can
> see occurring is they begin may use systems like GOOGLE
> gmail.com as a design baseline for practical and good
> 2821/2822 protocol behavior. They may follow their lead,
> including not expecting brackets and also converting them to
> questions marks for the trace line.

> Comments anyone?

This is the same problem that we had when some people were
assuming that whatever sendmail was doing was the spec, without
looking at the spec, 20 years ago.  It was a problem then, it is
a problem now... only the previous bad actor has cleaned up its
act and the new bad actor has.  It is not a problem with the
standard, IMO, unless someone can go find the relevant gmail
people and get them to tell us that their trace field
implementation ended up this way because they got confused by
the spec.  Otherwise, this is interesting, and sad, but not a
problem.

       john

Date: Sat, 28 Apr 2007 00:26:09 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: John C Klensin <[email-masked]>
Subject: 2821bis-03:   Receive line From-domain Address-literal usage
Sender: [email-masked]

Hi John,

I came across something today that may be of item of interest or may 
need some clarification.  I wasn't sure if this comment should or not be 
part of Issue 27 since it is labeled as close in your issues list).

Quite possibly, in summary, it may be deemed a new issue, possible two:

     - Clarifying Bracketed address-literals MUST requirement, and
     - discouraging alterations of the address-literal for
       the trace line.

The two may be related and possible be folded into 1 issue.

So what prompted me to post a comment about a potential new issue?

First, the Received: ABNF includes the From-domain entity:

    From-domain    = "FROM" FWS Extended-Domain
    Extended-Domain  = Domain /
                   ( Domain FWS "(" TCP-info ")" ) /
                   ( Address-literal FWS "(" TCP-info ")" )

For Address-literal, we have:

    address-literal  = "[" ( IPv4-address-literal /

                      IPv6-address-literal /
                      General-address-literal ) "]"
                      ; See Section 4.1.3

(Side Nit: Note the blank line)

Of course, putting aside the issues of mixed non-compliant practices in 
the field, per ABNF specification, the address-literal MUST be enclosed 
within brackets.

Well, we have a throw away Google gmail.com account that we use for 
testing various things.  Working from my home office machine using the 
MUA Thunderbird 2.0, I was exploring sending mail using the gmail.com 
account.  gmail.com allows MUA based offline posting and it requires the 
SUBMIT protocol (port 587) with TLS to be prepared in the MUA for this 
account.

I sent a test message which I quickly received at gmail.com.  I looked 
at the header lines and noticed something odd in the gmail.com server 
adding of its Received: trace line:

  Received: from ?192.168.1.101? ( [68.215.50.125])
     by mx.google.com with ESMTP id h17sm2595511wxd.2007.04.27.19.07.38;
     Fri, 27 Apr 2007 19:07:38 -0700 (PDT)

Note the from address-literal with the question marks - ?192.169.1.101?

The MUA client actually sent:

     EHLO [192.169.1.101]

So one of the largest ISPs in the world, is altering the Received line 
by using replacing the brackets for question marks.

Rhetorically, why would it do this?  Why not just use the bracketed 
address literal provided?

Again, Rhetorically, I ask, does the google server even support the 
reading of bracketed IP address-literal for EHLO/HELO or any other 
address-literal provided during the transaction?

If there are any google gmail.com lurking this list, they might want to 
chime in about the above.

To help explore these question, I went ahead and performed more test to 
see how it handled the EHLO/HELO.   I sent five (5) new messages with 
each using different EHLO with the following Received line results:

1) SMTP compliant bracketed address-address, IP matching

   EHLO [68.215.50.125]

	--> Received: from ?68.215.50.125?

2) No bracketed address-address, IP matching

   EHLO 68.215.50.125

	--> Received: from 68.215.50.125

3) Simulate what Outlook would do (sent computer name)

   EHLO hdev1

	--> Received: from hdev1

4) Use the real/correct ptr domain

   EHLO adsl-215-50-125.mia.bellsouth.net

	-->  Received: from adsl-215-50-125.mia.bellsouth.net

5) Use the incorrect ptr domain (125 changed to 126)

   EHLO adsl-215-50-126.mia.bellsouth.net

	-->  Received: from adsl-215-50-125.mia.bellsouth.net

Based on all this I can take a SWAG that

a) Google is not supporting bracketing address-literals,

b) Google replaces unexpected characters with '?' for
    the Received line address-literal field.

c) Even though the transaction requires the strong SUBMIT 587
    protocol, it it relaxed on not enforcing correct
    address-literal syntax IP matching nor rDNS check authentication.

The latter (c) is a separate issue, but it might reflect the importance 
or unimportance of the bracketed address-literals.

Why or how could any of this constitute for 2821bis?

Well, in our quest for clarifying and codifying existing and possible 
dominant behavior,  for new implementors what I can see occurring is 
they begin may use systems like GOOGLE gmail.com as a design baseline 
for practical and good 2821/2822 protocol behavior. They may follow 
their lead, including not expecting brackets and also converting them to 
questions marks for the trace line.

Comments anyone?

--
HLS

Date: Fri, 27 Apr 2007 18:14:23 -0400
From: John C Klensin <[email-masked]>
To: Chris Haynes <[email-masked]>, [email-masked]
Subject: Re: Re[2]: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

--On Friday, 27 April, 2007 22:00 +0100 Chris Haynes
<[email-masked]> wrote:

> On Friday, April 27, 2007 at 2:56:27 PM, John C Klensin wrote:
> 
> 
>> Apologies to those who reacted to this issue based on their
>> guesses about what the text might say without actually looking
>> at it.
> 
>>       john
> 
> If you intend to include me in the above (as in your off-list
> accusation), I _did_ read the text of bis-03 and I felt then
> and still feel the similarities with / differences from FTP
> are not '...worth noting...'.

I did not intend to include you.  I did run across several
people, in both notes posted to the list and in private notes,
who had.

> Just say whatever needs to be said about 1xy.

I am certain your opinion has been noted.  As I've said before,
I don't feel strongly about this, so will do whatever Tony tells
me to do.

    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue  33:  SMTP-Sender, etc.
Date:  Fri, 27 Apr 2007 23:53:00 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 65
Sender: [email-masked]

John C Klensin wrote:

>> Would it be possible to use SMTP-Sender instead of just sender?

> Possible?  yes.   I'll have to leave to others to figure out if
> this is a priority.

Maybe a case by case approach, not a global modification effort.

Anything related to terms like "sender", "originator", "return
path", "reverse path", etc. is obviously critical, their meaning
in RFC 821 depends on the original design, and the context when
it was designed.

Nobody would try "mail from B" for mail actually sent from A with
the known exceptions like "different local part is okay", or "as
long as A and B are behind the same MX it's fine".  It would be a
bad plan if something on the route from A to the receiver doesn't
work, to introduce an unrelated route to B for the error reports.

An additional route is a potential source for more trouble, and
if error reports don't make it they're lost, empty reverse path.

All considerations in this direction were likely "obvious" when
RFC 821 was written, it could afford to be sloppy with some terms.

What should a "mail from" be, certainly no arbitrary "bounces to"
a 3rd party.  From the ten or more aliases of this beast I think
"envelope sender" as used in RFC 3461 is the best.  It avoids any
confusion of (2)821 and (2)822, it doesn't mention the less clear
"from", it doesn't talk about reverse paths and return routes, and
it says "sender", the entity found guilty to have sent a mail.  On
whose behalf it sent the mail is beside the point.  The "envelope
sender" is related to the header field (Resent-) Sender, i.e. the
responsible agent for the message.  For the "envelope sender" it's
the responsible agent wrt SMTP.

IOW, be careful with those "senders", please.

> Everyone should remember that this was supposed to be a quick
> effort.

If anybody promised that this would be a "quick effort" then their
definition of "quick effort" is "not as bad as 2026bis, more like
2616bis or 3490bis, nothing to worry about, no grandson-of-1036" :-)

> I'd rather just declare the effort to switch to "client-server"
> terminology a failure.  That would mean going back to the 821
> terminology of "sender-SMTP" and "receiver-SMTP"

Possible, but not desirable, "client" and "server" is clear in all
contexts with two MTAs talking.  It's somewhat more confusing in
RFC 4408, where "the client" is actually an SMTP-server in its role
as DNS-client of a name server related to the alleged sender.  If
you use the client-server terminology only for SMTP, not for DNS,
it should be clear.

> not introducing more and entirely new terminology like
> "transmitter" or "transmitting-client"

Yes, RFC 2822 uses "transmitter", but it's unnecessary in 2821bis,
and "transmitting-client" would be a mouthful, also unnecessary.

Frank

Date: Fri, 27 Apr 2007 22:00:42 +0100
From: Chris Haynes <[email-masked]>
Reply-To: Chris Haynes <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re[2]: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

On Friday, April 27, 2007 at 2:56:27 PM, John C Klensin wrote:

> Apologies to those who reacted to this issue based on their
> guesses about what the text might say without actually looking
> at it.

>       john

If you intend to include me in the above (as in your off-list accusation), I _did_ read the text of bis-03 and I felt then and still feel the similarities with / differences from FTP are not '...worth noting...'.

Just say whatever needs to be said about 1xy.

Chris Haynes

Date: Fri, 27 Apr 2007 16:16:09 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Issue list
Sender: [email-masked]

--On Friday, 27 April, 2007 20:30 +0200 Frank Ellermann
<[email-masked]> wrote:

> Hector Santos wrote:
> 
>> http://www.isdg.net/public/ietf/drafts/rfc281bis-status.htm
> 
> Thanks, my browser was still not happy, I simplified the Excel
> output, see below.
> 
> I miss two articles about other issues posted here:
> 
> Kari  (2007-04-20): "for" clause on Received: header field 
> http://permalink.gmane.org/gmane.ietf.smtp/5708

As far as Pete and I are concerned, this has been resolved by
further stripping the Received line syntax from 2822.  If you
want useful syntax for Received headers, you look at 2821[bis],
which is authoritative.  Anything beyond the most basic lexical
syntax that still appears in 2822bis is an error.
  
> Frank (2007-04-23): General Address Literal
> http://permalink.gmane.org/gmane.ietf.smtp/5810

This note is a confusion of issues.  If you want an issue number
assigned to one or more of the issues in it, post separate notes
with meaningful subject lines.  

Beyond that, a decision on this is up to Tony, but I believe
that a request to _change_ the fundamental syntax used to
express different versions of IP and the associated literals is
out of scope.  I believe that we can make syntax changes at this
point to clarify or correct what is there, but not to invent new
(to 2821/2822) approaches.

Of course, if interoperability reports (which are not part of
_my_ issue list) indicate that the syntax for IPv6 literals is
not supported by at least two independent implementations, that
syntax can be removed.  But that would give us no standard
support for IPv6 literals at all, not an opportunity to invent a
new syntax at this time.  Just IMO, of course.

     john

Date: Fri, 27 Apr 2007 16:13:36 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Issue list
Sender: [email-masked]

--On Friday, 27 April, 2007 20:30 +0200 Frank Ellermann
<[email-masked]> wrote:

> Hector Santos wrote:
> 
>> http://www.isdg.net/public/ietf/drafts/rfc281bis-status.htm
> 
> Thanks, my browser was still not happy, I simplified the Excel
> output, see below.
> 
> I miss two articles about other issues posted here:
> 
> Kari  (2007-04-20): "for" clause on Received: header field 
> http://permalink.gmane.org/gmane.ietf.smtp/5708

As far as Pete and I are concerned, this has been resolved by
further stripping the Received line syntax from 2822.  If you
want useful syntax for Received headers, you look at 2821[bis],
which is authoritative.  Anything beyond the most basic lexical
syntax that still appears in 2822bis is an error.
  
> Frank (2007-04-23): General Address Literal
> http://permalink.gmane.org/gmane.ietf.smtp/5810

This note is a confusion of issues.  If you want an issue number
assigned to one or more of the issues in it, post separate notes
with meaningful subject lines.  

Beyond that, a decision on this is up to Tony, but I believe
that a request to _change_ the fundamental syntax used to
express different versions of IP and the associated literals is
out of scope.  I believe that we can make syntax changes at this
point to clarify or correct what is there, but not to invent new
(to 2821/2822) approaches.

Of course, if interoperability reports (which are not part of
_my_ issue list) indicate that the syntax for IPv6 literals is
not supported by at least two independent implementations, that
syntax can be removed.  But that would give us no standard
support for IPv6 literals at all, not an opportunity to invent a
new syntax at this time.  Just IMO, of course.

     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Issue list
Date:  Fri, 27 Apr 2007 20:30:54 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 470
Sender: [email-masked]

--------------2BC32941364D

Hector Santos wrote:

> http://www.isdg.net/public/ietf/drafts/rfc281bis-status.htm

Thanks, my browser was still not happy, I simplified the Excel
output, see below.

I miss two articles about other issues posted here:

Kari  (2007-04-20): "for" clause on Received: header field 
http://permalink.gmane.org/gmane.ietf.smtp/5708

Frank (2007-04-23): General Address Literal
http://permalink.gmane.org/gmane.ietf.smtp/5810

Frank

--------------2BC32941364D
Content-Type: text/html; charset=us-ascii; name="2821bisi.htm"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="2821bisi.htm"

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://validator.w3.org/sgml-lib/REC-xhtml1-20020801/xhtml1-transitional.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"><head>
    <meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />
    <title> RFC 2821 issues list </title>
</head><body>

<h1>RFC 2821bis issues list</h1>
<p>
  Note: ??? Implies &quot;issue
  may still be open even though text has changed&quot; or, with the stuff
  assigned to Ned &quot;waiting&quot; cref is also marked ???
</p>

<table border="1" cellpadding="2" cellspacing="2">
<tr> <th>
  -03 Issue#
</th><th>
  -02 Issue#
</th><th>
  -01
</th><th>
  Description
</th><th>
  Status
</th><th>
  Comments

</th></tr><tr><td>
  closed
</td><td>
  closed
</td><td>
  0a
</td><td>
  Remove &quot;ignorant&quot; from 7.1
</td><td>
  Done - 02b
</td><td>
  Promoting harmony

</td></tr><tr><td>
  closed
</td><td>
  closed
</td><td>
  0b
</td><td>
  MAIL FROM validation and EHLO
  validity tests</td>
  <td>No action</td>
  <td>&quot;Editorial judgment&quot;</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>1</td>
  <td>1</td>
  <td>Format of domain - trailing
  period</td>
  <td>02c. Sufficient? ???</td>
  <td>So far inconclusive, but leaning
  toward &quot;no dot, but some explanation about TLDs' - 20070328.
  Bunch of ccTLDs have MX records.
  Note sent to ICANN 20070329 to try to find
  out what for.Draft text in -02c
  20070413.</td>

 </tr>
 <tr>
  <td>2</td>
  <td>2</td>
  <td>2</td>
  <td>Check VRFY/EXPN text in Section
  3.5.3</td>
  <td>Ned???</td>
  <td>See Ned's note, 20070329.
  Ned to send text, I hope.
  Placeholder added -02c, 20070413.</td>

 </tr>
 <tr>
  <td>closed?</td>
  <td>3</td>
  <td>3</td>
  <td>EHLO validation in sect 4.1.4,
  para 6</td>
  <td></td>
  <td>Tony Finch suggests MUST-&gt;
  SHOULD, Hector has new language (some of which, IMO, goes too far).
  Notes on 20070329).
  Pulled text from 4.1.4 to eliminate
  redundancy; added more xrefs.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>4</td>
  <td>Sect
  4.2.1 (Reply Codes 4yz 5yz) &quot;encouraged&quot; -&gt; SHOULD </td>
  <td>Done 03b (see issue 16)</td>
  <td>Hector suggests (20070309) a
  rewrite to MAY based on no enforcement. David Skoll suggests MUST NOT. Closed
  4yz to SHOULD in -02, 5yz to SHOULD NOT in -03 (part of issue 16)</td>

 </tr>
 <tr>
  <td>
  5</td>
  <td>5</td>
  <td>5</td>
  <td>Sect 4.4. Syntax of ID in trace</td>
  <td>Fixed -02c</td>
  <td>Leave alone or change
  &quot;string&quot; to &quot;atom&quot;.
  Changed to Atom per consensus note.
  Might need to be Dot-string, per Ellermann 20070403</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>6</td>
  <td>Precedence error -- ABNF
  technical problem w/ grouping</td>
  <td>Fixed -02b</td>
  <td>Editorial: need parens to group
  alternatives</td>

 </tr>
 <tr>
  <td>
  7</td>
  <td>7</td>
  <td>7</td>
  <td>Switch conformity language to
  2119 form</td>
  <td></td>
  <td>Added to list 20070328</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>8</td>
  <td>8</td>
  <td>Number of recipients/ number of
  transactions per connection</td>
  <td>Dead?</td>
  <td>2821is clear on recipients needs
  separate extension to do either definitively.<i>
  Guidance on transactions/ connection??</i>
  Probably dead - no text needed 20070416</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>9</td>
  <td>Table of requirements and/or
  index and/or more items as sections.</td>
  <td>Fixes in -02b.
  Done</td>
  <td>4.5.3.1 and 4.5.3.2 lists
  converted to sections in -02b. TOC directives patched to include these.
  More would fall into the
  &quot;rework&quot; category.</td>

 </tr>
 <tr>
  <td>
  10</td>
  <td>10</td>
  <td>10</td>
  <td>Add discussion of mail rerouting
  via 251/551 to Security Considerations</td>
  <td>Ned???</td>
  <td>Ned's note, 20070329.
  Ned to send text, I hope.
  Placeholder inserted in 02c.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>11</td>
  <td>Change instances of &quot;A
  RR&quot; to language that is IPv6-inclusive)</td>
  <td>Fixed - 02b</td>
  <td>Discussed on list, number not
  formally assigned. Seems like a
  no-brainer.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>12</td>
  <td>IPv6 issues with MX records</td>
  <td>New text inserted, 02c</td>
  <td>Discuss implications of MX
  records with IPv6-only targets.
  Handed off to ADs. Draft text
  tentatively approved.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>13</td>
  <td>Syntax for parameters that
  contain email addresses.</td>
  <td>Done 02c</td>
  <td>Use xtext from RFC 3461 section
  4: <a href="http://www.apps.ietf.org/rfc/rfc3461.html#page-6">http://www.apps.ietf.org/rfc/rfc3461.html#page-6</a></td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>14</td>
  <td>Continuation lines for 220
  Greeting?</td>
  <td>Inserted 02c</td>
  <td>Tony Finch 20070405.
  Says 20070406 that multiline greetings are
  common and cause no interop problems.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>15</td>
  <td>Syntax change for reply lines to
  allow for continuation?</td>
  <td>Fixed 02c</td>
  <td>Tony Finch 20070405.
  Seems like clear bug. Note cref comment about Reply-text
  production.</td>

 </tr>
 <tr>
  <td>closed?</td>
  <td>16</td>
  <td>16</td>
  <td>General use of &quot;is
  permitted&quot;, etc</td>
  <td>Done 03b</td>
  <td>Klensin note, 20070407.
  General feeling is &quot;should
  edit&quot;. Get 02 posted first,
  then do this.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>17</td>
  <td
  >Should text
  be clarified to single code in continuations </td>
  <td>Done 02c</td>
  <td>Hector et al 20070409-11.
  After list discussions, text changed to
  require one code 20070413.</td>

 </tr>
 <tr>
  <td>closed</td>
  <td>closed</td>
  <td>18</td>
  <td>Should text be clarified to
  prohibit 1yz codes without extensions</td>
  <td>Done 02c</td>
  <td>Hector et al 20070409-11.
  After list discussions, text changed to
  clearly prohibit 20070413.Put in:
  Any unrecognized code SHOULD be treated as an error (4yz or 5yz under discussion
  20070423)</td>

 </tr>
 <tr>
  <td>
  19</td>
  <td>19</td>
  <td>19</td>
  <td>Comments on literals in EHLO -
  remove either &quot;Explained-literal&quot; or the text in 4.1.1.1 that
  describes it.</td>
  <td></td>
  <td>Tony Finch/ Mark Mallett
  20070412</td>

 </tr>
 <tr>
  <td>
  20</td>
  <td>20</td>
  <td>20</td>
  <td>RFC3552 and Security
  Considerations</td>
  <td></td>
  <td>Do we need to do anything?</td>

 </tr>
 <tr>
  <td>
  21</td>
  <td>21</td>
  <td>---</td>
  <td>Bounce / reject improvements in
  section 6.2</td>
  <td>Collapse into issue 25 - section
  6.2 isn't all that is affected</td>
  <td>Wasn't that debate ruled out of
  order?</td>

 </tr>
 <tr>
  <td>
  22</td>
  <td>22</td>
  <td>---</td>
  <td>May-&gt;Should, Should-&gt; Must
  changes for return-paths in 4.4</td>
  <td></td>
  <td>Conversation with Keith 20070426
  - note to list. Change text to remove
  apparent contradiction.</td>

 </tr>
 <tr>
  <td>
  23</td>
  <td>23</td>
  <td>---</td>
  <td>Section 2.3.11 now defines
  &quot;reply&quot;. Should there be a
  specific, numbered, definition for &quot;command&quot;?
  If so, text welcome.</td>
  <td></td>
  <td>Mark E Mallett 20070418</td>

 </tr>
 <tr>
  <td>
  24</td>
  <td>24</td>
  <td>---</td>
  <td>Revise all ABNF to make a closed
  system starting with a definition for &quot;command&quot; or
  &quot;command-verb&quot;</td>
  <td></td>
  <td>Pro: It would be nicer.
  Con: odds of being able to do this in a
  reasonable amount of time and without introducing new errors are low to
  none. Frank and Arnt to rework after
  all substantive issues are concluded.</td>

 </tr>
 <tr>
  <td>
  25</td>
  <td>25</td>
  <td>---</td>
  <td>Change all text that implies
  non-delivery messages to prohibit them entirely or permit them only with
  source authentication.</td>
  <td></td>
  <td>Frank Ellerman, repeatedly.
  This one needs to be definitively closed,
  again, IMO (JcK). Make sure any
  changes are reflected in 2821bis-03 6.2 and see Hector's &quot;2821bis-03
  comments&quot; note of 20070426</td>

 </tr>
 <tr>
  <td>closed?</td>
  <td>26</td>
  <td>---</td>
  <td>Revise description of
  reverse-paths to prevent repeated citations of &quot;server adds its
  own&quot;</td>
  <td>???Text in 03c</td>
  <td>I think this is a dead issue
  (JcK)</td>

 </tr>
 <tr>
  <td>closed?</td>
  <td>27</td>
  <td>---</td>
  <td>Straighten out and define
  terminology for talking about headers</td>
  <td>Done 03c</td>
  <td>Pete Resnick note 20070423.
  Header section/ Header field/ Received
  clause</td>

 </tr>
 <tr>
  <td>closed?</td>
  <td>28</td>
  <td>---</td>
  <td>Client behavior with
  unrecognized code</td>
  <td>Text in 03c</td>
  <td>Proposed text posted to list and
  inserted in 03c, 20070423</td>

 </tr>
 <tr>
  <td>
  30</td>
  <td>30</td>
  <td>---</td>
  <td>Should source route text be
  removed entirely and systems prohibited from paying attention to them?</td>
  <td></td>
  <td>Alternative to issue 26</td>

 </tr>
 <tr>
  <td>31</td>
  <td>---</td>
  <td>---</td>
  <td>Retain or drop new (03) FTP
  comparison text?</td>
  <td colspan="2"></td>

 </tr>
 <tr>
  <td>32</td>
  <td>----</td>
  <td>---</td>
  <td>Is requirement in section 2.1
  that server accept responsibility for messages appropriately stated as MUST?</td>
  <td></td>
  <td>formerly &quot;protocol requires
  that a server accept&quot;</td>

 </tr>
</table></body></html>

--------------2BC32941364D--

Date: Fri, 27 Apr 2007 13:59:41 -0400
From: John Leslie <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: 2821bis-03
Sender: [email-masked]

John C Klensin <[email-masked]> wrote:
> --On Friday, 27 April, 2007 03:57 +0200 Frank Ellermann
> <[email-masked]> wrote:
>> 
>>| However, in practice, some servers do not perform recipient
>>| verification until after the message text is received.  These
>>| servers SHOULD treat a failure for one or more recipients as
>>| a "subsequent failure" and return a mail message as discussed
>>| in Section 6 and, in  particular, in Section 6.1.
>> 
>> No, that's precisely what they SHOULD NOT do, because it's net
>> abuse in the case of unverified reverse paths.  It will get
>> them blacklisted, and after that they won't be able to report
>> serious problems.  There should not be any "you SHOULD spam"
>> in 2821bis.
>
> Issue 25.

] Change all text that implies non-delivery messages to prohibit them
] entirely or permit them only with source authentication.

   Related, yes; same, no.

   I've got to agree with Frank: There should not be any "you SHOULD spam"
in 2821bis. There's quite a distance between MUST NOT send a NDN and
SHOULD send a NDN (to a doubtful MailFrom).

   Further, I agree with Frank that sending indiscriminate NDNs _will_
get you blacklisted.

   Let me try stepping back a few paces...

   However much we try to assign lawyer-like meanings to MUSTard, a
MUST is something that means failure to do it indicates you're not
talking SMTP; a SHOULD means there might be exceptions, but in general
failure means you're not talking SMTP.

   OTOH, a lower-case "should" is merely a recommendation -- not an
indication of protocol violation.

   Frank is arguing that forged MailFroms are more common than valid
ones (an easily-observed fact); thus their use (lower-case) should
be deprecated as justification for sending unsolicited email.

   Instead, this text amounts to an accusation of failing to speak
SMTP if you _don't_ send unsolicited email, which IMHO is a rather
silly thing to say.

   NDNs, IMHO, are a historic artifact of a time when it was OK to
trust anyone using the Internet. Given trustworthy users, and given
the ordinary case of needing multiple hops, they were a reasonable
kludge.

   Today the vast majority of NDNs merely indicate that some spammer
forged your email address into a MailFrom -- often without any
particular evil intent except to bypass Verizon's dumb CBV. We need
to admit that spammers aren't going away anytime soon, nor is
Verizon's practice of callback verification.

   In running my ISP, I have to blackhole tens of thousands of NDNs
daily. I'm sure others simply blackhole _all_ NDNs. We should _stop_
encouraging folks to send them.

   I'd like to seriously suggest a middle ground where servers MAY
send NDNs, but (lower-case) should try to verify that the address
they send to represents a person or role account likely to take
corrective action.

   I don't believe we can retreat from SHOULD to SHOULD NOT; but a
retreat to MAY seems appropriate. Even the perfectly valid NDNs
generally don't reach a person who could take corrective action;
and the vast majority of users ignore them even if they get them.

   In an ideal world, we'd make NDNs opt-in for folks who can
understand what to do with them, and make the usual case to be
merely recording the incident in a log file.

--
John Leslie <[email-masked]>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  MUST NOT reject [was: 2821bis-03]
Date:  Fri, 27 Apr 2007 19:08:39 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 77
Sender: [email-masked]

Philip Guenther wrote:

>>>> If some servers try to count Resent-* blocks they are mad.
>>>>  But that's IMO a harmless madness, not a MUST NOT.

>>> Changes an explicit DRUMS decision... needs a stronger argument
>>> than your sense of taste, IMO.

>> IMO it needs a justification why that's relevant if you keep it,
>> for a specific MUST NOT after a general SHOULD NOT it has to be
>> important.  You can't enumerate and forbid all cases of dubious
>> reject decisions.  Something might be special with the Resent-*
>> case, and I've just no idea what this could be.
 
> My recall from DRUMS of this and some of the other "MUST NOT reject
> for <some reason>" was that they involved situations that were
> always intended to be valid but that had been often misunderstood
> by implementors, such that DRUMS decided to make a strong statement
> that those cases were indeed valid and that systems that reject
> messages because the implementor thought they weren't are broken
> system.

Okay, let's say there were some ad hoc "MUST NOT" in 2821 addressing
weird implementations at the time when it was written.  That doesn't
force us to keep them when they did the trick.  Another example in
this direction is the "Apparently-To" section:  This is now a "IANA
registered bad idea" with a reference to 2821 (+3552).  IMO we can
remove it from 2821bis, it's an obsolete bug.  Nobody is going to
reinvent Apparently-To if 2821bis drops its Apparently-To chapter.

> This was an area of 822/821 that was more murky than most, so it
> was called out.

It's even murkier today with those multiple Resent-blocks, and some
odd RFC 4406 applications happily ignoring a MUST in RFC 2822 while
assuming that the "may add sender" in RFC 4409 is actually a "must
add sender" plus "get it right in the presence of Resent-blocks" :-(

> Perhaps 2821 has done its job of clarification and implementors
> are no longer falling into that trap, but if so, it may be the
> very strength of the MUST NOT that has been getting the point 
> across.

I see your point, but there are various "MUST NOT reject this" and
"MUST NOT reject that" resulting in an overall impression that any
"reject" decision is dubious.  But that's not true anymore for a
world with 80% spam.  With "reject a.s.a.p." legit senders have a
chance "to make another plan" or to fix whatever the problem is.

With the obsolete "always try to accept" strategy legit mails are
lost when they end up in some "suspicious" folder.  So IMO any old
"MUST NOT reject" sends the wrong message to 2821(bis) readers, it
is propaganda for the broken "accept and bounce" design.

> Is a site that rejects all messages containing "unknown" header
> fields within its rights to do so?  Of course.  They may be
> idiots, but they're only hurting themselves.

Yes, and the sender will learn that they're idiots, and can "make
another plan", e.g. reject list subscriptions from their users.

> Is an implementation that always behaves that way conforming?
> No, it's broken, and we really don't want such an implementation
> to get into widespread use.

The general SHOULD NOT covers this, violating a SHOULD NOT needs a
good excuse, being an idiot isn't good enough:

| Server SMTP systems SHOULD NOT reject messages based on perceived
| defects in the RFC 822 or MIME (RFC2045 [21]) message header
| section or message body.

Frank

P.S. for John:  My editor counts 35 "822" and 15 "2822", i.e. about
                20 bare "822" where s/822/2822/ might be okay (?)

Date: Fri, 27 Apr 2007 13:44:32 -0400
From: John C Klensin <[email-masked]>
To: Douglas Otis <[email-masked]>, Frank Ellermann <[email-masked]>
cc: [email-masked]
Subject: rfc2821bis-03 Issue  33:  SMTP-Sender, etc.
Sender: [email-masked]

Issue 33 assigned.  See below.

--On Friday, 27 April, 2007 09:52 -0700 Douglas Otis
<[email-masked]> wrote:

> 
> 
> On Apr 26, 2007, at 11:41 PM, Frank Ellermann wrote:
>> John C Klensin wrote:
>> 
>>>> Nit: Consider changing "Sender" to "Transmitter",
>>>> "Transmission   Channel", or "Dispatcher" to avoid
>>>> confusion with "Sender:".    Perhaps adopt server/client
>>>> rather than abuse the word sender.    Although this will
>>>> touch many sentences, such a change will provide greater
>>>> clarity going forward.
>> 
>>> IMO, too large a change, expecially since SMTP-Sender and
>>> SMTP-  Receiver (the terms used in 821) is the terminology
>>> of last resort   when terms based on "client" and "server"
>>> are insufficiently precise.
>> 
>> +1
> 
> Would it be possible to use SMTP-Sender instead of just sender?

Possible?  yes.   I'll have to leave to others to figure out if
this is a priority.  Everyone should remember that this was
supposed to be a quick effort; we are already moving out of the
"quick" range, IMO.

>>> I can try this if enough people think it is important
>> 
>> It is very important, but I hope it's enough if you replace
>> some   "senders" by "clients" where the latter is what you
>> really want.    Global changes of terminology are tricky, I'm
>> glad that it   apparently worked for the "bounces".
> 
> It might also be clearer to use "transmitting-client" rather
> than just "client" when needed.  The section title could
> change to "SMTP-Senders and Receivers"".

Speaking as editor but with the usual understanding that I'll do
whatever Tony tells me is the consensus:  If we are going to try
to fine-tune this stuff because of perceptions of interactions
with terms used in other documents, I'd rather just declare the
effort to switch to "client-server" terminology a failure.  That
would mean going back to the 821 terminology of "sender-SMTP"
and "receiver-SMTP" (or some equivalent, although my instinct
would be to stick with 821) throughout, either just dropping
"SMTP client" and "SMTP server" entirely or noting the
equivalence in the terminology section but not using them
afterwards.   That is more work but, if the consensus is that we
have ambiguities here, then I think that is the right cure, not
introducing more and entirely new terminology like "transmitter"
or "transmitting-client"

As far as the ability to make global changes is concerned,
please note that the stem "bounc" is relatively rare in a
document like this while "transmit", "client", "receive",
"server", etc., are not.

Coming back to a comment above, my time availability after this
week and the general nature of things is such that, if we try to
make this sort of change, revise the ABNF for completeness and a
different approach (see earlier note about descriptive versus
normative uses of syntax productions), and make the added
changes needed to use RFC 2119, my assumption is that we are
talking about a completion date in the last quarter of this year
at the earliest, not, e.g., the beginning of next month.    That
is consistent with Alexey's suggestion that he do a review pass
on the SMTP after Lemonade winds down and the deadlines for
other work (in and out of IETF) for which I have responsibility.
That is just my personal opinion, of course.

    john

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: rfc2821bis-3 SMTP-Sender
Date: Fri, 27 Apr 2007 09:52:54 -0700
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Apr 26, 2007, at 11:41 PM, Frank Ellermann wrote:
> John C Klensin wrote:
>
>>> Nit: Consider changing "Sender" to "Transmitter", "Transmission  
>>> Channel", or "Dispatcher" to avoid confusion with "Sender:".   
>>> Perhaps adopt server/client rather than abuse the word sender.   
>>> Although this will touch many sentences, such a change will  
>>> provide greater clarity going forward.
>
>> IMO, too large a change, expecially since SMTP-Sender and SMTP- 
>> Receiver (the terms used in 821) is the terminology of last resort  
>> when terms based on "client" and "server" are insufficiently precise.
>
> +1

Would it be possible to use SMTP-Sender instead of just sender?

>> I can try this if enough people think it is important
>
> It is very important, but I hope it's enough if you replace some  
> "senders" by "clients" where the latter is what you really want.   
> Global changes of terminology are tricky, I'm glad that it  
> apparently worked for the "bounces".

It might also be clearer to use "transmitting-client" rather than  
just "client" when needed.  The section title could change to "SMTP- 
Senders and Receivers"".

-Doug

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue  22: Return-path required removal
Date:  Fri, 27 Apr 2007 17:29:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Pete Resnick wrote:

> Even in 2822, it was:
> fields          =       *(trace
>                            *(resent-date /
[...]
>                             resent-msg-id))
>                          *(orig-date /
[...]
 
> That's one <return> per *set* of <trace>.

ACK, I missed that point.  

> 2822upd includes <optional-field> up with
> the <trace> as well.

With T = trace, R = resent block, O = <optional-field>,
and A = anything else what you have now is: 

  fields = *( T *O *R ) *( A O ) 

That's almost the same as a syntactically simpler:

  fields = *( [T] O R ) *( A O )

One difference is that in your nested construct a T
(trace) is required if there's any R (resent-block).

But T can be empty, and with that a "required" T has
no effect for e.g. parsers trying to consider R in a
RFC 4409 8.1 "may add sender" scenario.

The complete R or worse *R concept is ugly like hell.

Frank

Date: Fri, 27 Apr 2007 09:56:27 -0400
From: John C Klensin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

--On Friday, 27 April, 2007 09:08 +0100 Alexey Melnikov
<[email-masked]> wrote:

> John C Klensin wrote:
> 
>> rfc2821bis-03 adds new text to explicitly compare SMTP's reply
>> code model to that of FTP in the hope of eliminating any
>> possible residual confusion about issues 18 and 28.  Should
>> that text be retained, dropped, or modified in some way?
>>  
>> 
> I was originally planning to propose to remove any comparison
> with FTP, possibly creating another draft (Chris Newman wrote
> one in 1998) that would compare reply codes from different
> protocols. However after checking -03, I only found the
> following paragraph:
> 
>    It is worth noting that the file transfer protocol (FTP
> [35]) uses a
>    very similar code architecture and that the SMTP codes are
> based on
>    the FTP model.  However, SMTP uses a one-command,
> one-response model,
>    while FTP is asynchronous and the 1yz codes are not part of
> the SMTP
>    model.
> 
> I think this is a reasonable text and it is small. I think it
> should be kept as is.

Thanks.  The main argument for this from my perspective is that
someone comparing the 2821 and 2821bis text is going to wonder
where the 1yz text went if we didn't say something.  This seemed
to me to be about the minimum we could stay that would not be
tied to the particulars of the recent discussions.

Apologies to those who reacted to this issue based on their
guesses about what the text might say without actually looking
at it.

      john

> 
> 

Date: Fri, 27 Apr 2007 09:08:52 +0100
From: Alexey Melnikov <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

John C Klensin wrote:

>rfc2821bis-03 adds new text to explicitly compare SMTP's reply
>code model to that of FTP in the hope of eliminating any
>possible residual confusion about issues 18 and 28.  Should that
>text be retained, dropped, or modified in some way?
>  
>
I was originally planning to propose to remove any comparison with FTP, 
possibly creating another draft (Chris Newman wrote one in 1998) that 
would compare reply codes from different protocols. However after 
checking -03, I only found the following paragraph:

   It is worth noting that the file transfer protocol (FTP [35]) uses a
   very similar code architecture and that the SMTP codes are based on
   the FTP model.  However, SMTP uses a one-command, one-response model,
   while FTP is asynchronous and the 1yz codes are not part of the SMTP
   model.

I think this is a reasonable text and it is small. I think it should be 
kept as is.

Date: Fri, 27 Apr 2007 05:35:50 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03 Issue  22: Return-path required removal
Sender: [email-masked]

John C Klensin wrote:
> Reviewing this issue briefly, there two connected questions 
> 
> 
> I have now reviewed these issues with Keith Moore, the author of
> 3834.   We have concluded that the current restrictions are
> about right: the only question is whether delivery servers,
> which need to add a Return-path of their own should be required
> (presumably with a SHOULD, possibly a MUST) to verify that there
> are no Return-path header lines present, and removing them if
> needed, before adding one.  Requiring it would not match current
> practice in several cases, but, if people are convinced that it
> was necessary to ensure good interoperability, it could be done.
> We are not convinced that it is necessary.

Hi John,

If the concern is to use SMTP to help assist post processing mailbots, 
then maybe we can approach some text added to 4.1.1.4  DATA (DATA) section.

MUA clients will not be sending a Return-path: (at least not with an 
original message) in its DATA block.

So this is an issue mostly with routers whose feed come from receivers 
who had prepended a Return-Path for the current hop.

One suggestion could be this:

     During transmission, the client (MTA router) SHOULD remove
     the current hop Return-Path: header line during the
     DATA transfer.

or

     During transmission, the client (MTA router) SHOULD
     NOT transmit on the current hop return-path header
     line during the DATA transfer.

That is why our router system does and I think it is BCP.  I say that 
because we evolved from UUCP/SLIP with the same concerns where "From " 
was dealt with the same and similar way between hops.  I recall the 
design issues and also the researching and discussions about how other 
systems did this.

In the end, each system can have their own internal method of storing 
intermediate mail for routing and/or gating and it may or may not 
include using Return-Path specifically to record a persistent entity for 
the purpose of a return address.  A good example is how we store our 
intermediate mail in format like so:

     ENV-FROM: address
     ENV-TO: [internal userid or 0] address
     ENV-TO: [internal userid or 0] address
     ENV-TO: [internal userid or 0] address
     ENV-TO: [internal userid or 0] address
     ...
     ENV-DATA:
     Return-Path:
     Received:

And from this, our router will create XQT/CMD/DAT uucp style file format 
files, one for each ENV-TO.   The DAT now just has:

     Return-Path:
     Received:

and the persistent return-path is used by the router for the next SMTP 
transaction MAIL FROM: command.   At the DATA command, it is removed 
(skipped) the first line and send the rest, making sure each header line 
conforms the CRLF EOL.

But was it necessary to use Return-Path?

No.  Not really.  We could of used ENV-FROM, like I've seen others do 
X-MAIL-FROM to store the return path during the intermediate routes.

> It is clear to us that RFC 3834, were it ever upgraded for
> Draft, should get some additional text specifying that, if
> multiple Return-paths are present, only the "top line" one
> counts or, perhaps better, that only the "top line" one counts
> whether or not there are others present.

+1. I agree 100%.  I think the system can only functionally work well 
when it honors the z-order "LIFO" (last in/first out) nature and 
methodology for network related header lines.

> Given the above, we don't see much value in a reference from
> 2821bis to 3834, although we don't see it as harmful either.

My only comment is I think if its was added, developers might get the 
erroneous idea that it is a required bounce format (DSN style) to follow 
  in order to send a bounce message.

If that is the goal, then IMO that is going to force a change for many 
systems, including ours.  It might be a welcome change, but a change 
nonetheless.

Yes, I don't see any harm with referencing 3824 as a suggested bounce 
format to convey the important information in a typical bounce message. 
But not as a required format.

Date: Fri, 27 Apr 2007 02:24:40 -0600
From: Philip Guenther <[email-masked]>
To: Frank Ellermann <[email-masked]>
cc: [email-masked]
Subject: Re: 2821bis-03
Sender: [email-masked]

On Fri, 27 Apr 2007, Frank Ellermann wrote:
> John C Klensin wrote:
>> [Frank wrote, citing the draft]
...
>>> | In particular, they MUST NOT reject messages in which the
>>> | numbers of Resent-fields do not match or Resent-to appears
>>> | without Resent-from and/or Resent- date.
>
>>> Delete this, please.  Receivers can reject whatever they don't
>>> like.  The complete Resent-* concept is unnecessary in a world
>>> using MIME.  If some servers try to count Resent-* blocks they
>>> are mad.  But that's IMO a harmless madness, not a MUST NOT.
>
>> Changes an explicit DRUMS decision... needs a stronger argument
>> than your sense of taste, IMO.
>
> IMO it needs a justification why that's relevant if you keep it,
> for a specific MUST NOT after a general SHOULD NOT it has to be
> important.  You can't enumerate and forbid all cases of dubious
> reject decisions.  Something might be special with the Resent-*
> case, and I've just no idea what this could be.

My recall from DRUMS of this and some of the other "MUST NOT reject
for <some reason>" was that they involved situations that were
always intended to be valid but that had been often misunderstood
by implementors, such that DRUMS decided to make a strong statement
that those cases were indeed valid and that systems that reject
messages because the implementor thought they weren't are broken
system.

So, what was special with the Resent-* case was that there *were*
known examples of implementors screwing up and rejecting because
they thought that a message with a Reject-To but no Reject-Date,
or something similar, was invalid.  This was an area of 822/821
that was more murky than most, so it was called out.  Perhaps 2821
has done its job of clarification and implementors are no longer
falling into that trap, but if so, it may be the very strength of
the MUST NOT that has been getting the point across.

<leaving direct topic....>

Just because a system may reject messages for whatever policy reasons
it chooses doesn't mean we don't want to exert strong pressure to
keep the avenues of SMTP operation and extension open.  Is a site
that rejects all messages containing "unknown" header fields within
its rights to do so?  Of course.  They may be idiots, but they're
only hurting themselves.  Is an implementation that always behaves
that way conforming?  No, it's broken, and we really don't want
such an implementation to get into widespread use.

(This exact sort of thing is a major problem at other layers; think
DNS record types vs nameserver support, or TCP reserved bits vs
firewalls.)

Philip Guenther

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-3 nits and typos
Date:  Fri, 27 Apr 2007 08:41:57 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

John C Klensin wrote:

>> ----
>> 2.1. Basic Structure:

>>   ...delivering a message or properly reporting the failure to
>> do so.

>> Nit: Change "reporting failure to do so." to "reporting
>> failure to the originator address.  When a possibility has
>> been determined that a failure subsequent to acceptance will
>> not be reported, refusing the message better retains delivery
>> status integrity and is generally preferred."

> That is not a nit, but part of what is now issue 25.

Yes, and it expresses a part of the solution for "THE" issue.

>> Nit: Consider changing "Sender" to "Transmitter",
>> "Transmission Channel", or "Dispatcher" to avoid confusion
>> with "Sender:".  Perhaps adopt server/client rather than abuse
>> the word sender.  Although this will touch many sentences,
>> such a change will provide greater clarity going forward.

> IMO, too large a change, expecially since SMTP-Sender and
> SMTP-Receiver (the terms used in 821) is the terminology of
> last resort when terms based on "client" and "server" are
> insufficiently precise.

+1

> I can try this if enough people think it is important

It is very important, but I hope it's enough if you replace
some "senders" by "clients" where the latter is what you
really want.  Global changes of terminology are tricky, I'm
glad that it apparently worked for the "bounces".

> based on the experience with changing informal text to
> SHOULD/ MAY/ MUST terminology --which I now expect to have
> to back out--

I don't know why you expect that, I saw only one dubius new
"MUST NOT" in the first three chapters.  Admittedly I also
can't say that I insist on much of the new MUSTard, but the
part about "MUST take the responsibility" sounded good.

> if people really want to get this done anytime soon... like
> this calendar year.

That's IMO realistic.  No new draft soon, please, I'd hate to
drop all WGs I watch only because 2821bis is more important
than all of them together.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: 2821bis-03
Date:  Fri, 27 Apr 2007 08:07:37 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 204
Sender: [email-masked]

John C Klensin wrote:

> ????????????????

More verbose, the sentence was in 2821:

   Such a transaction consists of a series of commands to specify the
   originator and destination of the mail and transmission of the
   message content (including any headers or other structure) itself.

In -03 you cleaned up various "headers" resulting in:

   Such a transaction consists of a series of commands to specify the
   originator and destination of the mail and transmission of the
   message content (including any lines in the header section or other
   structure) itself.

Is that either "including (any lines in the header section) OR other
structure", or is it "including any lines IN (the header section or
other structure)" ?  Or is the difference anyway irrelevant, assuming
that it's a difference at all ?  Maybe saying "message content itself,
including any lines" etc. (no parentheses) would be better readable.

 [BCP 14 keywords]
> Been discussed before.  On list as Issue 7.

Okay.  Not that I've seen many supporters to keep the "private"
terms in 2821bis since 2005, but maybe they'll show up later. :-)

>> Those language tags in RFC 2231 are okay.
> Mumble.
Just in case, I meant the tags in RFC 2231 chapter 5, not chapter 4.

>> - 2.3.11:  "command" + "verb", let's forget it, 2.4 is the next
>>   section and explains what's what right at the begin.

> I am not sure I understand this comment unless it is a
> suggestion that we give a negative answer to Issue 23 and move
> on.

Yes, if "issue 23" is related to the proposal to add a 2.3.x section
explaining "command" and "verb" as terminology.  The explanation in
2.4 is good enough, and section 2.4 follows immediately after 2.3.11.

>>| However, in practice, some servers do not perform recipient
>>| verification until after the message text is received.  These
>>| servers SHOULD treat a failure for one or more recipients as
>>| a "subsequent failure" and return a mail message as discussed
>>| in Section 6 and, in  particular, in Section 6.1.

>> No, that's precisely what they SHOULD NOT do, because it's net
>> abuse in the case of unverified reverse paths.  It will get
>> them blacklisted, and after that they won't be able to report
>> serious problems.  There should not be any "you SHOULD spam"
>> in 2821bis.

> Issue 25.

Certainly a part of THE issue, maybe form a set of issue numbers
belonging to THE issue, and we can defer it while discussing less
critical stuff.

At some point we'll need the "implementation and interoperability
report", it would be a waste of time to fix details in the long
EXPN and VRFY sections if they're later deprecated with pointers
to (2)821 for curious readers.

 [EAI]
> the 8BITMIME option is about valid objects that the relevant
> servers decide to not downgrade.   Those are not defects and are
> not being rejected on that basis.

When I wrote "option for 8BITMIME, duty for EAI" etc. the option
I had in mind was this:

| If such messages are transmitted in violation of this rule,
| receiving SMTP servers MAY clear the high-order bit or reject
| the message as invalid.

That's a "MAY [...] reject the message as invalid" in 2821bis-03,
not in EAI.  This MAY and the SHOULD NOT is no contradiction, but
I think receivers should be free to reject whatever they don't
like, not only 8bit garbage outside of 8BITMIME.  Stripping the
MSB is a peculiar recipe, I'd prefer if I never get the results.

>>| In particular, they MUST NOT reject messages in which the
>>| numbers of Resent-fields do not match or Resent-to appears
>>| without Resent-from and/or Resent- date.

>> Delete this, please.  Receivers can reject whatever they don't
>> like.  The complete Resent-* concept is unnecessary in a world
>> using MIME.  If some servers try to count Resent-* blocks they
>> are mad.  But that's IMO a harmless madness, not a MUST NOT.

> Changes an explicit DRUMS decision... needs a stronger argument
> than your sense of taste, IMO.

IMO it needs a justification why that's relevant if you keep it,
for a specific MUST NOT after a general SHOULD NOT it has to be
important.  You can't enumerate and forbid all cases of dubious
reject decisions.  Something might be special with the Resent-*
case, and I've just no idea what this could be.

  [551 "MUST NOT assume"]
>> I also don't see the point in "MUST NOT", you could just say
>> "cannot" also for 251, it's a fact, not an order.

> Mechanical translation.  And yet another reason to back out all
> of the changes to insert SHOULD/MUST language.

I'm not sure about the general effects, but here no MUSTard is
probably clearer.

>> The 251/551 section in RFC 821 is IMO clearer,  This needs more
>> discussion, it misses the point of taking the responsibility
>> for the mail.
[...]

> In practice, unless something has changed recently, 551 has been
> very little used for years... and 251 even less so.

The SPF FAIL logic in the case of "2821-forwarding to a 3rd party
implementing 4408" in esssence emulates a 551 by the forwarder,
that's the good case, "so that the sender can make another plan",
as you put it in RFC.ietf-eai-framework-05.  It would start to get
messy if the next hop accepts and discards the FAIL, maybe because
it checks accepted mails behind its border (too late).

251 is not very interesting, from the sender's POV it worked, and
then a sender won't care why it worked, and if that's a permanent
or temporary forwarding.  In theory it is still relevant, shorter
routes are always better wrt potential errors, they're also faster.

In practice we can ignore 251, but the wording in RFC 821 is good:

| The receiver takes responsibility for delivering the message.

And if it doesn't wish to take this responsibility (incl. bounces
in the RFC 821 reverse path design) it can pick 551:

| The receiver refuses to accept mail for this user, and the
| sender must either redirect the mail according to the information
| provided or return an error response to the originating user.

Roughly the same as "make another plan".  And nobody could be
interested to accept mail from dubious sources.  They'd get it
back in the case of any errors, it would sit in their queue
trying to reach the next hop or address of the <reverse-path>.

Forwarding was a serious service, and 551 a serious alternative.

> I don't understand why you think this has anything to do with
> either taking responsibilty for mail or unverified return paths...
> unless this is another manifestation of Issue 25.

Yes, it's clearly a manifestation of THE issue.  I can't tell at
the moment if issue 25 covers 551 and forwarding to 3rd parties.

>> - 3.5:  There are many at least potentially valid addresses in
>>   this chapter, please add "TLD" .example everywhere, and maybe
>>   replace foo.com by example.com

> These particular examples have been used since 821.  I am loathe
> to start making changes unless there is very general concern
> that it is worth the effort and risks.

It's a bad example for readers.  The poor owner of foo.com might
still wonder why his spam has such odd pattern with local parts
jsmith@ + hsmith@ + dweep@  In some places folks try desperately
to educate users that they shouldn't make up arbitrary domains,
addresses, phone numbers, IPs, etc. for examples.

>> - 3.6:  I guess you'll update that when appendix C is
>> eliminated,   therefore I skip it for now.

> If appendix C is eliminated, yes.

Some liked the idea, and IIRC nobody disliked it...

> The experience with the changes to SHOULD/MUST have turned me
> back into a minimalist position, i.e., I now believe that, if
> this document is to be finished any time soon, we have better
> stop talking about "nice" fixes and start concentrating on only
> those changes that are necessary.

...getting rid of cruft like source routes is necessary, they're
hopelessly broken since RFC 1123.  It's impossible to revive them
as they were, it won't work as expected and as designed.

>> 5.2.16 is a "sender rewriting system".
> I think this is a substantive change.

Yes, a substantive change in 2821 compared with 1123 5.2.16.  The
gateway needs to make sure that error reports reach the foreign
environment, but like a mailing list it's also interested in any
errors caused by its redistribution.  The normal user (sender) on
the other side has no clue what a gateway is, or how SMTP works,
if the gateway somehow screwed up.  If the gateway picks its own
fabrication of a From address also as MAIL FROM address it could
with a small plausible error "arrange" that error reports never
make it anywhere, and that would be a Bad Thing.

Frank

Date: Fri, 27 Apr 2007 01:02:27 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-03 Issue  22: Return-path required removal
Sender: [email-masked]

--On Friday, 27 April, 2007 05:33 +0200 Frank Ellermann
<[email-masked]> wrote:

>...
 
> 2822upd-01 still says: 
> 
>   trace           =       [return]
>                           1*received
> 
> That's at most one <return>, not more.

See Pete's note.  But, to try to explain this differently than I
did before, 821 and 2821 view the syntax rules as explanatory
adjuncts to definitive and normative text.  You are trying very
hard to make them specific and normative.  I think that is a
fine idea in principle but, coming back to Dave's original
comments on the problem, I think trying to make that change is
incompatible with the goal of getting 2821bis done quickly and
accurately.

At least as important, since 2821 is written on the theory that
MTAs should not be examining any header lines other than those
that they put in, the only header-related syntax needed by a
given instance of an MTA is the syntax needed to _supply_ at
most one Return-path header line and one or more Received header
lines.  All of the rest of the header section is, as far as that
model is concerned, an undifferentiated blob and, absent
extensions (BDAT in particular) is even undifferentiated from
the message body.

>> Requiring it would not match current practice in several
>> cases, but, if people are convinced that it was necessary to
>> ensure good interoperability, it could be done.
> 
> RFC 2822 and 3834 don't mention that there could be more than
> one, and that's what authors of "vacation" and "out of office"
> scripts hopefully read after their users got enough spamcop
> reports.
> 
>> We are not convinced that it is necessary.
> 
> You trust that "vacation" implementors get it right, and take
> the top <return> ?  

"Vacation" implementations are the problem of 3834.  If one were
to take a very narrow view of 3834, it is improperly, or at
least insufficiently, written because it is well-known that a
message may be received (e.g., by the agent for the MUA that
generates a vacation message) with more than one such header
line.  Both 821 and 2821 specify the conditions under which a
Return-path is to be inserted, but neither requires finding or
removing any that might exist.

> And do you think that 2822upd can stay as
> is with a single <return> in this case ?

Yes.  See above and Pete's note.

>> Given the above, we don't see much value in a reference from
>> 2821bis to 3834, although we don't see it as harmful either.
> 
> I think it would help.  The case of no Return-Path is harmful
> for authors of auto-responders, compare
> RFC.ietf-sieve-vacation-07.txt

The case of no Return-path is clearly problematic.  But there is
a MUST on inserting the Return-path.  Certainly 2821bis does not
reference specifications that would exhibit damage if other
"MUST"s were ignored.

     john

Date: Thu, 26 Apr 2007 23:59:37 -0500
To: IETF-POP3EXT:;, IETF-IMAPEXT:;, IETF-SMTP:;, LEMONADE:;, IMA:;
From: Pete Resnick <[email-masked]>
Subject: Fwd: draft-resnick-2822upd-01 posted
Sender: [email-masked]

Please follow the discussion on [email-masked].

--- begin forwarded text

Date: Thu, 26 Apr 2007 16:55:06 -0500
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: draft-resnick-2822upd-01 posted

You may have noticed that 2822upd-01 was posted.

<http://www.ietf.org/internet-drafts/draft-resnick-2822upd-01.txt>
Also:
<http://www.qualcomm.com/~presnick/draft-resnick-2822upd-01.html>
<http://www.qualcomm.com/~presnick/draft-resnick-2822upd-01.xml>
<http://www.qualcomm.com/~presnick/draft-resnick-2822upd-01.txt>

To make rfcdiff work, I've put a copy of RFC 2822 on my web site with 
some of the later sections rearranged so that they're in the same 
order as the current draft. You can see the diffs between the two 
here:

<http://tools.ietf.org/rfcdiff?url1=http://www.qualcomm.com/~presnick/rfc2822-fixed.txt&url2=http://www.ietf.org/internet-drafts/draft-resnick-2822upd-01.txt>

Though overall very few, there are real changes from 2822 in this 
document that need review. See the changes section at the bottom. I 
did *not* go whole hog and switch to Bruce's syntax, simply because I 
wanted to limit the number of changes and see if we can't get this 
thing to Draft along with 2821bis. But I did incorporate (hopefully 
all of) the corrections that you all have pointed out over the last 
couple of years.

Please review with a fine toothed comb. If there are more than just a 
few problems, perhaps I'll ask Tony Hansen or Philip Guenther (who 
have at assorted time offered assistance) to start an issues list.

Thanks in advance for your assistance.

pr

--- end forwarded text

-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 27 Apr 2007 00:38:56 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: Issue list
Sender: [email-masked]

--On Friday, 27 April, 2007 00:27 -0400 Hector Santos
<[email-masked]> wrote:

> Hi John,
> 
> Since I don't have Excel on my laptop, I converted it to HTML
> output for anyone who is just interested in seeing a quick
> snapshot.
> 
> http://www.isdg.net/public/ietf/drafts/rfc281bis-status.htm
> 
> I hope you don't mind, I will remove it when it is obsolete or
> behind hand if you wish.

Actually, I had made up and intended to send you a PDF file.
Sorry for the excel -- it was a mistake for which I apologize.

     john

Date: Fri, 27 Apr 2007 00:27:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: Issue list
Sender: [email-masked]

John C Klensin wrote:
> Hi.
> 
> Since several people have asked, the issue list is not
> accessible online, but is a spreadsheet that I've been using to
> keep from losing things.  I pass copies to Tony at regular
> intervals and, when necessary, we coordinate.
> 
> A snapshot of the important information is attached.  The first
> three columns essentially show whether an issue is still active
> (a number in column 1) and what its status was at the time the
> -02 and -01 drafts were posted.   "Closed?" in column one
> indicates that the item is on Tony's recent list as closed
> unless serious rejections are raised (my summary, not his
> words).  The rest should be more or less obvious.
> 
>       john

Hi John,

Since I don't have Excel on my laptop, I converted it to HTML output for 
anyone who is just interested in seeing a quick snapshot.

http://www.isdg.net/public/ietf/drafts/rfc281bis-status.htm

I hope you don't mind, I will remove it when it is obsolete or behind 
hand if you wish.

--
HLS

Date: Thu, 26 Apr 2007 23:27:05 -0500
To: Frank Ellermann <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2821bis-03 Issue  22: Return-path required removal
Cc: [email-masked]
Sender: [email-masked]

On 4/27/07 at 5:33 AM +0200, Frank Ellermann wrote:

>John C Klensin wrote:
>
>>the only question is whether delivery servers, which need to add a 
>>Return-path of their own should be required (presumably with a 
>>SHOULD, possibly a MUST) to verify that there are no Return-path 
>>header lines present, and removing them if needed
>
>2822upd-01 still says:
>
>   trace           =       [return]
>                           1*received
>
>That's at most one <return>, not more.

Uh, no. Even in 2822, it was:

fields          =       *(trace
                           *(resent-date /
                            resent-from /
                            resent-sender /
                            resent-to /
                            resent-cc /
                            resent-bcc /
                            resent-msg-id))
                         *(orig-date /
                         from /
                         sender /
                         reply-to /
                         to /
                         cc /
                         bcc /
                         message-id /
                         in-reply-to /
                         references /
                         subject /
                         comments /
                         keywords /
                         optional-field)

That's one <return> per *set* of <trace>. 2822upd includes 
<optional-field> up with the <trace> as well.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Thu, 26 Apr 2007 23:36:43 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Issue list
Sender: [email-masked]

--==========3AD88A23E8CB80DF46BA==========
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi.

Since several people have asked, the issue list is not
accessible online, but is a spreadsheet that I've been using to
keep from losing things.  I pass copies to Tony at regular
intervals and, when necessary, we coordinate.

A snapshot of the important information is attached.  The first
three columns essentially show whether an issue is still active
(a number in column 1) and what its status was at the time the
-02 and -01 drafts were posted.   "Closed?" in column one
indicates that the item is on Tony's recent list as closed
unless serious rejections are raised (my summary, not his
words).  The rest should be more or less obvious.

      john

--==========3AD88A23E8CB80DF46BA==========
Content-Type: application/vnd.ms-excel; name="RFC2821bis-status.xls"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="RFC2821bis-status.xls";
 size=24064

0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAABAAAALQAAAAAAAAAA
EAAA/v///wAAAAD+////AAAAACwAAAD/////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////

--==========3AD88A23E8CB80DF46BA==========--

Date: Thu, 26 Apr 2007 23:35:40 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Frank Ellermann wrote:
> John C Klensin wrote:
>  
>>         In either case, a formal handoff of responsibility for
>>         the message occurs: the protocol requires that a server
>>         MUST accept responsibility for either delivering a
>>         message or properly reporting the failure to do so.
>  
>> I think this is a simple rewording of a clear statement.
> 
> It's fine.  Of course my idea of "properly reporting" is to the
> "originator", and where that's dubious it's no proper reporting:
> 
> It's the duty of the SMTP server to decide if it can take this
> responsbility, and it MUST reject the mail when it's in doubt
> about the originator.  Otherwise it can't report properly, and
> would violate this clear MUST.  That's why there is a 551, and
> not only a 251.

Frank,

Don't you think there is a distinction between "Technical 
Responsibility" versus "Policy Responsibility?"

Its like, you MUST built this to behave as such for scenario A and 
scenario B.   We can view that as a responsibility, but I see that more 
like a technical requirement for the protocol itself.

After that, the issue of responsibility falls on the operator, the site 
itself, based on its local policy definitions.

A good example of this is issue of Black, White, Grey listing (BWG) and 
call-back verifiers (CBV).

CBV and Greylisting are 100% based on the theories of SMTP Technical 
Compliance and very little to do with subjective policy decisions.

CBV works on the premise that SMTP mandates that the Return-Path MUST be 
valid.  GreyListing works on the premise that SMTP mandates "retries" or 
the expectation of "retries" for proper operating systems. In both 
cases,  CBV and GreyListing makes no distinction for the real intentions 
of the sender - rejections can happen even for "good people" when they 
are not properly running a valid SMTP system,

Black and White listing is 100% local policy driven. It really doesn't 
matter if the sender is 100% technically SMTP compliant.

Truth be told, there is a legal bend to all this.

We make this very clear in our own add-on product disclaimers to 
separate the distinction of SMTP compliance vs Local Policy decisions.

http://www.winserver.com/wcsap

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue  22: Return-path required removal
Date:  Fri, 27 Apr 2007 05:33:00 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 47
Sender: [email-masked]

John C Klensin wrote:

> I have now reviewed these issues with Keith Moore, the author
> of 3834.

Good, I like this RFC so much that I wrote a script yesterday
to get a good explanation into the clipbrd for manual spamcop
reports:

| erroneous bounce to the From-address of a list member
| instead of the envelope sender address, for details see
| RFC 3834 http://tools.ietf.org/html/rfc3834#section-4

> the only question is whether delivery servers, which need to
> add a Return-path of their own should be required (presumably
> with a SHOULD, possibly a MUST) to verify that there are no
> Return-path header lines present, and removing them if needed

2822upd-01 still says: 

  trace           =       [return]
                          1*received

That's at most one <return>, not more.

> Requiring it would not match current practice in several cases,
> but, if people are convinced that it was necessary to ensure
> good interoperability, it could be done.

RFC 2822 and 3834 don't mention that there could be more than one,
and that's what authors of "vacation" and "out of office" scripts
hopefully read after their users got enough spamcop reports.

> We are not convinced that it is necessary.

You trust that "vacation" implementors get it right, and take the
top <return> ?  And do you think that 2822upd can stay as is with
a single <return> in this case ?

> Given the above, we don't see much value in a reference from
> 2821bis to 3834, although we don't see it as harmful either.

I think it would help.  The case of no Return-Path is harmful for
authors of auto-responders, compare RFC.ietf-sieve-vacation-07.txt

Frank

Date: Thu, 26 Apr 2007 23:14:45 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: 2821bis-03
Sender: [email-masked]

--On Friday, 27 April, 2007 03:57 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> http://tools.ietf.org/rfcdiff?url2=http%3A%2F%2Ftools.ietf.org
> %2Fid%2Fdraft-klensin-rfc2821bis-03.txt&difftype=--hwdiff
> 
> Top down only the diffs from -02 or what I see:
> 
> - What was wrong with "obsoletes 2821" (now "obsoletes 821") ?
>   The complete list of RFCs obsoleted by 2821 is longer.
> - appendix C (source routes) is still there.

Bug in my XML.  I was trying to get it to list both.  Fixed.

> | including any lines in the header section or other structure
> * including any other structure like lines in the header
> section
> 
> Or maybe get rid of the "or other structure", it's hard to
> parse the sentence after you've replaced the "headers".

????????????????
 
> - 2.3 Terminology:  Please use the BCP 14 terminology like most
>   other RFCs.

Been discussed before.  On list as Issue 7.

> | A MIME extension (RFC2047 [30]) defines an
> * Two MIME extensions (RFC2047 [30] and RFC2231 [??]) define an

Done.  I hope others agree and this doesn't need an issue
number, etc.
 
> Those language tags in RFC 2231 are okay.

Mumble.

> | EHLO command MUST BE
> * EHLO command MUST be

Done.

> | Recent work, RFC3463 [39], has specified
> * RFC3463 [39] specifies

Done
 
> RFC 1893 is no recent work anymore, RFC 3463 is a draft
> standard.
> 
> | However, it MUST not be construed as authorization
> * However, it MUST NOT be construed as authorization

Done

> I've a script to highlight "MUSTard".  Maybe a future version
> of the idnits offers an option -s "sofixit".
> 
> - 2.3.11:  "command" + "verb", let's forget it, 2.4 is the next
>   section and explains what's what right at the begin.

I am not sure I understand this comment unless it is a
suggestion that we give a negative answer to Issue 23 and move
on.

> | However, in practice, some servers do not perform recipient
> | verification until after the message text is received.  These
> | servers SHOULD treat a failure for one or more recipients as
> | a "subsequent failure" and return a mail message as discussed
> | in Section 6 and, in  particular, in Section 6.1.
> 
> No, that's precisely what they SHOULD NOT do, because it's net
> abuse in the case of unverified reverse paths.  It will get
> them blacklisted, and after that they won't be able to report
> serious problems.  There should not be any "you SHOULD spam"
> in 2821bis.

Issue 25.

> | Server SMTP systems SHOULD NOT reject messages based on
> | perceived defects in the RFC 822 or MIME (RFC2045 [21])
> message | header section or message body.
> 
> Rejecting broken mail objects is an option for 8BITMIME, a duty
> for EAI, and a duty for 4409.  This SHOULD NOT is utter
> dubious.

In reverse order, Submission (4409) servers are not SMTP
servers, there are no final EAI specs and I don't think the
current ones contain a requirement to reject _broken_ mail
objects, and the 8BITMIME option is about valid objects that the
relevant servers decide to not downgrade.   Those are not
defects and are not being rejected on that basis.

> | In particular, they MUST NOT reject messages in which the
> | numbers of Resent-fields do not match or Resent-to appears
> | without Resent-from and/or Resent- date.
> 
> Delete this, please.  Receivers can reject whatever they don't
> like.  The complete Resent-* concept is unnecessary in a world
> using MIME.  If some servers try to count Resent-* blocks they
> are mad.  But that's IMO a harmless madness, not a MUST NOT.

Changes an explicit DRUMS decision... needs a stronger argument
than your sense of taste, IMO.

> | if a 551 code is used, they MUST NOT assume that the client
> | will actually update address information or even return that
> | information to the user.
> 
> Please delete or "or even" etc., clients can and should return
> the complete error reply to the user.  I also don't see the
> point in "MUST NOT", you could just say "cannot" also for 251,
> it's a fact, not an order.

Mechanical translation.  And yet another reason to back out all
of the changes to insert SHOULD/MUST language.

> The 251/551 section in RFC 821 is IMO clearer,  This needs more
> discussion, it misses the point of taking the responsibility
> for the mail.  That's by definition not possible for an
> unverified reverse path, and 551 is a very sound alternative
> to 250/251 for servers not wishing to get into any trouble
> with bounces.

In practice, unless something has changed recently, 551 has been
very little used for years... and 251 even less so.  And I don't
understand why you think this has anything to do with either
taking responsibilty for mail or unverified return paths...
unless this is another manifestation of Issue 25.

> 550 is also okay, but unlike 551 not very helpful for both
> sides.

> - 3.5:  There are many at least potentially valid addresses in
>   this chapter, please add "TLD" .example everywhere, and maybe
>   replace foo.com by example.com

These particular examples have been used since 821.  I am loathe
to start making changes unless there is very general concern
that it is worth the effort and risks.

> - 3.6:  I guess you'll update that when appendix C is
> eliminated,   therefore I skip it for now.

If appendix C is eliminated, yes.  The experience with the
changes to SHOULD/MUST have turned me back into a minimalist
position, i.e., I now believe that, if this document is to be
finished any time soon, we have better stop talking about "nice"
fixes and start concentrating on only those changes that are
necessary.

> | Gateways are, of course, subject to the same rules for
> handling | source routes as those described for other SMTP
> systems in | Section 3.3
> 
> 3.3 also discusses source routes, but the main part is in 3.6
> (?)
> 
> | The translation algorithm used to convert mail from the
> Internet | protocols to another environment's protocol SHOULD
> ensure that | error messages from the foreign mail environment
> are delivered | to the return path from the SMTP envelope, not
> to the sender | listed in the "From:" field (or other fields)
> of the message body.
> 
> Normally you write "reverse path", is it intentionally a
> "return path" here, because you're talking about gateways ?

Yes, it was, but it is too subtle, so I've changed it to
"reverse path"

> For the part about the "sender" I propose:
> 
> * not to an address in the "From:", "Sender:", or similar
> header * fields of the message.
> 
> That should cover Sender:, Resent-From:, Resent-Sender:,
> Reply-To:. IMO "message body" is misleading if you actually
> mean the complete message (header and body), or here
> especially its header.  Maybe "mail object" or "message data"
> is better if "message" isn't clear.

Changed.  I think "message" is clear since the previous
sentence, as revised, talks about "header lines".

> | If the foreign environment has no equivalent concept, the
> gateway | must select and use a best approximation, with the
> message | originator's address as the default of last resort.
> 
> I think it should use the address of postmaster@ (or similar)
> at the gateway, and not a random address in something it made
> up while translating foreign addresses to Internet addresses
> as specified in 3.7.4.  The gateway is responsible, and errors
> should be reported via the gateway.  It's the same RFC 1123
> problem as for forwarders, and the solution in 1123 5.2.16
> keeps the responsibility where it belongs, at the gateway.
> 5.2.16 is a "sender rewriting system".

I think this is a substantive change.

> | Note that the key difference between handling aliases
> (Section | 3.9.1) and forwarding (this subsection) is the
> change to the | backward-pointing address in this case.
> 
> That's a point where you could mention that this used to be no
> key difference under RFC 821, because "in any case, the SMTP"
> added "its own identifier to the reverse path".  There might be
> better places to explain that RFC 1123 broke the original SMTP
> design in its quest to get rid of the source routes.

Send text, but my inclination is to not change this further,
especially to reflect the long-dead "copy own address into
reverse-path stuff".

> Okay, 3 chapters out of 17 (with appendices), enough for today.
> One disadvantage of minor changes in a diff is that I start to
> look at text I haven't read for years only because there's a
> diff mark, and then I start to compare 821/1123 with 2821bis...

Yet another reason to back out the SHOULD/MUST changes and to
start getting minimalist about this work.

thanks,
     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Date:  Fri, 27 Apr 2007 04:58:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

David F. Skoll wrote:

> I don't see anything that explicitly says that a "proper failure report"
> is either a 5yz code during the SMTP session or a non-delivery
> notification sent to the Return-Path.  I mean, *every* SMTP implementor
> *knows* this, but I think it should be spelled out.

What about this part in 3.6:

   If an SMTP server has accepted the task of relaying the mail and
   later finds that the destination is incorrect or that the mail cannot
   be delivered for some other reason, then it MUST construct an
   "undeliverable mail" notification message and send it to the
   originator of the undeliverable mail (as indicated by the reverse-
   path).

This paragraph should end there, the rest about DSNs belongs into a
separate paragraph.  There needs to be a discussion somewhere in the
spec. that today the reverse path doesn't necessarily indicate the 
originator / redistributor / gateway as last responsible party.  The
reverse path was mutilated by RFC 1123.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Date:  Fri, 27 Apr 2007 04:42:32 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

John C Klensin wrote:
 
>         In either case, a formal handoff of responsibility for
>         the message occurs: the protocol requires that a server
>         MUST accept responsibility for either delivering a
>         message or properly reporting the failure to do so.
 
> I think this is a simple rewording of a clear statement.

It's fine.  Of course my idea of "properly reporting" is to the
"originator", and where that's dubious it's no proper reporting:

It's the duty of the SMTP server to decide if it can take this
responsbility, and it MUST reject the mail when it's in doubt
about the originator.  Otherwise it can't report properly, and
would violate this clear MUST.  That's why there is a 551, and
not only a 251.

Frank

Date: Thu, 26 Apr 2007 22:41:10 -0400
From: John C Klensin <[email-masked]>
To: Douglas Otis <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-3 nits and typos
Sender: [email-masked]

--On Thursday, 26 April, 2007 18:35 -0700 Douglas Otis
<[email-masked]> wrote:

> 
> I have been unable to keep up with the ongoing threads, but
> had a quick look at the document.  Good work!
> 
> Here are a few minor comments that hopefully will not be going
> over old ground.
> 
> ----
> Abstract:
> 
> It consolidates, updates, and clarifies several previous
> documents, making all or parts of most of the obsolete.
> 
> Typo: change "most of the" to "most of them"

Amazing that no one has caught this before, but fixed.

> ----
> 2.1. Basic Structure:
> 
>   ...delivering a message or properly reporting the failure to
> do so.
> 
> Nit: Change "reporting failure to do so." to "reporting
> failure to the originator address.  When a possibility has
> been determined that a failure subsequent to acceptance will
> not be reported, refusing the message better retains delivery
> status integrity and is generally preferred."

That is not a nit, but part of what is now issue 25.
 
> ----
> 2.3.2. Senders and Receivers
> 
> Nit: Consider changing "Sender" to "Transmitter",
> "Transmission Channel", or "Dispatcher" to avoid confusion
> with "Sender:".  Perhaps adopt server/client rather than abuse
> the word sender.  Although this will touch many sentences,
> such a change will provide greater clarity going forward.

IMO, too large a change, expecially since SMTP-Sender and
SMTP-Receiver (the terms used in 821) is the terminology of last
resort when terms based on "client" and "server" are
insufficiently precise.  I can try this if enough people think
it is important but, based on the experience with changing
informal text to SHOULD/ MAY/ MUST terminology --which I now
expect to have to back out-- I recommend against embarking on
more adventures of this type if people really want to get this
done anytime soon... like this calendar year.

> ----
> 3.3. Mail Transactions
> 
> When the end of text is successfully received and stored the
> SMTP-receiver sends a 250 OK reply.
> 
> Typo: Add comma.
> 
> When the end of text is successfully received and stored, the
> SMTP-receiver sends a 250 OK reply.

Changed.
 
> ----
> 3.6. Relaying and Mail Routing
> 
> ...Of course this prohibition also applies to any
> modifications of these fields or text.
> 
> Nit: Append to this paragraph:
> 
> See section 7.9., Scope of Operation of SMTP Servers.

Reference added.  Not sure it is a good idea.

> ----
> 4.1.1.1. Extended HELLO (EHLO) or HELLO (HELO)
> 
>   Private-use commands MAY be listed .
> 
> Typo: change "MAY be listed ." to  "MAY be listed."

Fixed.

Thanks,
   john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Date:  Fri, 27 Apr 2007 04:24:58 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 9
Sender: [email-masked]

John C Klensin wrote:

> Should that text be retained, dropped, or modified in some way?

Retained, it's obviously relevant measured by the length of the
1yz thread.  And it doesn't bite anybody, it states a mere fact.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  2821bis-03
Date:  Fri, 27 Apr 2007 03:57:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 143
Sender: [email-masked]

http://tools.ietf.org/rfcdiff?url2=http%3A%2F%2Ftools.ietf.org%2Fid%2Fdraft-klensin-rfc2821bis-03.txt&difftype=--hwdiff

Top down only the diffs from -02 or what I see:

- What was wrong with "obsoletes 2821" (now "obsoletes 821") ?
  The complete list of RFCs obsoleted by 2821 is longer.
- appendix C (source routes) is still there.

| including any lines in the header section or other structure
* including any other structure like lines in the header section

Or maybe get rid of the "or other structure", it's hard to parse
the sentence after you've replaced the "headers".

- 2.3 Terminology:  Please use the BCP 14 terminology like most
  other RFCs.

| A MIME extension (RFC2047 [30]) defines an
* Two MIME extensions (RFC2047 [30] and RFC2231 [??]) define an

Those language tags in RFC 2231 are okay.

| EHLO command MUST BE
* EHLO command MUST be

| Recent work, RFC3463 [39], has specified
* RFC3463 [39] specifies

RFC 1893 is no recent work anymore, RFC 3463 is a draft standard.

| However, it MUST not be construed as authorization
* However, it MUST NOT be construed as authorization

I've a script to highlight "MUSTard".  Maybe a future version of
the idnits offers an option -s "sofixit".

- 2.3.11:  "command" + "verb", let's forget it, 2.4 is the next
  section and explains what's what right at the begin.

| However, in practice, some servers do not perform recipient
| verification until after the message text is received.  These
| servers SHOULD treat a failure for one or more recipients as
| a "subsequent failure" and return a mail message as discussed
| in Section 6 and, in  particular, in Section 6.1.

No, that's precisely what they SHOULD NOT do, because it's net
abuse in the case of unverified reverse paths.  It will get them
blacklisted, and after that they won't be able to report serious
problems.  There should not be any "you SHOULD spam" in 2821bis.

| Server SMTP systems SHOULD NOT reject messages based on
| perceived defects in the RFC 822 or MIME (RFC2045 [21]) message
| header section or message body.

Rejecting broken mail objects is an option for 8BITMIME, a duty
for EAI, and a duty for 4409.  This SHOULD NOT is utter dubious.

| In particular, they MUST NOT reject messages in which the
| numbers of Resent-fields do not match or Resent-to appears
| without Resent-from and/or Resent- date.

Delete this, please.  Receivers can reject whatever they don't
like.  The complete Resent-* concept is unnecessary in a world
using MIME.  If some servers try to count Resent-* blocks they
are mad.  But that's IMO a harmless madness, not a MUST NOT.

| if a 551 code is used, they MUST NOT assume that the client
| will actually update address information or even return that
| information to the user.

Please delete or "or even" etc., clients can and should return
the complete error reply to the user.  I also don't see the
point in "MUST NOT", you could just say "cannot" also for 251,
it's a fact, not an order.

The 251/551 section in RFC 821 is IMO clearer,  This needs more
discussion, it misses the point of taking the responsibility for
the mail.  That's by definition not possible for an unverified
reverse path, and 551 is a very sound alternative to 250/251
for servers not wishing to get into any trouble with bounces.

550 is also okay, but unlike 551 not very helpful for both sides.

- 3.5:  There are many at least potentially valid addresses in
  this chapter, please add "TLD" .example everywhere, and maybe
  replace foo.com by example.com

- 3.6:  I guess you'll update that when appendix C is eliminated,
  therefore I skip it for now.

| Gateways are, of course, subject to the same rules for handling
| source routes as those described for other SMTP systems in
| Section 3.3

3.3 also discusses source routes, but the main part is in 3.6 (?)

| The translation algorithm used to convert mail from the Internet
| protocols to another environment's protocol SHOULD ensure that
| error messages from the foreign mail environment are delivered
| to the return path from the SMTP envelope, not to the sender
| listed in the "From:" field (or other fields) of the message body.

Normally you write "reverse path", is it intentionally a "return
path" here, because you're talking about gateways ?  For the part
about the "sender" I propose:

* not to an address in the "From:", "Sender:", or similar header
* fields of the message.

That should cover Sender:, Resent-From:, Resent-Sender:, Reply-To:.
IMO "message body" is misleading if you actually mean the complete
message (header and body), or here especially its header.  Maybe
"mail object" or "message data" is better if "message" isn't clear.

| If the foreign environment has no equivalent concept, the gateway
| must select and use a best approximation, with the message
| originator's address as the default of last resort.

I think it should use the address of postmaster@ (or similar) at
the gateway, and not a random address in something it made up while
translating foreign addresses to Internet addresses as specified in
3.7.4.  The gateway is responsible, and errors should be reported
via the gateway.  It's the same RFC 1123 problem as for forwarders,
and the solution in 1123 5.2.16 keeps the responsibility where it
belongs, at the gateway.  5.2.16 is a "sender rewriting system".

| Note that the key difference between handling aliases (Section
| 3.9.1) and forwarding (this subsection) is the change to the
| backward-pointing address in this case.

That's a point where you could mention that this used to be no
key difference under RFC 821, because "in any case, the SMTP"
added "its own identifier to the reverse path".  There might be
better places to explain that RFC 1123 broke the original SMTP
design in its quest to get rid of the source routes.

Okay, 3 chapters out of 17 (with appendices), enough for today.
One disadvantage of minor changes in a diff is that I start to
look at text I haven't read for years only because there's a
diff mark, and then I start to compare 821/1123 with 2821bis...

Frank

To: [email-masked]
From: Douglas Otis <[email-masked]>
Subject: rfc2821bis-3 nits and typos
Date: Thu, 26 Apr 2007 18:35:27 -0700
Sender: [email-masked]

I have been unable to keep up with the ongoing threads, but had a  
quick look at the document.  Good work!

Here are a few minor comments that hopefully will not be going over  
old ground.

----
Abstract:

It consolidates, updates, and clarifies several previous documents,  
making all or parts of most of the obsolete.

Typo: change "most of the" to "most of them"

----
2.1. Basic Structure:

  ...delivering a message or properly reporting the failure to do so.

Nit: Change "reporting failure to do so." to "reporting failure to  
the originator address.  When a possibility has been determined that  
a failure subsequent to acceptance will not be reported, refusing the  
message better retains delivery status integrity and is generally  
preferred."

----
2.3.2. Senders and Receivers

Nit: Consider changing "Sender" to "Transmitter", "Transmission  
Channel", or "Dispatcher" to avoid confusion with "Sender:".  Perhaps  
adopt server/client rather than abuse the word sender.  Although this  
will touch many sentences, such a change will provide greater clarity  
going forward.

----
3.3. Mail Transactions

When the end of text is successfully received and stored the SMTP- 
receiver sends a 250 OK reply.

Typo: Add comma.

When the end of text is successfully received and stored, the SMTP- 
receiver sends a 250 OK reply.

----
3.6. Relaying and Mail Routing

...Of course this prohibition also applies to any modifications of  
these fields or text.

Nit: Append to this paragraph:

See section 7.9., Scope of Operation of SMTP Servers.

----
4.1.1.1. Extended HELLO (EHLO) or HELLO (HELO)

  Private-use commands MAY be listed .

Typo: change "MAY be listed ." to  "MAY be listed."

-Doug

Date: Thu, 26 Apr 2007 16:19:48 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-03 Issue  22: Return-path required removal
Sender: [email-masked]

Reviewing this issue briefly, there two connected questions 

First, whether the current language that indicated that old
Return-path headers MAY be removed in systems that have to
rebuild headers and construct new envelopes, that originating
systems should not send headers containing return-paths, that
ordinary relays must not tamper with headers to the extent of
determining whether return-paths are present, much less to
remove them, and that final delivery servers may, but are not
required to, remove earlier return paths.

Second whether a reference to RFC3834, "Recommendations for
Automatic Responses to Electronic Mail" should be added to
2821bis.

I have now reviewed these issues with Keith Moore, the author of
3834.   We have concluded that the current restrictions are
about right: the only question is whether delivery servers,
which need to add a Return-path of their own should be required
(presumably with a SHOULD, possibly a MUST) to verify that there
are no Return-path header lines present, and removing them if
needed, before adding one.  Requiring it would not match current
practice in several cases, but, if people are convinced that it
was necessary to ensure good interoperability, it could be done.
We are not convinced that it is necessary.

It is clear to us that RFC 3834, were it ever upgraded for
Draft, should get some additional text specifying that, if
multiple Return-paths are present, only the "top line" one
counts or, perhaps better, that only the "top line" one counts
whether or not there are others present.

Given the above, we don't see much value in a reference from
2821bis to 3834, although we don't see it as harmful either.

On reviewing the relevant text, I think the two paragraphs that
discuss this could probably be reworked a bit to make it more
clear that they do not contradict each other.  I've added that
to my list of things to try to do for -04.

Just our opinions, although I would hope that they are somewhat
informed.

     john

Date: Thu, 26 Apr 2007 13:18:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Dave Crocker wrote:
> 
> I see two issues with this text, suggesting the need for some added 
> text.  It's not that there is anything "wrong" with the proposed text, 
> but that it is missing some detail, and certainly none of this would 
> change SMTP semantics:
> 
> 1. A server is not "always" required to accept responsibility.  Rather, 
> it is required to accept responsibility after issue a success response 
> to some part of the protocol.  Until it issues that response, it is NOT 
> responsible for the message; the client is.
> 
> 2. In other words, what is missing is reference to the state-change that 
> carries the formal transfer of responsibility from client to server -- 
> and it needs to be stated in terms of both client and server I assume 
> that it is a 250 reply to DATA, but wouldn't stake my life on that.

+1.

In my view, the sticky point (both technical/functional and dare I say, 
legal) is message acceptance and then failing to deliver and also 
failing to notify the original sender.

Add to this sticky point the following statement in 821/2821 and 2821bis 
for "Scope of Operation of SMTP Servers":

    It is a well-established principle that an SMTP server may refuse to
    accept mail for any operational or technical reason that makes sense
    to the site providing the server.

which we often refer to as "Local Policy" decisions or reasons.

So we (server developers) have a small conflict between the technical 
requirements to maintain mail flow and delivery status completion, and a 
relaxation to provide policy requirements or features to operators.

So I think if we can express the "Must Take Responsibility" statement to 
one that a technical functional requirement, then specs will cover both 
specs, technical and policy (and dare I say, legal).

I think that will entail moving the term "responsibility" to the policy 
portion of the specs. So how about this:

2.  The SMTP Model

    In other words, message transfer can occur in a single connection
    between the original SMTP-sender and the final SMTP-recipient, or can
    occur in a series of hops through intermediary systems.  In either
    case, a formal transaction handoff for the message occurs: the
    protocol requires that a server MUST provide a mechanism to either
    delivering a message or properly reporting the failure to do so.

7.9  Scope of Operation of SMTP Servers

    It is a well-established principle that once a message is accepted
    at the DATA state, the site MUST take responsibility to deliver
    the message or to provide a non-delivery notification.  However, it
    is also an accepted principle, site local policies may refuse to
    accept mail or deliver notifications for any operational
    reason that makes sense to the site providing the server.

I think the above can be said better, but just trying to stress that
SMTP is a "mechanism" that is programmed and designed to follow an 
automation concept.  If the local policy wishes to change that 
automation than IMO it is (the site) that is inheriting the responsibility.

--
HLS

Date: Thu, 26 Apr 2007 12:40:33 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Peter J. Holzer wrote:

> 1) The entity responsible for sending the message (the "real sender")
> 2) The entity indicated by the reverse-path in the MAIL command (the
>    "alleged sender")

[...]

> And while I think that an MTA should make every effort to determine
> whether it can deliver the message before taking responsibility for it,
> I concede that there are situations where the MTA can only determine
> that it won't be able (or willing) to deliver the message AND that the
> reverse-path is very likely to be forged after it has accepted the
> message. In that case I believe it is better to silently drop the
> message than to send a NDN to an entity which is almost certainly not
> the real sender.

> Therefore I think that should only be a SHOULD, not a MUST. I'm not too
> happy with just changing MUST to SHOULD, though. Maybe a longer
> discussion of the alternatives is in order.

Well, I think we need to define exactly what is meant by:

      "properly reporting the failure [to deliver a message]"

Currently, I think everyone understands "properly reporting the
failure to deliver" to mean either a 5yz code during the SMTP session,
or a NDN to the Return-Path.  I think it's opening a huge can of worms
to allow any other meaning of a "proper failure report."  I also think
it's inevitable that SMTP server implementations will violate this
requirement for various policy reasons.

So I think a SHOULD is the only way around it.

Regards,

David.

Date: Thu, 26 Apr 2007 12:35:02 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Hector Santos wrote:

> Do you think this and your other points are already covered in 4.5.5 in
> particular with the 2nd paragraph?

>    All other types of messages (i.e., any message which is not required
>    by a standards-track RFC to have a null reverse-path) SHOULD be sent
>    with a valid, non-null reverse-path.

No, I don't think so.  I don't see anything that explicitly says
that a "proper failure report" is either a 5yz code during the SMTP
session or a non-delivery notification sent to the Return-Path.  I mean,
*every* SMTP implementor *knows* this, but I think it should be spelled
out.

Regards,

David.

Date: Thu, 26 Apr 2007 09:31:50 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Peter J. Holzer wrote:
  > There are two concepts of "sender":
     "alleged sender")

Actually, more than two.

For example, see:

<http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_8-4/anti-spam_efforts.html>

   Table 1.

Each of the entries can be called a "sender".

But you get extra credit for being one of the few folk to worry about
the nuances of the term.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Thu, 26 Apr 2007 12:26:54 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> There is no server notification "message" requirement for NULL return
>> paths.  That would be definitely be a server implementation (feature)
>> concept.
> 
> OK.  I forgot that.  How about just this:
> 
>     If the original message has a non-null return-path, then a
>     proper failure report MUST consist of a notification to the
>     sender of the message.
> 

Hi David,

Do you think this and your other points are already covered in 4.5.5 in 
particular with the 2nd paragraph?

    All other types of messages (i.e., any message which is not required
    by a standards-track RFC to have a null reverse-path) SHOULD be sent
    with a valid, non-null reverse-path.

--
HLS

Date: Thu, 26 Apr 2007 18:17:01 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--7AUc2qLy4jB3hD7Z
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-26 11:19:32 -0400, David F. Skoll wrote:
> Hector Santos wrote:
> > There is no server notification "message" requirement for NULL return
> > paths.  That would be definitely be a server implementation (feature)
> > concept.
>=20
> OK.  I forgot that.  How about just this:
>=20
>     If the original message has a non-null return-path, then a
>     proper failure report MUST consist of a notification to the
>     sender of the message.

I'm a bit uneasy about the "MUST" here.=20

There are two concepts of "sender":

1) The entity responsible for sending the message (the "real sender")

2) The entity indicated by the reverse-path in the MAIL command (the
   "alleged sender")

In general, an MTA which just accepted a message from another MTA has no
way of reliably determining the real sender and has to assume that the
alleged sender is identical with the real sender.

Of course we know that this assumption is often not true.

And while I think that an MTA should make every effort to determine
whether it can deliver the message before taking responsibility for it,
I concede that there are situations where the MTA can only determine
that it won't be able (or willing) to deliver the message AND that the
reverse-path is very likely to be forged after it has accepted the
message. In that case I believe it is better to silently drop the
message than to send a NDN to an entity which is almost certainly not
the real sender.

Therefore I think that should only be a SHOULD, not a MUST. I'm not too
happy with just changing MUST to SHOULD, though. Maybe a longer
discussion of the alternatives is in order.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--7AUc2qLy4jB3hD7Z
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--7AUc2qLy4jB3hD7Z--

Date: Thu, 26 Apr 2007 09:00:53 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

John C Klensin wrote:
> As part of the issue 16 cleanup (changing informal statements
> about requirements to more formal, 2119-like, language), the
> critical text about receiver responsibility for messages they
> accepted was changed from 
> 2821bis-03:
> 
> 	The responsibility of an SMTP client is to transfer mail
> 	messages to one or more SMTP servers, or report its
> 	failure to do so.
>  
> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.

I see two issues with this text, suggesting the need for some added 
text.  It's not that there is anything "wrong" with the proposed text, 
but that it is missing some detail, and certainly none of this would 
change SMTP semantics:

1. A server is not "always" required to accept responsibility.  Rather, 
it is required to accept responsibility after issue a success response 
to some part of the protocol.  Until it issues that response, it is NOT 
responsible for the message; the client is.

2. In other words, what is missing is reference to the state-change that 
carries the formal transfer of responsibility from client to server -- 
and it needs to be stated in terms of both client and server I assume 
that it is a 250 reply to DATA, but wouldn't stake my life on that.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Thu, 26 Apr 2007 11:19:32 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Hector Santos wrote:

> There is no server notification "message" requirement for NULL return
> paths.  That would be definitely be a server implementation (feature)
> concept.

OK.  I forgot that.  How about just this:

    If the original message has a non-null return-path, then a
    proper failure report MUST consist of a notification to the
    sender of the message.

Regards,

David.

Date: Thu, 26 Apr 2007 11:16:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

David F. Skoll wrote:
> John C Klensin wrote:
> 
>> 	In either case, a formal handoff of responsibility for
>> 	the message occurs: the protocol requires that a server
>> 	MUST accept responsibility for either delivering a
>> 	message or properly reporting the failure to do so.
> 
>> I think this is a simple rewording of a clear statement.
> 
> The only unclear thing to me is:
> 
> "...properly reporting the failure to do so."
> 
> Does everyone understand this to mean reporting the failure
> to the sender (in the case of a non-null Return-Path) or to
> some administrator (in the case of a null Return-Path)?  If everyone
> clearly understands it this way, then it's fine.  Otherwise, we may
> want to spell it out.  Otherwise, someone might consider dropping
> an e-mail and letting an administrator know about it to be
> "properly reporting the failure".
> 
> How about:
> 
>  	In either case, a formal handoff of responsibility for
>  	the message occurs: the protocol requires that a server
>  	MUST accept responsibility for either delivering a
>  	message or properly reporting the failure to do so.
> 	A proper failure report MUST consist of a notification to the
> 	sender of the message if the original message has a non-null
> 	return-path, or a notification to an administrator of the
> 	SMTP server if the message has a null return-path.
> 
> Or am I worrying too much? :-)

Hi David,

There is no server notification "message" requirement for NULL return 
paths.  That would be definitely be a server implementation (feature) 
concept.

For example, you have two models for this:

Dynamic (SMTP transaction level) Notification

    C: MAIL FROM:<>
    S: 250 OK Null Accept for Local Notification Delivery only
    C: RCPT TO:<unknown @ localdomain.com>
    S: 550 Sorry, doesn't exist

Post SMTP Notification (No Local User Validation or Local Hosted Domain)

    C: MAIL FROM:<>
    S: 250 OK Null Accept for Local Notification Delivery only
    C: RCPT TO:<localuser @ localdomain.com>
    S: 250 OK
    C: DATA
    S: 354 Send Data
    C: [upload data]
    S: 250 Message Accepted

The post smtp processor handles the message. If not deliverable to the 
local user or hosted domain, a bounce is NOT REQUIRED to be sent because 
there isn't a return path.

Each implementation can handle that their own way.  It is local policy 
driven.

In general, if it passes some security layer (AVS, etc) then its may be 
posted to some predefined local admin address or each trashed.  But this 
is not a requirement for SMTP.

Good Local Policy Feature for SMTP server to offer? Sure, but not a SMTP 
requirement.

--
HLS

Date: Thu, 26 Apr 2007 11:16:46 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

Jeff Macdonald wrote:

> Hmmm.... I read it as this:

> MUST accept: server returns a 250
> failure to do so: server returns a 5xx or 4xx

I think the paragraph refers to the situation in which a server accepts
a message with 250, and then subsequently finds it is unable to deliver
the message.  While this is not a desirable situation, it's sometimes
impossible to avoid.

Regards,

David.

Date: Thu, 26 Apr 2007 10:53:15 -0400
From: Jeff Macdonald <[email-masked]>
To: "David F. Skoll" <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

On Thu, Apr 26, 2007 at 10:15:59AM -0400, David F. Skoll wrote:
> 
> John C Klensin wrote:
> 
> > 	In either case, a formal handoff of responsibility for
> > 	the message occurs: the protocol requires that a server
> > 	MUST accept responsibility for either delivering a
> > 	message or properly reporting the failure to do so.
> 
> > I think this is a simple rewording of a clear statement.
> 
> The only unclear thing to me is:
> 
> "...properly reporting the failure to do so."
> 
> Does everyone understand this to mean reporting the failure
> to the sender (in the case of a non-null Return-Path) or to
> some administrator (in the case of a null Return-Path)?  If everyone
> clearly understands it this way, then it's fine.  Otherwise, we may
> want to spell it out.  Otherwise, someone might consider dropping
> an e-mail and letting an administrator know about it to be
> "properly reporting the failure".
> 
> How about:
> 
>  	In either case, a formal handoff of responsibility for
>  	the message occurs: the protocol requires that a server
>  	MUST accept responsibility for either delivering a
>  	message or properly reporting the failure to do so.
> 	A proper failure report MUST consist of a notification to the
> 	sender of the message if the original message has a non-null
> 	return-path, or a notification to an administrator of the
> 	SMTP server if the message has a null return-path.
> 
> Or am I worrying too much? :-)

Hmmm.... I read it as this:

MUST accept: server returns a 250
failure to do so: server returns a 5xx or 4xx

This is because the word 'protocol' appeared and my brain thinks in
terms of SMTP conversations when it sees protocol.

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Thu, 26 Apr 2007 10:48:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

John C Klensin wrote:
> rfc2821bis-03 adds new text to explicitly compare SMTP's reply
> code model to that of FTP in the hope of eliminating any
> possible residual confusion about issues 18 and 28.  Should that
> text be retained, dropped, or modified in some way?

I vote for the Drop.

As with Chris Haynes, I think the new "up-to-date definition of the 
reply code model should be complete and self-sufficient" on both counts; 
persistent reply codes and eliminating of the 1yz preliminary code.

Case in point: While our own FTP product experience helped carry the 
similar idea to possible mix-code, 1yz- usage for SMTP as well,  the 
consideration for mix codes may not only be from a FTP perspective but 
rather one simply based on the now declared incorrect ambiguous current 
"last code" only interpretation as shown by this SMTP client developer:

  http://utcc.utoronto.ca/~cks/space/blog/tech/SmtpResultConundrum

This fella was also surprised to find out that the popular Exim was not 
reading the reply codes based on a "last code" interpretation which he 
presumed "that's how everyone operated."

-- 
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

Date: Thu, 26 Apr 2007 10:15:59 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

John C Klensin wrote:

> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.

> I think this is a simple rewording of a clear statement.

The only unclear thing to me is:

"...properly reporting the failure to do so."

Does everyone understand this to mean reporting the failure
to the sender (in the case of a non-null Return-Path) or to
some administrator (in the case of a null Return-Path)?  If everyone
clearly understands it this way, then it's fine.  Otherwise, we may
want to spell it out.  Otherwise, someone might consider dropping
an e-mail and letting an administrator know about it to be
"properly reporting the failure".

How about:

 	In either case, a formal handoff of responsibility for
 	the message occurs: the protocol requires that a server
 	MUST accept responsibility for either delivering a
 	message or properly reporting the failure to do so.
	A proper failure report MUST consist of a notification to the
	sender of the message if the original message has a non-null
	return-path, or a notification to an administrator of the
	SMTP server if the message has a null return-path.

Or am I worrying too much? :-)

Regards,

David.

Date: Thu, 26 Apr 2007 10:04:56 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

On Thu, Apr 26, 2007 at 08:35:54AM -0400, John C Klensin wrote:
> 2821bis-03:
> 
> 	The responsibility of an SMTP client is to transfer mail
> 	messages to one or more SMTP servers, or report its
> 	failure to do so.
>  
> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.

+1

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Thu, 26 Apr 2007 10:03:10 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

On Thu, Apr 26, 2007 at 08:20:12AM -0400, John C Klensin wrote:
> 
> rfc2821bis-03 adds new text to explicitly compare SMTP's reply
> code model to that of FTP in the hope of eliminating any
> possible residual confusion about issues 18 and 28.  Should that
> text be retained, dropped, or modified in some way?

dropped

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Thu, 26 Apr 2007 14:40:12 +0100
From: Chris Haynes <[email-masked]>
Reply-To: Chris Haynes <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

On Thursday, April 26, 2007 at 1:20:12 PM, John C Klensin wrote:
> rfc2821bis-03 adds new text to explicitly compare SMTP's reply
> code model to that of FTP in the hope of eliminating any
> possible residual confusion about issues 18 and 28.  Should that
> text be retained, dropped, or modified in some way?

Please accept a comment from a lurker: I would find the references to FTP confusing and unhelpful.

They might have been 'helpful' when SMTP was a novelty and FTP well-established, aiding those who were presumed to already understand FTP, but today I see no value in the comparison. Any up-to-date definition of the reply code model should be complete and self-sufficient. 

Chris Haynes

Date: Thu, 26 Apr 2007 08:36:15 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: SMTP Interest Group <[email-masked]>
Subject: Re: 2821bis-03 comments
Sender: [email-masked]

--On Thursday, 26 April, 2007 03:27 -0400 Hector Santos
<[email-masked]> wrote:

> 
> Great work John!
> 
> Just a few comments.
> 
> 1) Removal of 1yz in 4.2.1
> 
> With the the removal of 1yz, it is probably a good idea to use
> a different example for reply codes in 4.2.1:
> 
>     For example:
> 
>        123-First line
>        123-Second line
>        123-234 text beginning with numbers
>        123 The last line

Fixed in -04 working copy

> 2) Recording the new FTP text in 4.2.1
> 
>     It is worth noting that the file transfer protocol (FTP
> [35]) uses a
>     very similar code architecture and that the SMTP codes are
> based on
>     the FTP model.  However, SMTP uses a one-command,
> one-response model,
>     while FTP is asynchronous and the 1yz codes are not part
> of the SMTP
>     model.
> 
> IMO, I am not sure this is necessary John. Might be best to
> not confuse future developers with this history note,
> especially since 1yz is not longer part of the spec and even
> more since one can argue the that DATA command is a one
> command, two response model. In a  sequence of:
> 
>      C: DATA
>      S: 354
>      C: [upload data]  <--- this is technically not a command
>      S: 250
> 
> The client is [arguably] not sending a new command to 354, but
> rather the mail block.

I didn't know whether to include that or not, but thought the
most efficient way to get comments was to put it in.   Comments
from others welcome --it is easily removed or tuned if needed/
desired.  I decided to put it in after you pointed us to Charles
Kozierok's "TCP/IP Guide" piece and a brief exchange of notes
with the author who apparently was, in fact, confused by the FTP
comparison.   So that I don't lose track, Issue 31 is assigned
to this.
 
> 3) In 2.1, you added a MUST to this line:
> 
>     protocol requires that a server MUST accept responsibility
> for either
>     delivering a message or properly reporting the failure to
> do so.
> 
> BRAVO, BRAVO!!
> 
> However, I am sending you an off-list comment about this.

Keep in mind that this wasn't intended to be a change of
substance, but a simple rewording of "...protocol requires that
a server accept..." per issue 16.   If people believe this is a
change in anything but wording, I should revert the text.
Issue 32.
 
> Any who, I think with the MUST above, you probably should
> upper case some "shoulds" in section 6.2 and I think if you
> want to appease the mail process/AVS service market, I think
> it may be appropriate to remove the clause regarding of
> "syntax" and "mail content":
> 
> I think it can be just:
> 
>     Utility and predictability of the Internet mail system
> requires that
>     messages that can be delivered SHOULD be delivered.  If
>     a message cannot be delivered, and cannot be rejected by
> the SMTP
>     server during the SMTP transaction, the message SHOULD be
> "bounced"
>     (returned with non-delivery notification messages) as
> described
>     above.  .........

I will take a look at the "shoulds" -- didn't recheck those on
the last pass.   My, and I hope our, goal here is to clarify
what the spec says and, as appropriate, to reflect existing
practice in the sending and handling of normal mail messages,
rather than to "appease" anyone.  I think this is part of issue
25.
 
> and in my opinion, the addition of the sentence would be
> helpful:
> 
>     If applicable, the SMTP server SHOULD attempt to determine
>     delivery status during the SMTP transaction to help
> minimize
>     undesirable BOUNCE traffic.

thanks for the careful reading.
    john

Date: Thu, 26 Apr 2007 08:35:54 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

As part of the issue 16 cleanup (changing informal statements
about requirements to more formal, 2119-like, language), the
critical text about receiver responsibility for messages they
accepted was changed from 

RFC 2821:

(from just below the figure)
	The responsibility of an SMTP client is to transfer mail
	messages to one or more SMTP servers, or report its
	failure to do so.

(mid-section)
	In either case, a formal handoff of responsibility for
	the message occurs: the protocol requires that a server
	accept responsibility for either delivering a message or
	properly reporting the failure to do so.

2821bis-03:

	The responsibility of an SMTP client is to transfer mail
	messages to one or more SMTP servers, or report its
	failure to do so.
 
	In either case, a formal handoff of responsibility for
	the message occurs: the protocol requires that a server
	MUST accept responsibility for either delivering a
	message or properly reporting the failure to do so.

I think this is a simple rewording of a clear statement.  If
people believe otherwise, the text should probably just be
reverted (unless more drastic changes are implemented as the
result of issue 25 (seriously restricting NDN messages.  I think
the apparent (but IMO not actual) redundancy between the
relevant two sentences is helpful, but would appreciate comments
on that subject if anyone disagrees.

       john

Date: Thu, 26 Apr 2007 08:20:12 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

rfc2821bis-03 adds new text to explicitly compare SMTP's reply
code model to that of FTP in the hope of eliminating any
possible residual confusion about issues 18 and 28.  Should that
text be retained, dropped, or modified in some way?

Date: Thu, 26 Apr 2007 03:27:05 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SMTP Interest Group <[email-masked]>, John C Klensin <[email-masked]>
Subject: 2821bis-03 comments
Sender: [email-masked]

Great work John!

Just a few comments.

1) Removal of 1yz in 4.2.1

With the the removal of 1yz, it is probably a good idea to use a 
different example for reply codes in 4.2.1:

    For example:

       123-First line
       123-Second line
       123-234 text beginning with numbers
       123 The last line

2) Recording the new FTP text in 4.2.1

    It is worth noting that the file transfer protocol (FTP [35]) uses a
    very similar code architecture and that the SMTP codes are based on
    the FTP model.  However, SMTP uses a one-command, one-response model,
    while FTP is asynchronous and the 1yz codes are not part of the SMTP
    model.

IMO, I am not sure this is necessary John. Might be best to not confuse 
future developers with this history note, especially since 1yz is not 
longer part of the spec and even more since one can argue the that DATA 
command is a one command, two response model. In a  sequence of:

     C: DATA
     S: 354
     C: [upload data]  <--- this is technically not a command
     S: 250

The client is [arguably] not sending a new command to 354, but rather 
the mail block.

3) In 2.1, you added a MUST to this line:

    protocol requires that a server MUST accept responsibility for either
    delivering a message or properly reporting the failure to do so.

BRAVO, BRAVO!!

However, I am sending you an off-list comment about this.

Any who, I think with the MUST above, you probably should upper case 
some "shoulds" in section 6.2 and I think if you want to appease the 
mail process/AVS service market, I think it may be appropriate to remove 
the clause regarding of "syntax" and "mail content":

I think it can be just:

    Utility and predictability of the Internet mail system requires that
    messages that can be delivered SHOULD be delivered.  If
    a message cannot be delivered, and cannot be rejected by the SMTP
    server during the SMTP transaction, the message SHOULD be "bounced"
    (returned with non-delivery notification messages) as described
    above.  .........

and in my opinion, the addition of the sentence would be helpful:

    If applicable, the SMTP server SHOULD attempt to determine
    delivery status during the SMTP transaction to help minimize
    undesirable BOUNCE traffic.

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 25 Apr 2007 16:07:17 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: I-D ACTION:draft-klensin-rfc2821bis-03.txt
Sender: [email-masked]

rfc2822bis-03 is now out at one of these URLs:

http://www.ietf.org/internet-drafts/draft-klensin-rfc2821bis-03.txt
http://tools.ietf.org/html/draft-klensin-rfc2821bis-03.txt

A diff from -02 can be seen here:

http://tools.ietf.org/rfcdiff?url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-03.txt

	Tony Hansen
	[email-masked]

[email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts 
> directories.
> 
> 	Title		: Simple Mail Transfer Protocol
> 	Author(s)	: J. Klensin
> 	Filename	: draft-klensin-rfc2821bis-03.txt
> 	Pages		: 94
> 	Date		: 2007-4-25
> 	
> This document is a largely self-contained specification of the basic
>    protocol for the Internet electronic mail transport.  It
>    consolidates, updates, and clarifies several previous documents,
>    making all or parts of most of the obsolete.  It covers the SMTP
>    extension mechanisms and best practices for the contemporary
>    Internet, but does not provide details about particular extensions.
>    Although SMTP was designed as a mail transport and delivery protocol,
>    this specification also contains information that is important to its
>    use as a "mail submission" protocol for "split-UA" mail reading
>    systems and mobile environments.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-klensin-rfc2821bis-03.txt

Date: Wed, 25 Apr 2007 13:39:49 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--ctP54qlpMx3WjD+/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

I consider that discussion closed, so I won't bring any new arguments,
but I think a short explanation is in order.

On 2007-04-23 05:29:02 -0400, John C Klensin wrote:
> --On Monday, 23 April, 2007 01:09 +0200 "Peter J. Holzer"
> <[email-masked]> wrote:
>=20
> > On 2007-04-22 16:46:06 -0400, Hector Santos wrote:
> >> What are you basically implying that we have a problem with
> >> clients not  supporting the 25 year specification
> >=20
> > No, we are concerned that clients which are supporting the 25
> > year old specification will be rendered non-compliant if your
> > interpretation is formalized.
>=20
> Peter, while I agree, let's be a little careful.

I thought I was, mentioning that the RFC(2)821 wasn't entirely clear at
least once per mail.

> Assume that one reasonable person can read the text as saying both
[...]
> Assume that another reasonable person can read the same text and

I do assume that, and in fact it was my aim to convince Hector that
there are different ways to interpret RFC 821 and that his
interpretation is not the only valid one (I also think that his is a
rather far-fetched interpretation, but that was secondary to my
argument). In that I have failed (he still refers to software not
supporting his view as "broken"). Since you obviously also missed what I
was arguing about, I must have been rather unclear. For that I
apologize.

	hp

PS: Thanks to Tony for analyzing some widely MTAs and adding facts to
this thread.

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--ctP54qlpMx3WjD+/
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--ctP54qlpMx3WjD+/--

Date: Wed, 25 Apr 2007 13:05:20 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--kORqDWCi7qDJ0mEj
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-24 12:58:53 +0200, Arnt Gulbrandsen wrote:
> Frank Ellermann writes:
> >John C Klensin wrote:
> >> Can everyone live with that, more or less? If not, please suggest=20
> >> alternate text and/or convince Tony that this particular dead horse=20
> >> needs more kicking.
> >
> >Are you saying "remove the complete 1yz cruft and be done with it" ?=20
> >Sounds like a brilliant idea.
>=20
> +1 and I'll buy all concerned a beer at the very next opportunity.

+1.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--kORqDWCi7qDJ0mEj
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--kORqDWCi7qDJ0mEj--

Date: Wed, 25 Apr 2007 00:53:43 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Updated status on 2821bis 4/24
Sender: [email-masked]

This is a recap of where we stand on issues that I consider to be closed
with respect to 2821bis-02. John is getting ready to submit -03 and
these will be included. I will send out separate messages on other
issues that are not yet closed.

Thank you everyone for your help.

By the way, the rate of adding new items vs. closing items *is* sloping
in the right direction.

	Tony Hansen
	[email-masked]

The following issues are closed. There has been no activity requesting
changes since -02 was posted:

1	Format of domain - trailing period
8	Number of recipients/ number of transactions per connection

Text has been changed in -03 for the following issues:

3	EHLO validation in sect 4.1.4, para 6
16	General use of "is permitted", etc
17	Should text be clarified to single code in continuations
	and dealing with 1yz
26	Revise description of reverse-paths to prevent repeated
	citations of "server adds its own…"
27	Straighten out and define terminology for talking about headers
28	Client behavior with unrecognized code

Unless there's a problem with the updated text in -03, these issues will
be considered closed.

	Tony Hansen
	[email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  General-Address-Literal (was: rfc2821bis-02 Issue 24)
Date:  Tue, 24 Apr 2007 23:39:43 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 70
Sender: [email-masked]

John C Klensin wrote:

> ultimately, ABNF (and most metalanguages like it) cannot express
> semantics or conditional relationships in the syntax...

They might be unable to express *all* conditions, and there is a
border beyond that ABNF would be unreadable for humans, but it's
possible to find a god balance.  Just look into RFC 2822, it's
beautiful.  It took me years to find something that's (arguably)
wrong in the RFC 2822 ABNF.  

Of course some concepts like NO-WS-CTL in RFC 2822 are horrible,
but that's not the fault of the ABNF.  And everybody can see it,
asking why on earth there's NO-WS-CTL in the RFC 2821 construct
<General-Address-Literal>.  This means NO-WS-CTL in a HELO.  This
can't be what you really want, and no amount of prose can fix it,
it's a bug, made visible by the ABNF.

> I think that quietly restricting the range of the second digit
> in syntax with no comments in the text is a bad idea.

But it was also clear, somewhere I asked if what we want is in
fact 3DIGIT instead of [2-5][0-5][0-9].  You can use both forms,
and then either explain in the prose that not all digits are
allowed, or explain that extensions can specify other codes. 

Here's the issue mentioned above:

   address-literal  = "[" ( IPv4-address-literal /
                     IPv6-address-literal /
                     General-address-literal ) "]"

In other words some syntax gibberish within square brackets.

   General-address-literal  = Standardized-tag ":" 1*dcontent

Whatever that is, SMTP servers are expected to accept it.  Now
checking 2822:

   dcontent      =     dtext / quoted-pair

Great, we get a bunch of horrible backslashes here.  In the HELO
and everywhere.  But it's far worse:

   dtext         =     NO-WS-CTL /     ; Non white space controls
                       %d33-90 /       ; The rest of the US-ASCII
                       %d94-126        ;  characters not including "[",
                                       ;  "]", or "\"

This means raw ASCII 1-8, 11-12, 14-31, and 127.  In a HELO and
all other places allowing <adress-literal>.  Complete ESCape
sequences, NAK, SUB, DEL, etc.  This can't be what you want. 

And I would of course ask for a *detailed* "implementation and
interoperability report" wrt NO-WS-CTL if it's kept in 2821bis.

It took about ten months to *kill* this crap at least for the
Message-ID in RFC.ietf-usefor-usefor-12, I hope it will be a bit
faster here.  

After all who needs a <General-Address-Literal>, it's certainly 
not better than those 1yz codes.  If we "must" have it it should
be at least exactly the same as a STD 66 <IPvFuture>, preferably
using the same name.

STD 66 like RFC 2822 and RFC.ietf-usefor-usefor-12 has a very
clear no-nonsense ABNF. 

Frank

Date: Tue, 24 Apr 2007 16:39:33 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-02 Issue 24
Sender: [email-masked]

--On Tuesday, 24 April, 2007 15:15 -0400 Hector Santos
<[email-masked]> wrote:

> John C Klensin wrote:
> 
>> This is another example of why I'm getting cold feet about
>> doing a significant ABNF overhaul.  The advantage of
>> "Reply-line" over "reply" is that the former is easily found
>> by searching the text, while "reply" is used all over the
>...
>> either of which I think is more clear because they are less
>> subtle. But, probably fortunately, we are not.
> 
> Ha! True, true. :-)

> But poor developers, who have to look at all this and say:
> 
>           "HUH? What does that all mean? All I want is the
>            server response code!"
> 
> The bottom line translation from ALL of this and related
> issues is:
> 
>    - The server command response code is the first 3 digits of
> the
>      receiver buffer.  These 3 digits MUST be numeric, 0 thru
> 9.
> 
>    - The response is terminated when the there is no
> continuation
>      character "-" at byte #4

I'd say there is a different lesson in this, which is that, for
specifications of the lineage that includes 821 and 2821, the
formal syntax is a supplement to, not a substitute for, the
restrictions, discussion, and semantics in the text.  I would
suggest that is generally true for IETF specifications and for
specifications that use ABNF in particular: one can push the
line of what is specified/ restricted in syntax a bit in either
direction but, ultimately, ABNF (and most metalanguages like it)
cannot express semantics or conditional relationships in the
syntax... pretending otherwise doesn't get us very far although
it may lead to making the syntax sufficiently obscure in the
hope of being very precise that people's eyes glaze over.

> This is how the broken codes out there are reading it and some
> even consider this a high-quality, low overhead, optimized
> streaming parser!

I think that is a mistake.  And I think that putting ABNF
through an automated parser generator for any purpose other than
consistency-checking is a mistake too.  Just my opinion, of
course.

> So how is this written in ABNF style?
> 
>    Reply-line  = Terminating-Reply-Line | 1*
> Non-Terminating-Reply-Line
>    Terminating-Reply-Line     = Reply-Code [SP text] CRLF
>    Non-Terminating-Reply-Line = Reply-Code "-" [text] CRLF
>    Reply-code   = %x31-35 %x30-35 %x30-39

Unfortunately, this proves my point.  Note that 

(i) Your discussion above says "numeric, 0 through 9".   The
system you correctly copied from 2821 or 2821bis through -02
says "numeric, first digit must be 1 through 5, second digit 0
through 5, third digit 0 through 9".  And 2821bis-03 is just
like that except it says "first digit 2 though 5" ( %x32-35).

Worse, I think that is wrong because the flexibility that is now
in  2821bis --more restrictive than the flexibility that appears
in 2821, but close-- may be too much: while 821 -> 2821 don't
supply meanings for x6z, x7z, etc., the "go to the first digit
if you don't understand the whole code" rule is more than
sufficient to permit interoperability if an application needs,
e.g., 561.  Inventing 561 today would violate a "SHOULD" about
using extended codes rather than making up more codes in the
three-digit series but, unless someone can somehow demonstrate
that the are no uses of such codes out there (I actually know
some, but they are/were in SMTP-clones on non-TCP transports,
not in SMTP), I think that quietly restricting the range of the
second digit in syntax with no comments in the text is a bad
idea.

(ii) The syntax you have given might permit 
   250 terminating line
   250-non-terminating line 1
   250-non-terminating line 2
which is exactly backwards.  And it would definitely permit
   250-non-terminating line
   250-non-terminating line 1
   250-non-terminating line 2
with no terminating line at all.

No problem, except as an illustration that this is not as easy
to get right as we often assume.

> Quickly followed with a very strong normative text that says:
> 
>    The reply-code in each reply-line MUST be persistent (the
> same).

I hope that is clear now; please check -03 when it is posted and
suggest either text or reorganizations if you think they are
necessary after doing so.
 
     john

Date: Tue, 24 Apr 2007 15:15:33 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Philip Guenther <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-02 Issue 24
Sender: [email-masked]

John C Klensin wrote:

> This is another example of why I'm getting cold feet about doing
> a significant ABNF overhaul.  The advantage of "Reply-line" over
> "reply" is that the former is easily found by searching the
> text, while "reply" is used all over the place.   If we had an
> index to productions, possibly with numbered productions, I
> might feel differently about that, but we aren't going to get
> that for this version (too much work and incompatible with the
> tools).    I think that, if this were being done to my personal
> taste, we'd probably end up with
> 
>    Reply-sequence = *( Reply-continuation-line ) Reply-line
>    Reply-continuation-line =  Reply-code "-" [ text ] CRLF
>    Reply-line = Reply-code [ SP text ] CRLF
> 
> or even
> 
>    Reply-sequence = [ Reply-continuation-lines ] Reply-line
>    Reply-continuation-lines = 1*Reply-continuation-line
>    Reply-continuation-line =  Reply-code "-" [ text ] CRLF
>    Reply-line = Reply-code [ SP text ] CRLF
> 
> either of which I think is more clear because they are less
> subtle. But, probably fortunately, we are not.

Ha! True, true. :-)

But poor developers, who have to look at all this and say:

          "HUH? What does that all mean? All I want is the
           server response code!"

The bottom line translation from ALL of this and related issues is:

   - The server command response code is the first 3 digits of the
     receiver buffer.  These 3 digits MUST be numeric, 0 thru 9.

   - The response is terminated when the there is no continuation
     character "-" at byte #4

This is how the broken codes out there are reading it and some even 
consider this a high-quality, low overhead, optimized streaming parser!

So how is this written in ABNF style?

   Reply-line  = Terminating-Reply-Line | 1* Non-Terminating-Reply-Line
   Terminating-Reply-Line     = Reply-Code [SP text] CRLF
   Non-Terminating-Reply-Line = Reply-Code "-" [text] CRLF
   Reply-code   = %x31-35 %x30-35 %x30-39

Quickly followed with a very strong normative text that says:

   The reply-code in each reply-line MUST be persistent (the same).

I'm no ABNF guru, but the above make sense to me if I was reading it!

-- 
Hector Santos, CTO
Santronics Software, Inc
http://www.santronics.com

Date: Tue, 24 Apr 2007 14:13:36 -0400
From: John C Klensin <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-02 Issue 24 (was: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

(moving threads)

--On Monday, 23 April, 2007 17:05 -0600 Philip Guenther
<[email-masked]> wrote:

> On Mon, 23 Apr 2007, John C Klensin wrote:
> ...
>> Assuming that, by "the suggestion below", you meant
>> 
>>    Reply-line     = *( Reply-code "-" [ text ] CRLF ) /
>>                   Reply-code [ SP text ] CRLF
> 
> Uh, what's that "/" doing there?  Shouldn't it just read:
> 
>      Reply-line     = *( Reply-code "-" [ text ] CRLF )
>                     Reply-code [ SP text ] CRLF
> 
> ("zero or more continuation lines, followed by a
> non-continuation line")

The "/" is gone from the working draft.  Thanks to you and
others.
 
> I would also tend to just call that "Reply" instead of
> "Reply-line", given that it can be multiple lines.
> 
> (I'm not trying to start the ABNF bashing early, but better to
> avoid errors as close to when they occur...)

This is another example of why I'm getting cold feet about doing
a significant ABNF overhaul.  The advantage of "Reply-line" over
"reply" is that the former is easily found by searching the
text, while "reply" is used all over the place.   If we had an
index to productions, possibly with numbered productions, I
might feel differently about that, but we aren't going to get
that for this version (too much work and incompatible with the
tools).    I think that, if this were being done to my personal
taste, we'd probably end up with

   Reply-sequence = *( Reply-continuation-line ) Reply-line
   Reply-continuation-line =  Reply-code "-" [ text ] CRLF
   Reply-line = Reply-code [ SP text ] CRLF

or even

   Reply-sequence = [ Reply-continuation-lines ] Reply-line
   Reply-continuation-lines = 1*Reply-continuation-line
   Reply-continuation-line =  Reply-code "-" [ text ] CRLF
   Reply-line = Reply-code [ SP text ] CRLF

either of which I think is more clear because they are less
subtle. But, probably fortunately, we are not.

       john

From: [email-masked]
Cc: [email-masked], Frank Ellermann <[email-masked]>
Date: Tue, 24 Apr 2007 07:13:49 -0700 (PDT)
Subject: Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply  code
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Frank Ellermann writes:
> > John C Klensin wrote:
> >>  Can everyone live with that, more or less? If not, please suggest
> >>  alternate text and/or convince Tony that this particular dead horse
> >>  needs more kicking.
> >
> > Are you saying "remove the complete 1yz cruft and be done with it" ?
> > Sounds like a brilliant idea.

> +1 and I'll buy all concerned a beer at the very next opportunity.

+1 as well and TIA for the beer!

				Ned

Date: Tue, 24 Apr 2007 11:57:24 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Frank Ellermann <[email-masked]>
Subject: Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> Frank Ellermann writes:
>> John C Klensin wrote:
>>>  Can everyone live with that, more or less? If not, please suggest 
>>>  alternate text and/or convince Tony that this particular dead horse 
>>>  needs more kicking.
>>
>> Are you saying "remove the complete 1yz cruft and be done with it" ? 
>> Sounds like a brilliant idea.
> 
> +1 and I'll buy all concerned a beer at the very next opportunity.

+1

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 27: Received clauses
Date:  Tue, 24 Apr 2007 17:41:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Paul Overell wrote:

> If we wish to distinguish between <mailbox> and <Mailbox> then
> we ought to use different names, not just different case.

"Different case" is the state of the art in 2821 for this term
and <Atom>.  A different name would be nice, but what about the
existing references in other RFCs to <Mailbox> ?  That's why I
wrote "likely too late".

>| Rule names are case-insensitive

Sure, as seen for 2821bis <domain> and 2821bis <Domain>, not to
be confused with a 2822 <domain>.  It's not directly possible
to mix the RFC 2821 and 2822 ABNF.

For <Atom> it's very simple, we can just say 1*atext in the few
places where it's used.  Rewriting the 2821bis ABNF in a way
that allows to add the complete 2822 ABNF would be a pain, what
about <domain>, <mailbox>, <path>, <local-part>, and so on ?

And while the syntax differs the semantics is almost identical.

Frank

Date: Tue, 24 Apr 2007 11:32:58 -0400
From: John C Klensin <[email-masked]>
To: Paul Overell <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 27 (and 28): Received clauses
Sender: [email-masked]

--On Tuesday, 24 April, 2007 12:51 +0100 Paul Overell
<[email-masked]> wrote:

> In message <[email-masked]>, Frank Ellermann
> <[email-masked]> writes
>> 
>> Pete Resnick wrote:
>> 
>>> All of those have surrounding CFWS.
>> 
>> That's why 2821bis should stay away from using similar terms
>> like <Atom>, there are no comments and no line folding in the
>> envelope.  For <Mailbox> any attempt to clean it up is likely
>> too late, but it deserves a note in 2821bis that <Mailbox> is
>> syntactically very different from <mailbox>.
>> 
> 
> If we wish to distinguish between <mailbox> and <Mailbox> then
> we ought to use different names, not just different case.
>...

Sigh.
First, I agree with you in principle.

Second, this, and a few other notes in this thread, have caused
me to change my (personal) mind again about doing an ABNF
overhaul.  A week ago, I was taking the position that an ABNF
overhaul, however attractive, was just going to be too hard to
get right (a position very similar to the one Dave took in a
recent note).  Then I was convinced, yesterday, that it was
possible to do it and check it carefully enough to make the
effort worthwhile.

For whatever my personal opinion is worth, I've just changed my
mind again:

The grammatical element "Mailbox" is in upper-case in 2821 to
distinguish it from the informal term "mailbox", which is used
throughout the text.  The alternative would have been to use
"<mailbox>" and "mailbox", which is more or less what 821 did.
The choice of word and capitalization has nothing to do with
what was done in 2822 -- 2821 references 2822 only for terminal
and near-terminal productions and is, I hope, fairly clear about
where it does that.

Now, while it would be better if they didn't use similar-looking
terms for different productions, fixing that is not a matter of
fixing the ABNF -- it requires revisions to the text of the
specifications.  When I made the changes for "headers" that are
the topic of this issue, I discovered that it wasn't as easy as
I expected to make those corrections without ending up with some
extremely clumsy and hard-to-read sentences.  I think I've
gotten that right (or close enough), but my prediction is that
the odds of getting thing right if we change "Mailbox" into some
other term is that they are about zero.  

So I'm back to where we started, i.e., with a personal
conviction that we should fix things that are clearly bugs, take
care of low-lying fruit in the clarifications area, but
otherwise avoid any changes to things that are not broken and,
in particular, any changes that have sweeping implications to
the text.

Just my opinion, of course.

     john

Date: Tue, 24 Apr 2007 11:03:46 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: David F Skoll <[email-masked]>, [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

--On Monday, 23 April, 2007 11:00 -0700
[email-masked] wrote:

>> --On Monday, 23 April, 2007 09:03 -0700
>> [email-masked] wrote:
> 
>> >> John C Klensin wrote:
>> > 
>> >> > The strawman proposed text would be a provision that a
>> >> > client SHOULD (or perhaps even MUST) send a RSET or QUIT
>> >> > in response to a code whose first digit is undefined by
>> >> > either 2821bis or a negotiated extension.
>> > 
>> >> I would agree with that.  Something like:
>> > 
>> >> If an SMTP client receives a reply code that is not defined
>> >> by this RFC or a negotiated extension, it SHOULD send a
>> >> RSET or QUIT command, and SHOULD treat the unknown reply
>> >> code as a 5xx code.
>> > 
>> > I'm with you up to the 5yz code. I think 4yz would be more
>> > appropriate - that way you don't bounce messages
>> > unnecessarily when dealing with a broken server.
> 
>> Hmm.  Do you want to requeue it and keep trying for four or
>> five days on the theory that the server might spontaneously
>> fix itself?
> 
> Believe it or not, servers do spontaneously "fix" themselves
> on a regular basis.  One reason for this is that bugs tend to
> lurk in infrequently used code. Error handling code tends to
> be infrequently used, so when some temporary failure occurs
> (e.g., the directory server is rebooted and is unavailable for
> some period of time) the SMTP server finds itself using some
> buggy code that spits out incomprehensible garbage instead of
> a proper message. Then the temporary problem clears and the
> server no longer engages the buggy code, "fixing" the problem
> (until the next time, at least).
>...

Thanks.

I find this discussion (all of it, not just that quoted) very
persuasive, as well as educational.  See if the text I suggested
(before seeing this) is satisfactory or, if we need to go
further, please suggest something.

    john

Date: Tue, 24 Apr 2007 15:40:56 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 24: ABNF overhaul
Cc: John C Klensin <[email-masked]>, [email-masked], Dave Crocker <[email-masked]>
Sender: [email-masked]

Dave Crocker writes:
> Given that ABNF is, essentially, like writing code, any massive 
> overhaul to it would be certain to a) take a long time, and b) 
> introduce new errors.  This is not the stuff of a bis or ter effort.

I would say b) turn old lack of clarity into new errors.

> I suggest that new SMTP ABNF would make a nice, independent project 
> for someone.  It would take the task out of the current critical path 
> and would permit the ABNF to be subject to extended and independent 
> analysis.

I have to disagree with this. The SMTP ABNF is not practically separable 
from the SMTP RFC, and attempting to separate the two will lead to 
hopeless squabbles in the years to come.

(FYI, I've volunteered.)

> But what I've described would take years to happen.

We'll see.

Arnt

Date: Tue, 24 Apr 2007 05:29:26 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-02 Issue 24: ABNF overhaul
Sender: [email-masked]

John C Klensin wrote:
> Pro: Clearly, a complete ABNF cleanup, to make the set closed
> and consistent and, ideally, to eliminate and formalize as many
> of the comments as possible would be a nice thing.

I'd class it as considerably more than "nice".  The current document 
uses ABNF at a level I would call "minimal" and I have occasionally 
found some questions a challenge to resolve because of this.

That said...

> However,
> Con: The odds of being able to do this in a reasonable amount of
> time and to do so without introducing new errors --either within

+1.

Given that ABNF is, essentially, like writing code, any massive overhaul 
to it would be certain to a) take a long time, and b) introduce new 
errors.  This is not the stuff of a bis or ter effort.

I suggest that new SMTP ABNF would make a nice, independent project for 
someone.  It would take the task out of the current critical path and 
would permit the ABNF to be subject to extended and independent analysis.

If a community consensus emerges that the new ABNF is preferable to what 
is already in the official SMTP, then it would be worth considering 
re-issuing the document with those changes folded in.  But what I've 
described would take years to happen.

d/bv
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 24 Apr 2007 05:20:22 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Sender: [email-masked]

Paul Overell wrote:
> If we wish to distinguish between <mailbox> and <Mailbox> then we ought 
> to use different names, not just different case.  From RFC4234 ABNF:
>>
>> 2.1.  Rule Naming
>>   NOTE:
>>
>>      Rule names are case-insensitive

+1

/d
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 24 Apr 2007 12:51:49 +0100
To: [email-masked]
From: Paul Overell <[email-masked]>
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Sender: [email-masked]

This is a PGP signed message sent according to RFC3156 [PGP/MIME]

--=_Turnpike_27HwUhNU9eLG5L8T=
Content-Type: text/plain;charset=us-ascii;format=flowed
Content-Transfer-Encoding: quoted-printable

In message <[email-masked]>, Frank Ellermann=20
<[email-masked]> writes
>
>Pete Resnick wrote:
>
>> All of those have surrounding CFWS.
>
>That's why 2821bis should stay away from using similar terms
>like <Atom>, there are no comments and no line folding in the
>envelope.  For <Mailbox> any attempt to clean it up is likely
>too late, but it deserves a note in 2821bis that <Mailbox> is
>syntactically very different from <mailbox>.
>

If we wish to distinguish between <mailbox> and <Mailbox> then we ought=20
to use different names, not just different case.  From RFC4234 ABNF:

>
>2.1.  Rule Naming
>

>
>   NOTE:
>
>      Rule names are case-insensitive
>

Regards

--=20
Paul Overell         Internet Platform Development Manager, Thus plc

--=_Turnpike_27HwUhNU9eLG5L8T=
Content-Type: application/pgp-signature
Content-Disposition: attachment; filename=signature.asc

--=_Turnpike_27HwUhNU9eLG5L8T=--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 27: Received clauses
Date:  Tue, 24 Apr 2007 13:11:31 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Pete Resnick wrote:
 
> All of those have surrounding CFWS.

That's why 2821bis should stay away from using similar terms 
like <Atom>, there are no comments and no line folding in the
envelope.  For <Mailbox> any attempt to clean it up is likely
too late, but it deserves a note in 2821bis that <Mailbox> is
syntactically very different from <mailbox>.

Frank

Date: Tue, 24 Apr 2007 12:58:53 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

Frank Ellermann writes:
> John C Klensin wrote:
>>  Can everyone live with that, more or less? If not, please suggest 
>>  alternate text and/or convince Tony that this particular dead horse 
>>  needs more kicking.
>
> Are you saying "remove the complete 1yz cruft and be done with it" ? 
> Sounds like a brilliant idea.

+1 and I'll buy all concerned a beer at the very next opportunity.

Arnt

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date:  Tue, 24 Apr 2007 12:54:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Philip Guenther wrote:

> Shouldn't it just read:

>      Reply-line     = *( Reply-code "-" [ text ] CRLF )
>                     Reply-code [ SP text ] CRLF

Yes.  We need the "/" for an alternative for the <Greeting>
but not here.  And maybe we want 3DIGIT for the Reply-Code
with the prose saying that only 2yz up to 5yz are used.

The opposite approach, the ABNF says [2-5]yz, and the prose
says "unless an extension allows e.g. 1yz", is also okay.

The folded <Greeting> ABNF is unfortunately gibberish:

   Greeting       = ( "220 " (Domain / address-literal) [ SP text ] CRLF
                  ) /
                  ( "220-" (Domain / address-literal) [ SP text ] CRLF
                  *( "220-" [ text ] CRLF )
                  "220" [ SP text ] CRLF )

Based on the <Reply-text> idea I get:

   Greeting       =  "220 " (Domain / address-literal)
                     [ SP Reply-text ] CRLF

   Greeting       =/ "220-" (Domain / address-literal)
                     [ SP Reply-text ] CRLF
                    *( "220-" [ Reply-text ] CRLF )
                     "220" [ SP Reply-text ] CRLF

The alignment of columns is significant in this <artwork> :-)
Unchecked, the ABNF parser is apparently down at the moment.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code
Date:  Tue, 24 Apr 2007 12:30:23 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

John C Klensin wrote:

> Can everyone live with that, more or less?  If not, please
> suggest alternate text and/or convince Tony that this particular
> dead horse needs more kicking.

Are you saying "remove the complete 1yz cruft and be done with 
it" ?  Sounds like a brilliant idea.  Somewhere you'd have to
say that extensions are free to invent their own reply codes
starting with other digits (including "1"), to be used if and
only if the corresponding extension is used by mutual agreement.

Frank

Date: Tue, 24 Apr 2007 01:39:07 -0500
To: Kari Hurtta <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Cc: [email-masked]
Sender: [email-masked]

On 4/24/07 at 9:10 AM +0300, Kari Hurtta wrote:

>Pete Resnick <[email-masked]> writes:
>
>>       received = "Received:" *received-tokens ";" date-time CRLF
>>
>>       received-tokens = word / angle-addr / addr-spec / domain
>
>You mean
>
>       received = "Received:" *( [CVFS] received-tokens ) ";" date-time CRLF
>
>       received-tokens = word / angle-addr / addr-spec / domain
>
>perhaps?
>
>(at least <word> does not include [CVFS] )

Yes it does.

    word            =       atom / quoted-string

    atom            =       [CFWS] 1*atext [CFWS]

    quoted-string   =       [CFWS]
                          DQUOTE *([FWS] qcontent) [FWS] DQUOTE
                          [CFWS]

All of those have surrounding CFWS.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
From: Kari Hurtta <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 27: Received clauses
Date:  24 Apr 2007 09:10:07 +0300
Organization:  Ilmatieteen laitos (FMI)
Lines: 36
Cache-Post-Path: [email-masked]
Sender: [email-masked]

( sorry about duplicate, I made error on headers)

Pete Resnick <[email-masked]> writes:

> On 4/23/07 at 9:15 PM -0400, Bruce Lilly wrote:
> 
> > OK -- I suggest using an ABNF rule name for the construct of
> > "clause" (rather than -- or in addition to -- 2822 "name-val-pair")
> > to make the association between RFC text and ABNF clearer.
> >[...]
> >Also, please be very careful: a literal
> >   clause = *(word / domain / addr-spec / angle-addr)
> >would allow currently illegal atrocities
> 
> So, as with other stuff, 2822upd is going to have a more general
> syntax that it will be up to 2821bis to limit. In this case, I'm
> simply going to have:
> 
>      received = "Received:" *received-tokens ";" date-time CRLF
> 
>      received-tokens = word / angle-addr / addr-spec / domain
> 
> and leave it to 2821bis to use syntax (including defining proper
> "clauses") that won't allow such "atrocities".

You mean

      received = "Received:" *( [CVFS] received-tokens ) ";" date-time CRLF

      received-tokens = word / angle-addr / addr-spec / domain

perhaps?

(at least <word> does not include [CVFS] )

/ Kari Hurtta

Date: Mon, 23 Apr 2007 22:50:06 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Sender: [email-masked]

I'm happy with this, as long as it's very clear in 2822upd that the
developer writer must refer to *821 to find the true definition for what
their code should generate. I also have no problem with it being a
normative reference.

	Tony Hansen
	[email-masked]

Pete Resnick wrote:
> 
> On 4/23/07 at 9:15 PM -0400, Bruce Lilly wrote:
> ...
>> 2. 2821 specifies the "for" clause name as having a value of
>>    1 * ( Path / Mailbox ), whereas 2822 permits only a single
>>    addr-spec or one or more angle-addrs
> 
> Allowed above by any number of "angle-addr" or "addr-spec". 2821bis can
> limit as desired.
> 
>> Although it may be tempting to specify a generic field in 2822bis and
>> punt the specifics to 2821bis, [...] it may be (depending on how loose
>> the 2822bis specification is) impossible in general to parse such
>> fields. 2822 as written hasn't quite fallen over that cliff, but it's
>> teetering on the edge:
>>   Received:;1 Jan 2007 02:34:56 -0700
> 
> I would claim that 2822 is already well over that cliff and the
> suggested change above really just finishes the task. I'd prefer to keep
> the lack of normative reference from 2822 to 2821, and making it generic
> does that.
> 
> pr

Date: Mon, 23 Apr 2007 21:39:19 -0500
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Cc: Bruce Lilly <[email-masked]>
Sender: [email-masked]

On 4/23/07 at 9:15 PM -0400, Bruce Lilly wrote:

>OK -- I suggest using an ABNF rule name for the construct of 
>"clause" (rather than -- or in addition to -- 2822 "name-val-pair") 
>to make the association between RFC text and ABNF clearer.
>[...]
>Also, please be very careful: a literal
>   clause = *(word / domain / addr-spec / angle-addr)
>would allow currently illegal atrocities

So, as with other stuff, 2822upd is going to have a more general 
syntax that it will be up to 2821bis to limit. In this case, I'm 
simply going to have:

     received = "Received:" *received-tokens ";" date-time CRLF

     received-tokens = word / angle-addr / addr-spec / domain

and leave it to 2821bis to use syntax (including defining proper 
"clauses") that won't allow such "atrocities".

>Please don't forget the existing (2821/2822) discrepancies:
>1. 2821 permits "String" which in turn includes "Quoted-string", while
>    2822 does not allow "quoted-string" in a value-item [*]

I've got "word" which is "atom / quoted-string".

>2. 2821 specifies the "for" clause name as having a value of
>    1 * ( Path / Mailbox ), whereas 2822 permits only a single
>    addr-spec or one or more angle-addrs

Allowed above by any number of "angle-addr" or "addr-spec". 2821bis 
can limit as desired.

>Although it may be tempting to specify a generic field in 2822bis 
>and punt the specifics to 2821bis, [...] it may be (depending on how 
>loose the 2822bis specification is) impossible in general to parse 
>such fields. 2822 as written hasn't quite fallen over that cliff, 
>but it's teetering on the edge:
>   Received:;1 Jan 2007 02:34:56 -0700

I would claim that 2822 is already well over that cliff and the 
suggested change above really just finishes the task. I'd prefer to 
keep the lack of normative reference from 2822 to 2821, and making it 
generic does that.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 23 Apr 2007 22:32:46 -0400
From: Tony Hansen <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code
Sender: [email-masked]

Thank you John. This suggestion is minimalistic, yet pithy. I'd much
rather we went this route than the earlier suggestions in this thread. I
think those were starting to go too far towards invention.

Here's to avoiding rat holes.

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> Hi.
> 
> After thinking about the several comments on this, together with
> other comments on the list, I want to propose threading the
> needle with the following text, tentatively placed at the end of
> the introduction to section 4.2:
> 
> 		In the absence of extensions negotiated with the client,
> 		SMTP servers MUST NOT send reply codes whose first
> 		digits are other than 2, 3, 4, or 5.   Clients that
> 		receive such out-of-range codes SHOULD normally treat
> 		them as fatal errors and terminate the mail transaction.
> 
> Note that 
> (1) This makes it incumbent on the server to not send junk.
> That is where the real responsibility lies, IMO.
> 
> (2) It focuses on the first digit only.  I believe that the
> existing text is adequately clear that, if one doesn't
> understand the second or third digits, one should fall back on
> the first.
> 
> (3) It avoids dealing with the question of what an "undefined"
> code is entirely, dodging another rathole.
> 
> (4) If I've done it correctly, the final sentence dodges the
> issue of whether to treat the unrecognized code as 4yz or 5yz.
> We affirm the fact that this is a bad situation, suggest that
> clients should be prepared for it, and use a SHOULD to indicate
> "fatal... terminate".    From the standpoint of the standard,
> such codes are really sufficiently out of scope that imposing
> requirements on the client as to whether, e.g., to treat the
> error as temporary or permanent seems like overkill.
> 
> Can everyone live with that, more or less?  If not, please
> suggest alternate text and/or convince Tony that this particular
> dead horse needs more kicking.
> 
>      john
> 

Date: Mon, 23 Apr 2007 21:50:24 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

At 13:32 -0400 on 04/23/2007, David F. Skoll wrote about Re: New 
issue (was Re: rfc2821bis-01 Issue 18: Usability of:

>However, it's a SHOULD, so a client could treat it as a 4yz if it
>really wants.  Actually, the more I think about it, the more it might
>make sense to treat it as a 4yz if there are any lower-priority MX
>hosts left to try, and a 5yz if not.  But putting that in an RFC is
>kind of ugly; I wouldn't know how to word it nicely.

Why not just say that. IOW: Something on the order of:

If there is a an undefined response code, the client SHOULD terminate 
the submission by treating it as a 5yz code. In the event that there 
are any lower-priority MX hosts left to try, the client MAY instead 
treat it as a 4yz and try the lower-priority MX hosts until the 
message is accepted by one of the MX hosts or all have been used at 
which point the final undefined response code SHOULD be treated as a 
5yz.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 27: Received clauses
Date: Mon, 23 Apr 2007 21:15:33 -0400
Cc: Pete Resnick <[email-masked]>
Sender: [email-masked]

On 2007-04-23, Pete Resnick wrote:

> And here I was thinking that I was going to get to duck all of this.

It may be worse than you think...

> Some confusion in terminology (and some broken syntax in 2822) led to 
> a short discussion resulting in the following:
> 
> - 2821bis and 2822upd will both define "Header Section" as the 
> complete set of header fields at the beginning of a message.

"Message Header Section" might also avoid the ambiguity with the set(s)
of header fields which occur in composite MIME messages (RFC 2046
"MIME-part-headers").

> - 2821bis and 2822upd will both define "Header Field" as a line in a 
> message with a field name followed by a colon followed by a field 
> body.
> 
> - 2821bis and 2822upd will both avoid using the terms "header" and 
> "headers" in isolation to avoid ambiguity.

Minor nit above notwithstanding, this is an improvement in consistency
(e.g with FYI18) and will help reduce ambiguity.  Thanks.
 
> - 2821bis will use the term "received clause" to refer to a keyword 
> and the set of stuff that follows it before the next keyword (so, 
> "for [email-masked]" is a received clause).

OK -- I suggest using an ABNF rule name for the construct of "clause"
(rather than -- or in addition to -- 2822 "name-val-pair") to make the
association between RFC text and ABNF clearer.

> - 2822upd will simplify the syntax of received clauses (what is now 
> "name-val-list") such that it's simply "*(word / domain / addr-spec / 
> angle-addr)" and let 2821bis do the real definition.

Please don't forget the existing (2821/2822) discrepancies:
1. 2821 permits "String" which in turn includes "Quoted-string", while
   2822 does not allow "quoted-string" in a value-item [*]
2. 2821 specifies the "for" clause name as having a value of
   1 * ( Path / Mailbox ), whereas 2822 permits only a single
   addr-spec or one or more angle-addrs [technically, 2822 also permits
   an atom, a domain, or a msg-id] (821 permitted a single "path" and
   822 a single addr-spec); parsing multiple values attached to a name
   is quite difficult to do correctly and efficiently, and is
   unnecessary if multiple "for" clauses are permitted [822 permitted
   multiple "with" clauses, but not "for"; 821 permitted/required one
   instance each of "from" and "by", and at most one of "via", "with",
   "id", and "for"; 2822 permits any number of any named clauses;
   2821 requires single "from" and "by" (with different and non-*822
   syntax from 821) and at most one each of "with", "via", "id", and
   "for" clauses (the latter two also having different syntax from
   821 and *822)] [+] [&]
3. 2822 does not specify order of clauses; 822 and *821 specify order
   as "from" "by" "via" "with" id" "for".
4. 2822 does not associate a value type with specific names, and
   therefore permits atrocities such as
      Received: from <[email-masked]> by <[email-masked]> via domain.net
        with [1.2.3.4] id ESMTP for UUCP ; 1 Jan 2007 01:23:45 -0600
   whereas 822 and *821 specify (inconsistently, unfortunately) the
   specific value types for each specified permitted name
5. 2821's structured "TCP-Info" is syntactically equivalent to an *822
   comment ["ignored by the formal semantics" [822]/"semantically
   interpreted as a single space character" [2822]]

Also, please be very careful: a literal
  clause = *(word / domain / addr-spec / angle-addr)
would allow currently illegal atrocities as valid "clause"s such as
"with Microsoft SMTPSVC" [an error actually produced by certain
non-*82{1,2}-compliant software] and "from by with for id via";
both examples would be quite difficult to parse.  Sticking with the
historical and present specification of a name-value pair [ideally
with specified value type for each (registered!) name ] would be
preferable to introduction of new opportunities for interoperability
problems. [+] Retaining the requirement for a name-value pair could
be done as simply as:
  clause = name-value-pair CFWS
with name-val-pair and the implicit item-name and item-value much as
in 2822 [again, ideally with complete specification of value type as
in 822].

Although it may be tempting to specify a generic
field in 2822bis and punt the specifics to 2821bis, failing a MUST/
MUST NOT specification that states in effect that *only* SMTP receivers
are permitted to insert Received fields [#], that leaves the possibility
that some non-SMTP agent (and therefore not subject to 2821bis'
specification) will insert such a field having syntax incompatible
with the 2821bis specification; as a mail user agent or other agent
which may have to parse such fields is unable to determine whether a
given instance was inserted by an SMTP receiver (and subject to 2821bis
syntax) or by some other agent (with only a generic 2822bis
specification), it may be (depending on how loose the 2822bis
specification is) impossible in general to parse such fields.  2822 as
written hasn't quite fallen over that cliff, but it's teetering on the
edge:
  Received:;1 Jan 2007 02:34:56 -0700
is a legal 2822 Received field (with no substantive trace information).
And the "from by with for id via" atrocity above *is* legal 2822 if
parsed as three name-val-pairs ("by", "for" and "via" being legal
2822 "atom"s associated as values with "from", "with", and "id" names).
[The careful reader may note that that illustrates the precise problem
with non-paired "clause" constructs as well as with permitting multiple
values with a single name -- one cannot tell which tokens to group
together.]

Another issue worth mentioning w.r.t. Received fields is the special
treatment required by RFC 2047, viz. comments in Received fields are
forbidden from incorporating encoded-words (for no documented reason),
which are perfectly legal in all other structured fields which permit
comments. Therefore:
  Date: 1 Jan 2007 02:34:56 +0100
   (=?iso-8859-1*de?q?Mittel_Europ%E4ische_Zeit?=)
is perfectly legal (and somewhat informative), whereas
  Received:;1 Jan 2007 02:34:56 +0100
   (=?iso-8859-1*de?q?Mittel_Europ%E4ische_Zeit?=)
is quasi-legal (if and only if one considers the comment as not
containing an encoded-word, in which case it is uninformative gibberish
[more so had I used B encoding in the example rather than (for legibility)
Q encoding])

* although ABNF rule names are de jure case-insensitive, another potential
  source of confusion and ambiguity can be removed by making references to
  common ABNF rule names case-consistent between 2821bis and 2822bis, e.g.
  all lower-case "quoted-string"

+ given the historical and continued inconsistencies with the specified
  syntax of Received fields, as well as [clearly broken, though widely
  deployed] implementations which fail to conform to *any* of those
  specifications, and the unsuitability of the fields for the purpose of
  *reliably* tracing messages [$], it might be worthwhile to define a
  replacement successor to the Received field (which itself replaced RFC
  788's Mail-From field) in order to provide a clean slate

# I hope that the Security Considerations sections of both documents will
  acknowledge the fact that trace fields by their nature preclude
  end-to-end message (including message header section) integrity/privacy
  mechanisms. I.e. by modifying the message in transit (specifically by
  inserting new message header fields), any message integrity checksum
  over the entire message is rendered invalid.  It would be nice to do
  away with that problem, but doing so would require substantive changes
  to both specifications.

$ viz. the trivially-forged and therefore unreliable HELO/EHLO name is
  recorded in the machine-readable "from" clause, whereas the reliable
  client IP address is (optionally!) relegated to a "semantically
  interpreted as a single space character" [quoting 2822] comment.
  This is a security-related issue with the *821 specification of
  Received fields (forgery may cause blame (for SPAM, copyright
  infringement, so-called "hate crimes", etc.) to be wrongly attributed).
  Note that RFC 788 did not specify use of the easily-forged HELO name
  in its Mail-From field.

& both documents' Security Considerations sections should note that
  "for" clauses are likely to compromise privacy and may compromise
  security by revealing information which should be confined to non-
  message "envelope" (i.e. [E]SMTP command parameters) data (the
  poster child case is Bcc recipient data revealed via [possibly
  multiple] "for" value angle-addrs)

From: [email-masked]
Cc: [email-masked], David F Skoll <[email-masked]>, [email-masked]
Date: Mon, 23 Apr 2007 11:00:53 -0700 (PDT)
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Monday, 23 April, 2007 09:03 -0700
> [email-masked] wrote:

> >> John C Klensin wrote:
> >
> >> > The strawman proposed text would be a provision that a
> >> > client SHOULD (or perhaps even MUST) send a RSET or QUIT in
> >> > response to a code whose first digit is undefined by either
> >> > 2821bis or a negotiated extension.
> >
> >> I would agree with that.  Something like:
> >
> >> If an SMTP client receives a reply code that is not defined
> >> by this RFC or a negotiated extension, it SHOULD send a RSET
> >> or QUIT command, and SHOULD treat the unknown reply code as a
> >> 5xx code.
> >
> > I'm with you up to the 5yz code. I think 4yz would be more
> > appropriate - that way you don't bounce messages unnecessarily
> > when dealing with a broken server.

> Hmm.  Do you want to requeue it and keep trying for four or five
> days on the theory that the server might spontaneously fix
> itself?

Believe it or not, servers do spontaneously "fix" themselves on a regular
basis.  One reason for this is that bugs tend to lurk in infrequently used
code. Error handling code tends to be infrequently used, so when some temporary
failure occurs (e.g., the directory server is rebooted and is unavailable for
some period of time) the SMTP server finds itself using some buggy code that
spits out incomprehensible garbage instead of a proper message. Then the
temporary problem clears and the server no longer engages the buggy code,
"fixing" the problem (until the next time, at least).

Of course there are also permanent server failures. But these actually tend to
be rarer since a server that pukes out incomprehensible garbage on a regular
basis tends to be found and fixed because mail isn't getting through.

> Or would it be better to bounce the puppy and hope that
> someone will complain? I think I can argue that one either
> way.

You certainly can argue that in such cases it would be better to bounce and in
the process report the problem, but the issue with that is you're reporting the
problem to the end user, who is far more likely to  "blame the messenger" for
the problem than they are to ferret out the actual culprit. And it is hardly
surprising that they do this: How many users understand the structure of email
to the point where they can parse a DSN and determine where the problem was?
(The absolute crap that often appears in the "human readable" part of some DSNs
doesn't help matters any...)

So now you have an end user who likely assumes that the problem is with their
mail service. Most of them won't bother to report the problem, which means that
all implementing a bounce policy has done is tarnish your own reputation with
this user and get nothing fixed in the process. And then there's the occasional
user who will actually report the problem - to their service provider, who
these days is unlikely to have either the desire or the time to get in touch
with the admin of the broken machine and work out the problem. So that doesn't
help much either.

Compare this with what happens when the message is retained and retried. For
one thing, if there's another MX trying it immediately may get the message
delivered. Alternately, if the problem goes away fairly soon the message will
make it through, maybe delayed a bit but not so much that a complaint is
likely.

If the outage lasts for any length of time the user will typically start
getting delay DSNs. But the psychology of a delay DSN is different: It
basically says that we're working to deliver your mail but there's a problem.
The indication of effort, as opposed to "we gave up", especially when it
happens fairly quickly, can sometimes be enough to change a negative user
experience into a positive one.

Even more important, decent service providers monitor their queues fairly
carefully. They do NOT monitor failed DSNs in a similar fashion. And while they
aren't going to care about two messages stuck trying to be sent to the ISP in
Greater Tuna (down no doubt because Dixie Deebury cut off their power ;-), if
otherbigisp.com that receives significant traffic is down they'll check into
and figure out why thousands of messages to otherbigisp.com are stuck in their
queues.

				Ned

Date: Mon, 23 Apr 2007 19:25:17 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 30: Deprecate use of source routes completely and remove text
Sender: [email-masked]

Hi.

This has been discussed, but just to give it a number and insure
that it doesn't fall through the cracks...

2821 sort of danced around the issue of source routes, going a
little further than 1123 did in discouraging their use but not
getting rid of them entirely.  The method used to do this in
2821 is a little fuzzy, since it left the text in place
instructing relays to copy their own domains into the
backward-pointing address.  That was discussed as issue 26 and
the working draft for -03 now has that issue straightened out
(assuming I've gotten it right).

However, that discussion raised another possibility, which is
that now is the time to deprecate source routes entirely.  More
precisely that would:

	(i) Leave the syntax in place and continue to require
	that systems be able to accept and parse it.
	
	(ii) Forbid systems from using source routes, either
	changing the "they should be ignored but may be used"
	language that now appears in Section 3.3 (and more or
	less repeated in 3.6) to forbidding their use for
	routing or leaving those two sections alone (I consider
	eliminating the redundancy to be an editorial matter if
	I can figure out how to do it).
	
	(iii) Continue with (ii) by changing the "SHOULD not
	generate" in the second paragraph of 4.1.1.3 to MUST NOT.
	
	(iv) Get rid of Appendix C, replacing it with some
	handwaving to the effect that, if people really need to
	know how this obsolete feature worked, they should read
	821.
	
	(v) Strengthen the prohibitions in Appendix F.2.

So far, we've heard from a couple of people on-list who feel
that this is a much better idea than the patching contemplated
by issue 26 and no one who has argued for keeping them.  Whether
it is realistic to further deprecate them or not probably
depends on interoperability testing although we could arguably
prohibit the things as a bad idea that is not widely supported.

     john

Date: Mon, 23 Apr 2007 17:05:20 -0600
From: Philip Guenther <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

On Mon, 23 Apr 2007, John C Klensin wrote:
...
> Assuming that, by "the suggestion below", you meant
>
>    Reply-line     = *( Reply-code "-" [ text ] CRLF ) /
>                   Reply-code [ SP text ] CRLF

Uh, what's that "/" doing there?  Shouldn't it just read:

     Reply-line     = *( Reply-code "-" [ text ] CRLF )
                    Reply-code [ SP text ] CRLF

("zero or more continuation lines, followed by a non-continuation line")

I would also tend to just call that "Reply" instead of "Reply-line", given 
that it can be multiple lines.

(I'm not trying to start the ABNF bashing early, but better to avoid 
errors as close to when they occur...)

Philip Guenther

Date: Mon, 23 Apr 2007 18:59:41 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 28: Client behavior on unrecognized reply code (was: Re: New issue...)
Sender: [email-masked]

Hi.

After thinking about the several comments on this, together with
other comments on the list, I want to propose threading the
needle with the following text, tentatively placed at the end of
the introduction to section 4.2:

		In the absence of extensions negotiated with the client,
		SMTP servers MUST NOT send reply codes whose first
		digits are other than 2, 3, 4, or 5.   Clients that
		receive such out-of-range codes SHOULD normally treat
		them as fatal errors and terminate the mail transaction.

Note that 
(1) This makes it incumbent on the server to not send junk.
That is where the real responsibility lies, IMO.

(2) It focuses on the first digit only.  I believe that the
existing text is adequately clear that, if one doesn't
understand the second or third digits, one should fall back on
the first.

(3) It avoids dealing with the question of what an "undefined"
code is entirely, dodging another rathole.

(4) If I've done it correctly, the final sentence dodges the
issue of whether to treat the unrecognized code as 4yz or 5yz.
We affirm the fact that this is a bad situation, suggest that
clients should be prepared for it, and use a SHOULD to indicate
"fatal... terminate".    From the standpoint of the standard,
such codes are really sufficiently out of scope that imposing
requirements on the client as to whether, e.g., to treat the
error as temporary or permanent seems like overkill.

Can everyone live with that, more or less?  If not, please
suggest alternate text and/or convince Tony that this particular
dead horse needs more kicking.

     john

Date: Mon, 23 Apr 2007 18:56:42 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> The ABNF says:
> 
>    Reply-line     = Reply-code [ SP text ] CRLF
> 
> it does not say:
> 
>    Reply-line     = Reply-code [ SP | "-" text ] CRLF
> 
> nor does it say:
> 
>    Reply-line     = Reply-code

The RFC says:

   Formally, a reply is defined to be the sequence: a
   three-digit code, <SP>, one line of text, and <CRLF>, or a multiline
   reply (as defined in section 4.2.1).

The ABNF you are quoting describes only a non-multiline reply.  Multiline
replies are defined in section 4.2.1.

--
David.

Date: Mon, 23 Apr 2007 18:40:35 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Monday, 23 April, 2007 17:59 -0400 Tony Hansen
<[email-masked]> wrote:

> 
> <process hat on>
> 
> Can we please stop this? You are both right. The spec is
> inconsistent, and depending on how you interpret it, you get
> different answers. No, the code is not broken. Yes, the code
> is broken.
> 
> Our job here is make the spec better. Let's stop blaming
> people, their code or how they interpreted the spec. Casting
> aspersions helps no one. Instead, let's put the blame where it
> truly lies: on shortcomings in the spec. And then let's work
> on fixing that spec to eliminate the inconsistencies.
> 
> </process hat off>
> 
> <spec writer hat on>
> 
> Section 4.2 clearly says that a server response may be single
> line or multiline, and all commands are expected to be able to
> receive either one. It then goes on to define (in the text and
> ABNF) the format of the single line response, and punts to
> section 4.2.1 for the definition of the multiline format.
> Unfortunately, section 4.2.1 only uses a textual description
> for the multiline response and does NOT give the corresponding
> ABNF to match.
> 
> There are several ways of remedying this. The suggestion below
> to augment the ABNF in section 4.2 is a perfectly reasonable
> way to do it.
> 
> </spec writer hat off>

<befuddled editor hat on>
Assuming that, by "the suggestion below", you meant

    Reply-line     = *( Reply-code "-" [ text ] CRLF ) /
                   Reply-code [ SP text ] CRLF
    Reply-code     = %x31-35 %x30-35 %x30-39

that change is consistent with what is already in rfc2821bis-02.
I have a note in my file that says that Frank, and possibly
others, have noted that "text" is not completely defined in
2821, but I assume that will be straightened out in the ABNF
cleanup pass.

If that is not what you intend, please say so.
</befuddled editor hat off>

   john

Date: Mon, 23 Apr 2007 17:59:19 -0400
From: Tony Hansen <[email-masked]>
To: Hector Santos <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

<process hat on>

Can we please stop this? You are both right. The spec is inconsistent,
and depending on how you interpret it, you get different answers. No,
the code is not broken. Yes, the code is broken.

Our job here is make the spec better. Let's stop blaming people, their
code or how they interpreted the spec. Casting aspersions helps no one.
Instead, let's put the blame where it truly lies: on shortcomings in the
spec. And then let's work on fixing that spec to eliminate the
inconsistencies.

</process hat off>

<spec writer hat on>

Section 4.2 clearly says that a server response may be single line or
multiline, and all commands are expected to be able to receive either
one. It then goes on to define (in the text and ABNF) the format of the
single line response, and punts to section 4.2.1 for the definition of
the multiline format. Unfortunately, section 4.2.1 only uses a textual
description for the multiline response and does NOT give the
corresponding ABNF to match.

There are several ways of remedying this. The suggestion below to
augment the ABNF in section 4.2 is a perfectly reasonable way to do it.

</spec writer hat off>

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> 
> David F. Skoll wrote:
>> Hector Santos wrote:
> 
>>> You are correct.  Exim is broken!! <g>  but I guess will correct that to
>>> make it unbroken by changing the specification.
>>
>> Why do you contend that it is broken?  Seriously, if you're going to
>> claim that an implementation is broken, you have to point to something in
>> the RFC that the implementation violates (a MUST or a MUST NOT).
> 
> The ABNF says:
> 
>    Reply-line     = Reply-code [ SP text ] CRLF
> 
> it does not say:
> 
>    Reply-line     = Reply-code [ SP | "-" text ] CRLF
> 
> nor does it say:
> 
>    Reply-line     = Reply-code
> 
> Now of course, if you are going kludge an endorsement by changing the
> standard to:
> 
>    Reply-line     = *( Reply-code "-" [ text ] CRLF ) /
>                   Reply-code [ SP text ] CRLF
>    Reply-code     = %x31-35 %x30-35 %x30-39
> 
> in order to make BROKEN behavior officially compliant in 2821bis, then
> hey, more power to you.
> 
>> Unless you can point to something that says an implementation MUST NOT
>> use the reply code in the first line of a multiline reply, 
> 
> see above
> 
>> you should
>> stop claiming that Exim or Qmail are broken because they use the first
>> line.
> 
> I understand the truth something does hurt. Its BROKEN CODE - PERIOD!
> 
> -- 
> HLS
> 
> 

Date: Mon, 23 Apr 2007 17:51:52 -0400
From: John C Klensin <[email-masked]>
To: Philip Guenther <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

--On Monday, 23 April, 2007 14:12 -0600 Philip Guenther
<[email-masked]> wrote:

> 
> On Mon, 23 Apr 2007, Hector Santos wrote:
> ...
>> This 4.3.2 sentence makes it clear that "1yz " is not valid,
>> 
>>   Unless extended using the mechanisms described in Section
>>   2.2, SMTP servers MUST NOT transmit reply codes to an SMTP
>>   client that are other than three digits or that do not
>>   start in a digit between 2 and 5 inclusive.
>> 
>> But it is also inconsistent with the 4.2 sentence:
>> 
>>   An SMTP server SHOULD send only the reply codes listed in
>>   this document.
> 
> Those two statements are not actually inconsistent.  The
> first, from 4.3.2, only restricts what the server can use for
> the first digit of a reply code.  The second is a (looser)
> restriction on the entire reply code.  For example, a server
> that used the reply code "472" would comply with the 4.3.2
> MUST, but since "472" is not listed in RFC 2821, the server
> would not be following the recommendation of the SHOULD from
> 4.2.

FWIW, that has been my understanding since long before 2821.

   john

Date: Mon, 23 Apr 2007 17:29:13 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:

>> You are correct.  Exim is broken!! <g>  but I guess will correct that to
>> make it unbroken by changing the specification.
> 
> Why do you contend that it is broken?  Seriously, if you're going to
> claim that an implementation is broken, you have to point to something in
> the RFC that the implementation violates (a MUST or a MUST NOT).

The ABNF says:

    Reply-line     = Reply-code [ SP text ] CRLF

it does not say:

    Reply-line     = Reply-code [ SP | "-" text ] CRLF

nor does it say:

    Reply-line     = Reply-code

Now of course, if you are going kludge an endorsement by changing the 
standard to:

    Reply-line     = *( Reply-code "-" [ text ] CRLF ) /
                   Reply-code [ SP text ] CRLF
    Reply-code     = %x31-35 %x30-35 %x30-39

in order to make BROKEN behavior officially compliant in 2821bis, then 
hey, more power to you.

> Unless you can point to something that says an implementation MUST NOT
> use the reply code in the first line of a multiline reply, 

see above

> you should
> stop claiming that Exim or Qmail are broken because they use the first
> line.

I understand the truth something does hurt. Its BROKEN CODE - PERIOD!

--
HLS

Date: Mon, 23 Apr 2007 17:15:54 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

David F. Skoll wrote:

>> For a completely unrecognized code, the client SHOULD ignore the code.
> 
> NO!  This is untenable.  A client CANNOT ignore the code, because SMTP
> *demands* that a client send a command after the server has sent a
> reply.

David, that is not quiet true. i.e. multi-line responses.

But then again, it all depends on the new 2821bis endorsement for a 
proper valid reply-code line from where a reply code is extracted.

In short, the client SHOULD not be sending a command after reading a 
continuation line.

The issue here was:

    150-eh? Got Milk?
    250 DONE

If your code is aborting on 150 and issueing a COMMAND on 150, then you 
are clearly more than broken.  But I guess we are endorsing the 
behavior, or are are outlawing multi-line responses?

> If a server sends:
> 
>      666 Mail diverted to the nether regions of the universe
> 
> The client *cannot* simply ignore it, or you deadlock.  

Are you stating how your client will behave or guessing other clients 
will deadlock as well?

How about if it was more realistically?

       120-Hold on. How about a cuban Cigar?
       550 I don't SMOKE!

> Now perhaps in an ideal world the client could respond:
> 
>      UHOH Dunno what the heck you mean

What ideal world would that be?

> but that's alas not part of the standard.  Therefore, the ONLY sane response
> is a QUIT or a RSET.

Again, a perfectly valid argument, but nonetheless a quess/opinion.

If you have read any further in my message, I recognize there are enough 
broken code out there that don't follow reply code specification anyway 
(but will with the few strokes of the pen), an abort proposal is 
probably ok.

Keep in mind, that you are PROMOTING change, and with CHANGE, it opens 
the door to doing things right.  So in the same vain you disagee with 
mixed continuation reply code and preached an extension, the same 
ideology applies here for aborting on unrecognized reply codes.

So if you are ready to make your delegate design changes to your 
programming staff to promote an ABORT on unrecognized reply codes, you 
might want to first be open to the idea of what is already existing in 
the specs to utilize these new 2821bis endorsed reply code control concepts.

I'm dead serious. What if your client software did encounter a 
transaction where the server did attempt to KEEP you alive with?

      C: DATA
      S: 352 start your engine
      [client begins upload]
      S: 150-Hmmm, you got some heavy stuff dude?
      S: 150-Need to do a check. please grab a seat...
      S: 250 Ok Charlie, its all good!
      C: QUIT

Are you honestly going to not support this in your commercial product 
line? As managers, we would be very sensitive to such issues.

Do not assume just become it is WRITTEN (no mixed codes in 2821bis) so 
it will be DONE.  We already proved how erroneous that is with broken 
software not following proper reply code reading.

I am not re-awakening the mix code issue. No, thats been decided. But 
rather how you are proposing the client should react to such a scenario.

Also keep in mind that implementators have different ways in performing 
their retries and in addition, in the advent of security, some servers 
may actually see dropping clients as suspicious.

--
HLS

Date: Mon, 23 Apr 2007 13:50:47 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Mon, Apr 23, 2007, Tony Hansen wrote:

> In order to help put this issue to bed, I'd like to get more input from
> actual SMTP deployments.

MeTA1 (successor of sendmail X): checks all lines and complains if
different codes are used.

Date: Mon, 23 Apr 2007 16:45:58 -0400
From: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> You are correct.  Exim is broken!! <g>  but I guess will correct that to
> make it unbroken by changing the specification.

Why do you contend that it is broken?  Seriously, if you're going to
claim that an implementation is broken, you have to point to something in
the RFC that the implementation violates (a MUST or a MUST NOT).

Unless you can point to something that says an implementation MUST NOT
use the reply code in the first line of a multiline reply, you should
stop claiming that Exim or Qmail are broken because they use the first
line.

> Corrected on Exim.  Qmail is broken. But it really doesn't matter any
> more, the decision was made to endorse the broken behavior.

Sigh.  In what way is requiring all lines of a multiline response to
contain the same code "broken"?  It's already specified that way in the RFC
according to my understanding.  We're just making it more obvious.

>> My conclusions stand: there is code out there that is in popular use
>> that will BREAK if the reply code on the first continuation line is
>> different from the final non-continuation line.

> Thats a given.  So they are BROKEN per x821 specs.

When it comes to (1) breaking your commercial SMTP server's
non-standard and weird use of multiline replies, or (2) breaking Exim
and Qmail's perfectly valid interpretation of RFC 2821, I'm afraid
there's no contest.

Regards,

David.

Date: Mon, 23 Apr 2007 16:32:47 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Tony Hansen wrote:
> Tony Hansen wrote:
>> A MUST will support and codify current behavior. A SHOULD will not and
>> can not. Therefore a MUST is necessary to move 2821bis to draft standard.
> 
> <process hat on>
> 
> I'm going to declare consensus on this issue. There has been much
> discussion, and not everyone is totally happy with the final
> conclusions, but enough evidence has been brought forward that I am
> convinced that this is the only viable way to move forward to draft
> standard.
> 
> </process hat off>

When I brought up the issue, I gambled that this would be the ultimate 
conclusion.  I wasn't going to bring it up knowing perfectly well there 
will be perfectly logically reasoning to lock this down.  However, my 
1st mistake was mis-reading you as I thought you were be the #1 person 
who would understand the entire picture because of OPES.  If this 
synergy was not present, I would be 100% on board with the consensus. My 
2nd mistake was underestimated how broken the few but very popular code 
are which is unfortunately enough to make endorsing decisions which is 
not otherwise done.  Its unfair, but thats reality.

In any case, I will be happy with this if the TEXT makes it very clear 
for clients to be more correct on reading its response codes.

In other words, they should FIX their code and simply stating servers 
MUST send persistent reply codes regardless if they are continuation 
lines or not should be coupled with clients focusing on the last 
reply-code[sp text] ABNF.  Otherwise, there is no point in any text 
regarding multi-line parsing.

         The first 3 digit in the receive buffer is
         the primary response code.

Period.

This reminds me of the similar issue with the white space after the MAIL 
FROM: command. An undocumented acceptable practice.

Thanks Tony (and John) for your patience.

--
HLS

Date: Mon, 23 Apr 2007 14:12:11 -0600
From: Philip Guenther <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "[email-masked]" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

On Mon, 23 Apr 2007, Hector Santos wrote:
...
> This 4.3.2 sentence makes it clear that "1yz " is not valid,
>
>   Unless extended using the mechanisms described in Section 2.2, SMTP
>   servers MUST NOT transmit reply codes to an SMTP client that are
>   other than three digits or that do not start in a digit between 2 and
>   5 inclusive.
>
> But it is also inconsistent with the 4.2 sentence:
>
>   An SMTP server SHOULD send only the reply codes listed in this
>   document.

Those two statements are not actually inconsistent.  The first, from 
4.3.2, only restricts what the server can use for the first digit of a 
reply code.  The second is a (looser) restriction on the entire reply 
code.  For example, a server that used the reply code "472" would comply 
with the 4.3.2 MUST, but since "472" is not listed in RFC 2821, the server 
would not be following the recommendation of the SHOULD from 4.2.

Philip Guenther

Date: Mon, 23 Apr 2007 16:08:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Tony Hansen wrote:

> You missed the most important piece of information, the END of the function:
> 
> return buffer[0] == okdigit;

You are correct.  Exim is broken!! <g>  but I guess will correct that to 
make it unbroken by changing the specification.

>>> Smail uses the *first* code.
>> Looking for source. Do you have a URL?
> 
> ftp://ftp.planix.com/pub/Smail/smail-3.2.0.121.tar.gz

Ok, I'll trust your review of it.

>>> hMailServer appears to use the *first* code.
>> Per specification, it looks for the first NON-CONTINUATION reply code,
>> however, this code is buggy and shouldn't be included in your test.  It
>> only looks for hard coded "250" values and it only supports 1 extra
>> multiple line from what I can see.  This would be the least of its
>> problems.  Throw this one out.
> 
> We're both wrong on this one. The state machine of hMailServer *skips*
> over *all* continuation lines. It then uses the reply code in the *last*
> response line.

This code was little harder to follow on whether it skipped all 
continuation lines.

> 
>>> qmail uses the *first* code.

>> Yes it does - but it should noted it is doesn't care for continuation
>> lines which is obvious broken code and n violation of the formal
>> reply-code[sp] specification.
>>
>> unsigned long smtpcode()
>> {
>>   unsigned char ch;
>>   unsigned long code;
>>
>>   if (!stralloc_copys(&smtptext,"")) temp_nomem();
>>
>>   get(&ch); code = ch - '0';
>>   get(&ch); code = code * 10 + (ch - '0');
>>   get(&ch); code = code * 10 + (ch - '0');
>>   for (;;) {
>>     get(&ch);
>>     if (ch != '-') break;   <---------- BREAK ON NON-DASH
>>     while (ch != '\n') get(&ch);
>>     get(&ch);
>>     get(&ch);
>>     get(&ch);
>>   }
>>   while (ch != '\n') get(&ch);
>>
>>   return code;  <--- return first code -- BROKEN CODE!
>> }
>>
>> It simply isn't following specs.  Do you wish to encourage this behavior?
> 
> Excuse me: It's looping over all continuation lines, stopping on the
> first non-continuation line, and then returning the code for those
> response lines.

Right, I was agreeing with you. You said above it was using the first 
code which you were correct. It only parses the first 3 digits in the 
buffer, but continues to read/dump, breaking until the 4th digit is no 
longer a "dash".

> That is definitely following the spec that we've been arguing over. Yes,
> it takes a couple of short cuts, only looking for the dash to decide if
> it should continue or not and ignoring bad input lines that don't have
> either a space or dash in the 4th character of the line, but otherwise
> it's following the use-first-code interpretation of the specs.

Hmmmmm, based on the code above, it is parsing the first 3 digits as 
numeric, then begins a loop

   loop:
      - read 4th
      - compare with dash, break if not dash
      - filter LF
      - read 3 characters (with a presumption they are digits)

but it returns the first converted number for 3 digits in the buffer, 
thus ignoring the fact that the first line may be a continuation.

Broken per x821?  Compliant under 2821bis?

> As I said above, your analysis of both Exim and qmail is flawed; my
> conclusions on those remains.

Corrected on Exim.  Qmail is broken. But it really doesn't matter any 
more, the decision was made to endorse the broken behavior.

> My conclusions stand: there is code out there that is in popular use
> that will BREAK if the reply code on the first continuation line is
> different from the final non-continuation line.

Thats a given.  So they are BROKEN per x821 specs.

> Part of the rules we are following for moving to draft standard is that
> we MUST look at how current code has interpreted the proposed standard.

Which is the premise I used to begin this entire issue.  Lets make them 
compliant now in 2821bis.  That should give a few people a pat on the back.

> A MUST will support and codify current behavior.  A SHOULD will not and
> can not. 

I disagree with your logic but thats ok.  Enough nerves were touch here 
with broken BIG software that it is more important to make them 
compliant and endorse future similar behavior by virtue of the new draft.

> Therefore a MUST is necessary to move 2821bis to draft standard.

So does that mean the ABNF will change?  You are endorsing broken 
behavior and future designs to ignore continuation lines as unnecessary 
other than to terminate a response. Only the first 3 digits in the 
received buffer is important.

--
HLS

Date: Mon, 23 Apr 2007 16:06:34 -0400
From: "David F. Skoll" <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

Hector Santos wrote:

>    Consequently, a sender-SMTP MUST be prepared to handle codes not
>    specified in this document and MUST do so by interpreting the first
>    digit only.

The meanings of first digits other than 2 through 5 are not defined.
First-digit "1" is mentioned, but not really defined.

> For a completely unrecognized code, the client SHOULD ignore the code.

NO!  This is untenable.  A client CANNOT ignore the code, because SMTP
*demands* that a client send a command after the server has sent a
reply.

If a server sends:

     666 Mail diverted to the nether regions of the universe

The client *cannot* simply ignore it, or you deadlock.  The server is
waiting for the client to do something, and the client sits there doing
nothing.  So the client HAS to react somehow.

Now perhaps in an ideal world the client could respond:

     UHOH Dunno what the heck you mean

but that's alas not part of the standard.  Therefore, the ONLY sane response
is a QUIT or a RSET.

Regards,

David.

Date: Mon, 23 Apr 2007 15:41:13 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked], David F Skoll <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

John C Klensin wrote:
> 
> Hmm.  Do you want to requeue it and keep trying for four or five
> days on the theory that the server might spontaneously fix
> itself?  Or would it be better to bounce the puppy and hope that
> someone will complain?    I think I can argue that one either
> way.
> 
>> It does need to be a SHOULD so there's an out for dealing with
>> specific cases of brokenness where 5yz treatment has been
>> determined to be more appropriate operationally.

John, I think we all need to sleep on this more. :-)

First, this is definitely functionality change.

Second, the existing text in various spots already supports the idea of 
handling unrecognized codes. I've cut snippets below for your quick 
reading.

But before I continue, I would like to note that after researching many 
of the codes out there, most do not support the level of details for the 
reply codes as written in the text.  So even though it is already 
documented, as we seen with the existence of broken software now deemed 
"high quality," I don't know what is the worth for the level of details 
here.  i.e, it might be too complex.

That said, the text in 4.2. SMTP Replies, already has:

    Consequently, a sender-SMTP MUST be prepared to handle codes not
    specified in this document and MUST do so by interpreting the first
    digit only.

This is reiterated and reinforced in 4.3.2. Command-Reply Sequences.

    Since some servers may generate other
    replies under special circumstances, and to allow for future
    extension, SMTP clients SHOULD, when possible, interpret only the
    first digit of the reply and MUST be prepared to deal with
    unrecognized reply codes by interpreting the first digit only.

This 4.3.2 sentence makes it clear that "1yz " is not valid,

    Unless extended using the mechanisms described in Section 2.2, SMTP
    servers MUST NOT transmit reply codes to an SMTP client that are
    other than three digits or that do not start in a digit between 2 and
    5 inclusive.

But it is also inconsistent with the 4.2 sentence:

    An SMTP server SHOULD send only the reply codes listed in this
    document.

In 4.2.1.  Reply Code Severities and Theory, we have:

    An unsophisticated SMTP client, or one that receives an unexpected
    code, will be able to determine its next action (proceed as planned,
    redo, retrench, etc.) by examining this first digit.  An SMTP client
    that wants to know approximately what kind of error occurred (e.g.,
    mail system error, command syntax error) may examine the second
    digit.  The third digit and any supplemental information that may be
    present is reserved for the finest gradation of information.

This provides the framework on how to deal with undefined reply codes.

Rather than open a can of worms with defining new "RETRY" functionality 
for aborting clients, you can consider to simply use, change or define 
one of first digit codes as "ignore."

Ideally, x2z would be appropriate since we are partly dealing with 
connection issues.

In other words, these can be use to provide quideline as what may be 
recommended or augmented into the retry logic/recommendations.

For a completely unrecognized code, the client SHOULD ignore the code. 
It MAY begin its own timeout consideration.  It MAY also abort.  But I 
don't think that is not appropriate given what we already define about.

But like I said, I'm at a point now with the more than expected time and 
resources spent on this to do full source code analysis of many of these 
codes, that I recognized these level of spec reply code details are 
simply not followed.

As with the case with the first reply code issue, there is enough 
powerful voices here that would rather now endorse this BROKEN code 
behavior as high quality and compliant code in 2821bis, and even 
encourage the design as simple, optimized and high quality.

So given that the specification is far too complex and no common support 
for reply codes other than the pure basic codes, adding new layman logic 
to guide clients to "sanely" abort on unrecognized reply codes is as 
simple as it gets.

--
HLS

Date: Mon, 23 Apr 2007 15:33:36 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Tony Hansen wrote:
> 
> A MUST will support and codify current behavior. A SHOULD will not and
> can not. Therefore a MUST is necessary to move 2821bis to draft standard.

<process hat on>

I'm going to declare consensus on this issue. There has been much
discussion, and not everyone is totally happy with the final
conclusions, but enough evidence has been brought forward that I am
convinced that this is the only viable way to move forward to draft
standard.

</process hat off>

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 15:12:20 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector, thank you for wanting to dig further into the code. See below.

Hector Santos wrote:
>
> Tony Hansen wrote:
>> In order to help put this issue to bed, I'd like to get more input from
>> actual SMTP deployments.
>
> ...
> Are you looking for CONTINUATION or NON-CONTINUATION lines?

The issue is the *set* of response lines that come back in response to a
command. This consists of all CONTINUATION lines followed by one
NON-CONTINUATION lines. So the answer is "yes" to both.

> But with Exim, per specification Exim definitely uses the first
> *non-continuation* reply code. Below is the loop in the function
> smtp_read_response() with loop break based on the first non-dash line
> encountered. The behavior is also documented as such in the comment block:
> ...

You missed the most important piece of information, the END of the function:

return buffer[0] == okdigit;

THIS is where it determines which code is important. And buffer is
pointing at the BEGINNING of the concatenated list of all CONTINUATION
and the NON-CONTINUATION response lines.

>> Smail uses the *first* code.
>
> Looking for source. Do you have a URL?

ftp://ftp.planix.com/pub/Smail/smail-3.2.0.121.tar.gz

>> hMailServer appears to use the *first* code.
>
> Per specification, it looks for the first NON-CONTINUATION reply code,
> however, this code is buggy and shouldn't be included in your test.  It
> only looks for hard coded "250" values and it only supports 1 extra
> multiple line from what I can see.  This would be the least of its
> problems.  Throw this one out.

We're both wrong on this one. The state machine of hMailServer *skips*
over *all* continuation lines. It then uses the reply code in the *last*
response line.

>> qmail uses the *first* code.
>
> Yes it does - but it should noted it is doesn't care for continuation
> lines which is obvious broken code and n violation of the formal
> reply-code[sp] specification.
>
> unsigned long smtpcode()
> {
>   unsigned char ch;
>   unsigned long code;
>
>   if (!stralloc_copys(&smtptext,"")) temp_nomem();
>
>   get(&ch); code = ch - '0';
>   get(&ch); code = code * 10 + (ch - '0');
>   get(&ch); code = code * 10 + (ch - '0');
>   for (;;) {
>     get(&ch);
>     if (ch != '-') break;   <---------- BREAK ON NON-DASH
>     while (ch != '\n') get(&ch);
>     get(&ch);
>     get(&ch);
>     get(&ch);
>   }
>   while (ch != '\n') get(&ch);
>
>   return code;  <--- return first code -- BROKEN CODE!
> }
>
> It simply isn't following specs.  Do you wish to encourage this behavior?

Excuse me: It's looping over all continuation lines, stopping on the
first non-continuation line, and then returning the code for those
response lines.

That is definitely following the spec that we've been arguing over. Yes,
it takes a couple of short cuts, only looking for the dash to decide if
it should continue or not and ignoring bad input lines that don't have
either a space or dash in the 4th character of the line, but otherwise
it's following the use-first-code interpretation of the specs.

>> postfix appears to use the *last* code.
>> jsmtp uses the *last* code.
>>
>> Courier generally uses the *first* code. In some cases it actually
>> treats 1yz, 2yz and 3yz as equivalent and will totally miss any
>> subsequent 4yz/5yz.
>
> What about the first non-continuation for its reply?

I repeat, it generally uses the *first* code in a multiline response. It
correctly stops when it reaches the non-continuation line, but it's the
*first* code that's important to it.

>> The use of the first response code appears to be used just as often as
>> the last response code. The codes in between seem to be universally
>> ignored.
>
> IMO, that is not a valid conclusion.
>
> With the exception of one package (QMAIL) which is clearly violating
> 821/2821 formal specifications, at least one of the codes (hMailServer)
> is very SMTP-buggy and is not worth any consideration.  You were
> incorrect with Exim and it isn't clear if analysis  s considering mixed
> non-continuation codes which is already conflictive with the
> specification.  You also didn't analyze a good bit of other popular main
> stream packages in the market, including none of the major MUAs which we
> have tested.

As I said above, your analysis of both Exim and qmail is flawed; my
conclusions on those remains.

The MUAs are only one piece of the MUA->MTA->...->MTA->MDA SMTP flow.
That they tend to use the last code is a datapoint.

We've also heard about a couple other major MTAs, PMDF and Sun Messaging
Server, that use the first code.

My conclusions stand: there is code out there that is in popular use
that will BREAK if the reply code on the first continuation line is
different from the final non-continuation line.

Part of the rules we are following for moving to draft standard is that
we MUST look at how current code has interpreted the proposed standard.

In this particular case, allowing anything other than the same code for
the first continuation line and the last non-continuation line will
break code. Lots of the code.

Making it a SHOULD would allow future implementations to break lots of
code that exists today. That is not a viable output of moving to draft
standard.

> IMO, his is precisely the reason why it should be SHOULD and not a
> MUST.
>
> A MUST encourages broken code behavior by falsely encouraging poor
> logic and pre 2821bis non-compliant logic that it is OK to read the
> first code because there is NEW presumption that the codes will be
> persistent.

Calling the interpretation of using the first reply code "poor logic and
non-compliant" is an not borne out.

> By using a MUST you are saying that FUTURE clients should use a
> persistent reply code when in fact it is very conceivable that it
> might encounter a failure.

Yes.

> The solution is quite simple - follow the 25 year old spec of looking
> for the first non-continuation line.

Nothing has changed with respect to looking for the first
non-continuation line for when to stop looking for response lines.

> Maybe Postels conservative/liberal axiom should apply:
>
>     Servers SHOULD send persistent reply codes, however, clients
>     SHOULD be aware for mixed reply codes scenarios and SHOULD
>     only react to the first non-continuation reply code.
>
> This is consistent with Postel's rule, it is consistent with past,
> present and future considerations, backward compatible and it is not a
> functional change on the specs. On the other hand, a MUST will attempt
> to enforce change. The MUST should not imply that CLIENT should begin
> enforcing persistent reply codes or just rely on the first line. It
> encourage incorrect designs of not ignoring continuation lines.
>
> The point again is that it has been POSSIBLE for 25 years and a MUST
> would be changing that now and also encourage designs that may not be
> honored.

A MUST will support and codify current behavior. A SHOULD will not and
can not. Therefore a MUST is necessary to move 2821bis to draft standard.

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 13:34:11 -0500
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: RFC2821bis-02 Issue 27: Received clauses
Sender: [email-masked]

And here I was thinking that I was going to get to duck all of this.

Some confusion in terminology (and some broken syntax in 2822) led to 
a short discussion resulting in the following:

- 2821bis and 2822upd will both define "Header Section" as the 
complete set of header fields at the beginning of a message.

- 2821bis and 2822upd will both define "Header Field" as a line in a 
message with a field name followed by a colon followed by a field 
body.

- 2821bis and 2822upd will both avoid using the terms "header" and 
"headers" in isolation to avoid ambiguity.

- 2821bis will use the term "received clause" to refer to a keyword 
and the set of stuff that follows it before the next keyword (so, 
"for [email-masked]" is a received clause).

- 2822upd will simplify the syntax of received clauses (what is now 
"name-val-list") such that it's simply "*(word / domain / addr-spec / 
angle-addr)" and let 2821bis do the real definition.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 23 Apr 2007 14:33:13 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> Well, since you have no real mail system design or production
>> experience, I think I will just skip replying to you.
> 
> Please check http://www.roaringpenguin.com.
> 
> I am the principal developer of that product.
> 
> Just because I haven't personally coded up a production SMTP client doesn't
> mean I have no "real mail system design or production experience".  We have
> our software running on hundreds of client systems, some of which process
> many millions of messages/day.
> 
>> Its ashame that
>> vote counts has any weight at all in this critical design issue.  You
>> are simply not the right person to comment on the matter.
> 
> Comments like that are simply unbecoming of this list.

Frist, your are right, I apologize.

Second, the problem is that you're quessing now in how "things" should work.

--
HLS

Date: Mon, 23 Apr 2007 14:31:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: David F Skoll <[email-masked]>, [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

[email-masked] wrote:
> 
>> John C Klensin wrote:
> 
> 
>> Well, it's not just 1yz that worries me.  It's differing reply codes
>> in a multiline reply.  As Tony Hansen pointed out, there are SMTP
>> implementations that do not simply use the last reply code, and
>> although their quality may be in question, they are in widespread use
>> and breaking them is not a good idea.
> 
> There's no quality issue here. As you yourself have pointed out, the
> specifications are pretty clear that all the codes are supposed to be the same
> and multiple acts of creative reading are needed to support any other
> conclusion. And someone else has pointed out that using the first code can
> simplify implementation. So: The standards permit use of the first code and it
> can be simpler. Seems like a reasonably high quality choice to me.

+1. Very high quality!

Since we have so many high quality 821/2821 "broken" software now, and 
the censensus is a MUST for persistent reply codes, John might as well 
change the ABNF to support ignoring continuation lines and just use the 
first line reply code.

    Reply-line     = Reply-code [ SP | "-" text ] CRLF

That way EXIM and SUN can claim 100% SMTP compliancy!

--
HLS

Date: Mon, 23 Apr 2007 14:28:50 -0400
From: Jeff Macdonald <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], "David F. Skoll" <[email-masked]>
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

On Mon, Apr 23, 2007 at 06:18:15PM +0200, Arnt Gulbrandsen wrote:
> 
> David F. Skoll writes:
> >(I'm not too sure what to do if an undefined code is received in 
> >response to a RCPT command. It's clear to me that you don't want to 
> >continue the SMTP transaction, but do you want to fail all the 
> >recipients and generate a non-delivery notification?
> 
> The latter. Absolutely. The response is of a kind the client doesn't 
> know, so it has absolutely no basis for assuming that the error relates 
> only to this client.

+1

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Mon, 23 Apr 2007 13:32:14 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

John C Klensin wrote:

> --On Monday, 23 April, 2007 09:03 -0700
> [email-masked] wrote:

[David Skoll]
>>> If an SMTP client receives a reply code that is not defined
>>> by this RFC or a negotiated extension, it SHOULD send a RSET
>>> or QUIT command, and SHOULD treat the unknown reply code as a
>>> 5xx code.

[Ned]
>> I'm with you up to the 5yz code. I think 4yz would be more
>> appropriate - that way you don't bounce messages unnecessarily
>> when dealing with a broken server.

[John]
> Hmm.  Do you want to requeue it and keep trying for four or five
> days on the theory that the server might spontaneously fix
> itself?  Or would it be better to bounce the puppy and hope that
> someone will complain?    I think I can argue that one either
> way.

I prefer 5yz.  It makes the problem immediately obvious; there's
nothing worse than finding out days later that important e-mail didn't
get through because of a broken server.  As a general principle, I
think anything that doesn't "make sense" should be treated as 5yz.  I
have a lot of experience from our support line that mail
administrators often ignore 4yz failures in their logs, whereas they
react pretty quickly to complaints about bounces!

However, it's a SHOULD, so a client could treat it as a 4yz if it
really wants.  Actually, the more I think about it, the more it might
make sense to treat it as a 4yz if there are any lower-priority MX
hosts left to try, and a 5yz if not.  But putting that in an RFC is
kind of ugly; I wouldn't know how to word it nicely.

Regards,

David.

Date: Mon, 23 Apr 2007 13:05:56 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], David F Skoll <[email-masked]>
cc: [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

--On Monday, 23 April, 2007 09:03 -0700
[email-masked] wrote:

>> John C Klensin wrote:
> 
>> > The strawman proposed text would be a provision that a
>> > client SHOULD (or perhaps even MUST) send a RSET or QUIT in
>> > response to a code whose first digit is undefined by either
>> > 2821bis or a negotiated extension.
> 
>> I would agree with that.  Something like:
> 
>> If an SMTP client receives a reply code that is not defined
>> by this RFC or a negotiated extension, it SHOULD send a RSET
>> or QUIT command, and SHOULD treat the unknown reply code as a
>> 5xx code.
> 
> I'm with you up to the 5yz code. I think 4yz would be more
> appropriate - that way you don't bounce messages unnecessarily
> when dealing with a broken server.

Hmm.  Do you want to requeue it and keep trying for four or five
days on the theory that the server might spontaneously fix
itself?  Or would it be better to bounce the puppy and hope that
someone will complain?    I think I can argue that one either
way.

> It does need to be a SHOULD so there's an out for dealing with
> specific cases of brokenness where 5yz treatment has been
> determined to be more appropriate operationally.

After some sleep, either way, I think it can't be more than a
SHOULD.

>...

We agree about the multiline stuff.

     john

Date: Mon, 23 Apr 2007 18:18:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Cc: "David F. Skoll" <[email-masked]>
Sender: [email-masked]

David F. Skoll writes:
> (I'm not too sure what to do if an undefined code is received in 
> response to a RCPT command. It's clear to me that you don't want to 
> continue the SMTP transaction, but do you want to fail all the 
> recipients and generate a non-delivery notification?

The latter. Absolutely. The response is of a kind the client doesn't 
know, so it has absolutely no basis for assuming that the error relates 
only to this client.

Falling back to the next MX is vastly better than assuming anything 
about a 7xx response.

Arnt

From: [email-masked]
Cc: [email-masked]
Date: Mon, 23 Apr 2007 09:03:52 -0700 (PDT)
Subject: Re: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
To: David F Skoll <[email-masked]>
Sender: [email-masked]

> John C Klensin wrote:

> > The strawman proposed text would be a provision that a client
> > SHOULD (or perhaps even MUST) send a RSET or QUIT in response to
> > a code whose first digit is undefined by either 2821bis or a
> > negotiated extension.

> I would agree with that.  Something like:

> If an SMTP client receives a reply code that is not defined by this
> RFC or a negotiated extension, it SHOULD send a RSET or QUIT command, and
> SHOULD treat the unknown reply code as a 5xx code.

I'm with you up to the 5yz code. I think 4yz would be more appropriate - that
way you don't bounce messages unnecessarily when dealing with a broken server.

It does need to be a SHOULD so there's an out for dealing with specific cases
of brokenness where 5yz treatment has been determined to be more appropriate
operationally.

> (I'm not too sure what to do if an undefined code is received in response
> to a RCPT command.  It's clear to me that you don't want to continue the
> SMTP transaction, but do you want to fail all the recipients and
> generate a non-delivery notification?  Or do you want to just fail the
> one recipient and retry the others (if any)?)

> > That brings us back to 1yz.

> Well, it's not just 1yz that worries me.  It's differing reply codes
> in a multiline reply.  As Tony Hansen pointed out, there are SMTP
> implementations that do not simply use the last reply code, and
> although their quality may be in question, they are in widespread use
> and breaking them is not a good idea.

There's no quality issue here. As you yourself have pointed out, the
specifications are pretty clear that all the codes are supposed to be the same
and multiple acts of creative reading are needed to support any other
conclusion. And someone else has pointed out that using the first code can
simplify implementation. So: The standards permit use of the first code and it
can be simpler. Seems like a reasonably high quality choice to me.

				Ned

From: [email-masked]
Cc: [email-masked]
Date: Mon, 23 Apr 2007 07:30:00 -0700 (PDT)
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
To: Tony Hansen <[email-masked]>
Sender: [email-masked]

> In order to help put this issue to bed, I'd like to get more input from
> actual SMTP deployments.

> For example, we've heard from one of the Exim developers that it uses
> the first code returned. And sendmail's & Hector's code reportedly uses
> the last.

> So I decided to check out some of the accessible open source code bases,
> focusing on the server code that *sends* email to the next MTA.

> I verified that sendmail definitely uses the last code and the Exim code
> definitely uses the *first* code of the response set.

> AT&T's own MTA's code uses the last code, ignoring the code on all
> previous continuation lines.

> Smail uses the *first* code.

> hMailServer appears to use the *first* code.

> qmail uses the *first* code.

> postfix appears to use the *last* code.

> jsmtp uses the *last* code.

> Courier generally uses the *first* code. In some cases it actually
> treats 1yz, 2yz and 3yz as equivalent and will totally miss any
> subsequent 4yz/5yz.

Here are two more: PMDF generally uses the first code and so does the Sun
Messaging Server. Additionally, neither of these will treat 1xy as anything
other than a server error.

> The use of the first response code appears to be used just as often as
> the last response code. The codes in between seem to be universally ignored.

> Other than ours and sendmail, I can't say much about how much all of
> these code bases are used, but some of them are certainly used with
> multiple millions of messages daily, and I think the sample is large
> enough to draw some conclusions:

> It clearly indicates to me that at least the first and last reply codes
> need to be the same. If they are NOT the same, >>>CODE BREAKS<<<. This
> translates, to me, as indicating that we *NEED* to use "MUST" and not
> "SHOULD".

Which is why I supported option (iii) in the earlier poll, which I beleve
showed an overwhelming consensus for requiring all the codes be the same.

				Ned

Date: Mon, 23 Apr 2007 10:14:16 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: New issue (was Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

John C Klensin wrote:

> The strawman proposed text would be a provision that a client
> SHOULD (or perhaps even MUST) send a RSET or QUIT in response to
> a code whose first digit is undefined by either 2821bis or a
> negotiated extension.

I would agree with that.  Something like:

If an SMTP client receives a reply code that is not defined by this
RFC or a negotiated extension, it SHOULD send a RSET or QUIT command, and
SHOULD treat the unknown reply code as a 5xx code.

(I'm not too sure what to do if an undefined code is received in response
to a RCPT command.  It's clear to me that you don't want to continue the
SMTP transaction, but do you want to fail all the recipients and
generate a non-delivery notification?  Or do you want to just fail the
one recipient and retry the others (if any)?)

> That brings us back to 1yz.

Well, it's not just 1yz that worries me.  It's differing reply codes
in a multiline reply.  As Tony Hansen pointed out, there are SMTP
implementations that do not simply use the last reply code, and
although their quality may be in question, they are in widespread use
and breaking them is not a good idea.

Regards,

David.

Date: Mon, 23 Apr 2007 07:44:39 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Tony Hansen wrote:
> In order to help put this issue to bed, I'd like to get more input from
> actual SMTP deployments.

> For example, we've heard from one of the Exim developers that it uses
> the first code returned. And sendmail's & Hector's code reportedly uses
> the last.

> So I decided to check out some of the accessible open source code bases,
> focusing on the server code that *sends* email to the next MTA.

Tony,

I appreciate you taking the time to explore this. Btw, Hector's code is 
called Wildcat! SMTP (wcSMTP)

Are you looking for CONTINUATION or NON-CONTINUATION lines?

> I verified that sendmail definitely uses the last code and the Exim code
> definitely uses the *first* code of the response set.

Already confirm with sendmail last week.

But with Exim, per specification Exim definitely uses the first 
*non-continuation* reply code. Below is the loop in the function 
smtp_read_response() with loop break based on the first non-dash line 
encountered. The behavior is also documented as such in the comment block:

for (;;)
   {
   if ((count = read_response_line(inblock, ptr, size, timeout)) < 0)
     return FALSE;

     HDEBUG(D_transport|D_acl|D_v)
     debug_printf("  %s %s\n",
            (ptr == buffer)? "SMTP<<" : "      ", ptr);

   /* Check the format of the response: it must start with three digits;
      if these are followed by a space or end of line, the response is
-->  complete. If they are followed by '-' this is a multi-line response
-->  and we must look for another line until the final line is reached. 

      The only use made of multi-line responses is to pass them back as
      error messages. We therefore just concatenate them all within the
      buffer, which should be large enough to accept any reasonable
      number of lines. */

   if (count < 3 ||
      !isdigit(ptr[0]) ||
      !isdigit(ptr[1]) ||
      !isdigit(ptr[2]) ||
      (ptr[3] != '-' && ptr[3] != ' ' && ptr[3] != 0))
     {
     errno = ERRNO_SMTPFORMAT;    /* format error */
     return FALSE;
     }

   /* If the line we have just read is a terminal line, line, we
      are done. Otherwise more data has to be read. */

   if (ptr[3] != '-') break;  <--------- BREAK OUT OF LOOP

   /* Move the reading pointer upwards in the buffer and
      insert \n between the components of a multiline response.
      Space is left for this by read_response_line(). */

   ptr += count;
   *ptr++ = '\n';
   size -= count + 1;
   }

> AT&T's own MTA's code uses the last code, ignoring the code on all
> previous continuation lines.

> Smail uses the *first* code.

Looking for source. Do you have a URL?

> hMailServer appears to use the *first* code.

Per specification, it looks for the first NON-CONTINUATION reply code, 
however, this code is buggy and shouldn't be included in your test.  It 
only looks for hard coded "250" values and it only supports 1 extra 
multiple line from what I can see.  This would be the least of its 
problems.  Throw this one out.

> qmail uses the *first* code.

Yes it does - but it should noted it is doesn't care for continuation 
lines which is obvious broken code and n violation of the formal 
reply-code[sp] specification.

unsigned long smtpcode()
{
   unsigned char ch;
   unsigned long code;

   if (!stralloc_copys(&smtptext,"")) temp_nomem();

   get(&ch); code = ch - '0';
   get(&ch); code = code * 10 + (ch - '0');
   get(&ch); code = code * 10 + (ch - '0');
   for (;;) {
     get(&ch);
     if (ch != '-') break;   <---------- BREAK ON NON-DASH
     while (ch != '\n') get(&ch);
     get(&ch);
     get(&ch);
     get(&ch);
   }
   while (ch != '\n') get(&ch);

   return code;  <--- return first code -- BROKEN CODE!
}

It simply isn't following specs.  Do you wish to encourage this behavior?

> postfix appears to use the *last* code.
> jsmtp uses the *last* code.
> 
> Courier generally uses the *first* code. In some cases it actually
> treats 1yz, 2yz and 3yz as equivalent and will totally miss any
> subsequent 4yz/5yz.

What about the first non-continuation for its reply?

> The use of the first response code appears to be used just as often as
> the last response code. The codes in between seem to be universally ignored.

IMO, that is not a valid conclusion.

With the exception of one package (QMAIL) which is clearly violating 
821/2821 formal specifications, at least one of the codes (hMailServer) 
is very SMTP-buggy and is not worth any consideration.  You were 
incorrect with Exim and it isn't clear if analysis  s considering mixed 
non-continuation codes which is already conflictive with the 
specification.  You also didn't analyze a good bit of other popular main 
stream packages in the market, including none of the major MUAs which we 
have tested.

> It clearly indicates to me that at least the first and last reply codes
> need to be the same. If they are NOT the same, >>>CODE BREAKS<<<. This
> translates, to me, as indicating that we *NEED* to use "MUST" and not
> "SHOULD".

IMO, his is precisely the reason why it should be SHOULD and not a MUST.

A MUST encourages broken code behavior by falsely encouraging poor logic 
and pre 2821bis non-compliant logic that it is OK to read the first code 
because there is NEW presumption that the codes will be persistent.

By using a MUST you are saying that FUTURE clients should use a 
persistent reply code when in fact it is very conceivable that it might 
encounter a failure.

The solution is quite simple - follow the 25 year old spec of looking 
for the first non-continuation line.

Maybe Postels conservative/liberal axiom should apply:

     Servers SHOULD send persistent reply codes, however, clients
     SHOULD be aware for mixed reply codes scenarios and SHOULD
     only react to the first non-continuation reply code.

This is consistent with Postel's rule, it is consistent with past, 
present and future considerations, backward compatible and it is not a 
functional change on the specs. On the other hand, a MUST will attempt 
to enforce change. The MUST should not imply that CLIENT should begin 
enforcing persistent reply codes or just rely on the first line. It 
encourage incorrect designs of not ignoring continuation lines.

The point again is that it has been POSSIBLE for 25 years and a MUST 
would be changing that now and also encourage designs that may not be 
honored.

--
HLS

Date: Mon, 23 Apr 2007 07:17:10 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra  return-path headers in 4.4
Sender: [email-masked]

John,

My input, if I am following the issue right, is that this will mandate 
code change. Second, I never believe that SMTP software should alter 
headers or be responsible for cleaning it up.  I always felt this 
invites trouble - and it has and most likely will conflict with DKIM 
mail integrity issues if and when that protocol becomes widely adopted.

--
HLS

John C Klensin wrote:
> 
> 
> --On Monday, 23 April, 2007 03:34 +0200 Frank Ellermann
> <[email-masked]> wrote:
> 
>> Hector Santos wrote:
>>
>>> This changes CODE!!! My code!!
>> Pardon ?  Users of a server using your code certainly expect
>> that there is precisely one Return-Path, reflecting the MAIL
>> FROM as seen by the final delivery MTA.
>>
>> I'd bet that your code already does this, or do you keep crappy
>> Return-Paths inserted by prior hops in the header ?  What would
>> users do with that, where "user" can be anything from a stupid
>> vacation script and other auto-responders up to gateways.
>>
>>> What does this have to to with SMTP?
>> SMTP specifies who, when. what, and how for the Return-Path.
>> In RFC 2822 you also find that there's at most one
>> Return-Path, not more.
>> ...
> 
> Frank, Hector,
> 
> FWIW, let me give a different view of this issue, a view that is
> consistent with 2821.  For the purposes of this note, I'll
> ignore both whether it is consistent with the current state of
> 2821bis or with various perceptions of today's changing reality.
> 
> Along the transport path, multiple Return-path headers may
> happen and may happen regardless of whatever a standard says.
> They may happen because someone misbehaves.  They may happen
> because something thought it was a final delivery server and
> then ended up having to relay or gateway the mail because of
> something that happened downstream ... and did so at a point
> where the relevant code could not rescan the headers to be sure
> that the previously-inserted Return-path header was removed.
> 
> In any of those cases, a receiving MUA, or the delivery system
> immediately in front of it, has to be prepared to deal with a
> header collection that contains more than one Return-path and to
> figure out either
> 
> 	(i) how to ignore all but one of them or 
> 	
> 	(ii) to decide that a message with two (or more)
> 	Return-paths in it is such a heinous situation that the
> 	entire message should be immediately discarded without
> 	comment.  Even if one liked NDN messages, one can't
> 	return one in this case because making a choice of where
> 	to return it turns the case into (i).
> 
> I suggest to you that any real mail product that serves real
> customers who can choose other products will make the first
> choice --to figure out how to deliver the message if it can
> possibly be delivered -- because it is generally the only choice
> users will accept.   It may decide to deliver it with several
> demerit points for the excessive Return-path headers, or drop it
> in a "possibly junk" folder, but it will deliver it.    And, if
> the user decides to accept and process the thing, the MUA, or
> mail store, or delivery system will somehow figure out which
> Return-path header(s) to ignore.
> 
> Now, I think that is the reality, regardless of what any
> standard says.   The implications of that reality are the reason
> why 2821 says "you may take that out" rather than "you are
> really, really, evil if you don't take it out".  Even if the
> latter were there, it would not guarantee the receiver that
> multiple Return-paths would not show up.  And, if the receiver
> can't be guaranteed that, then it needs a coping strategy -- one
> that might fall anywhere on the spectrum from "simplistic" to
> "really clever heuristics".
> 
> In theory, one could assign "likelihood that a message is bogus"
> scores on the basis on whether the standard say MAY, SHOULD NOT,
> or MUST NOT.  But in practice, for this sort of situation (many
> others are different), that would be likely to be
> non-productive.  The dynamics of these situations change, the
> "badness" of an extra Return-path header varies with context
> (e.g., I'd be reluctant to get very excited about two of three
> Return-path headers with exactly the same content) and the life
> expectancy of a standard is just too long relative to changes in
> bad guy (or sloppy implementation) behavior to provide good
> guidance.
> 
> So while I'm happy to change the document in whatever way Tony
> says there is consensus, I am having a lot of trouble seeing
> what all the excitement is about.
> 
> regards,
>     john
> 
> 
> 
> 
> 
> 

Date: Mon, 23 Apr 2007 05:47:54 -0400
From: John C Klensin <[email-masked]>
To: "David F. Skoll" <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Sunday, 22 April, 2007 21:56 -0400 "David F. Skoll"
<[email-masked]> wrote:

>> Second, given the fact that 1yx are not currently defined for
>> current commands, it should be ignored.
> 
> Umm... no, I don't think so!
> 
> If you're writing a state machine (SMTP client) and it
> receives some unknown reply code from the server, the ONLY
> sane course of action is to QUIT or RSET.  The poor client has
> no idea what the server means; it could be expecting the
> client to alter its state in some way rendering subsequent
> commands useless or dangerous.

Let me suggest looking at this particular piece of the argument
a different way, because it is an area in which we could put
some words into 2821bis if there was agreement on those words.

Many codes are "not currently defined" in 2821.  2821 and its
predecessors are pretty clear about what to do about them: the
rule is "use the first digit if you don't recognize the specific
code".  That leaves us
   6yz, 7yz, 8yz, 9yz, and 0yz
which are clearly and unambiguously "not defined".  What is a
client to do if it receives one of them?  Ignore it?  Does the
client assume that, because 6 > 5 and the code system tends
toward "worse" as the first digits get higher, that  650 is even
more permanently fatal than 550 and that 750 is more fatal than
that?  One could, but I suggest that it is a real stretch.
Absent extensions, does one assume that it is harmless because
it is lower-numbered than 2yz and that it should be ignored?  Or
does one assume that a 0yz code arises from some sort of
wrapping and so is "worse" or "more fatal and permanent" than
9yz would be, whatever that means?  Whatever the answers, it is
clear to me that 2821 (and 821, and FTP) offer no guidance at
all about what to do with one of these undefined codes.

The strawman proposed text would be a provision that a client
SHOULD (or perhaps even MUST) send a RSET or QUIT in response to
a code whose first digit is undefined by either 2821bis or a
negotiated extension.

That brings us back to 1yz.  I suggest that there are two
possible interpretations of it and that logic doesn't permit
anything else.  It is either an undefined code, in which case
the same analysis that applies to 0yz, 6yz...9yz above applies
also to it.  Or it is, because of the language in 821 (and
carried forward) a defined code  with a definition of "client
must now issue another command to tell the server whether to
continue or abort".   There are no cases in 821 or 2821 that
justify an interpretation of "required to ignore this defined
code" and nothing there, at least that I can find, that makes
ignoring an undefined code a safe thing to do.

      john

Date: Mon, 23 Apr 2007 08:42:05 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> Well, since you have no real mail system design or production
> experience, I think I will just skip replying to you.

Please check http://www.roaringpenguin.com.

I am the principal developer of that product.

Just because I haven't personally coded up a production SMTP client doesn't
mean I have no "real mail system design or production experience".  We have
our software running on hundreds of client systems, some of which process
many millions of messages/day.

> Its ashame that
> vote counts has any weight at all in this critical design issue.  You
> are simply not the right person to comment on the matter.

Comments like that are simply unbecoming of this list.

--
David.

Date: Mon, 23 Apr 2007 07:44:39 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Tony Hansen wrote:
> In order to help put this issue to bed, I'd like to get more input from
> actual SMTP deployments.

> For example, we've heard from one of the Exim developers that it uses
> the first code returned. And sendmail's & Hector's code reportedly uses
> the last.

> So I decided to check out some of the accessible open source code bases,
> focusing on the server code that *sends* email to the next MTA.

Tony,

I appreciate you taking the time to explore this. Btw, Hector's code is 
called Wildcat! SMTP (wcSMTP)

Are you looking for CONTINUATION or NON-CONTINUATION lines?

> I verified that sendmail definitely uses the last code and the Exim code
> definitely uses the *first* code of the response set.

Already confirm with sendmail last week.

But with Exim, per specification Exim definitely uses the first 
*non-continuation* reply code. Below is the loop in the function 
smtp_read_response() with loop break based on the first non-dash line 
encountered. The behavior is also documented as such in the comment block:

for (;;)
   {
   if ((count = read_response_line(inblock, ptr, size, timeout)) < 0)
     return FALSE;

     HDEBUG(D_transport|D_acl|D_v)
     debug_printf("  %s %s\n",
            (ptr == buffer)? "SMTP<<" : "      ", ptr);

   /* Check the format of the response: it must start with three digits;
      if these are followed by a space or end of line, the response is
-->  complete. If they are followed by '-' this is a multi-line response
-->  and we must look for another line until the final line is reached. 

      The only use made of multi-line responses is to pass them back as
      error messages. We therefore just concatenate them all within the
      buffer, which should be large enough to accept any reasonable
      number of lines. */

   if (count < 3 ||
      !isdigit(ptr[0]) ||
      !isdigit(ptr[1]) ||
      !isdigit(ptr[2]) ||
      (ptr[3] != '-' && ptr[3] != ' ' && ptr[3] != 0))
     {
     errno = ERRNO_SMTPFORMAT;    /* format error */
     return FALSE;
     }

   /* If the line we have just read is a terminal line, line, we
      are done. Otherwise more data has to be read. */

   if (ptr[3] != '-') break;  <--------- BREAK OUT OF LOOP

   /* Move the reading pointer upwards in the buffer and
      insert \n between the components of a multiline response.
      Space is left for this by read_response_line(). */

   ptr += count;
   *ptr++ = '\n';
   size -= count + 1;
   }

> AT&T's own MTA's code uses the last code, ignoring the code on all
> previous continuation lines.

> Smail uses the *first* code.

Looking for source. Do you have a URL?

> hMailServer appears to use the *first* code.

Per specification, it looks for the first NON-CONTINUATION reply code, 
however, this code is buggy and shouldn't be included in your test.  It 
only looks for hard coded "250" values and it only supports 1 extra 
multiple line from what I can see.  This would be the least of its 
problems.  Throw this one out.

> qmail uses the *first* code.

Yes it does - but it should noted it is doesn't care for continuation 
lines which is obvious broken code and n violation of the formal 
reply-code[sp] specification.

unsigned long smtpcode()
{
   unsigned char ch;
   unsigned long code;

   if (!stralloc_copys(&smtptext,"")) temp_nomem();

   get(&ch); code = ch - '0';
   get(&ch); code = code * 10 + (ch - '0');
   get(&ch); code = code * 10 + (ch - '0');
   for (;;) {
     get(&ch);
     if (ch != '-') break;   <---------- BREAK ON NON-DASH
     while (ch != '\n') get(&ch);
     get(&ch);
     get(&ch);
     get(&ch);
   }
   while (ch != '\n') get(&ch);

   return code;  <--- return first code -- BROKEN CODE!
}

It simply isn't following specs.  Do you wish to encourage this behavior?

> postfix appears to use the *last* code.
> jsmtp uses the *last* code.
> 
> Courier generally uses the *first* code. In some cases it actually
> treats 1yz, 2yz and 3yz as equivalent and will totally miss any
> subsequent 4yz/5yz.

What about the first non-continuation for its reply?

> The use of the first response code appears to be used just as often as
> the last response code. The codes in between seem to be universally ignored.

IMO, that is not a valid conclusion.

With the exception of one package (QMAIL) which is clearly violating 
821/2821 formal specifications, at least one of the codes (hMailServer) 
is very SMTP-buggy and is not worth any consideration.  You were 
incorrect with Exim and it isn't clear if analysis  s considering mixed 
non-continuation codes which is already conflictive with the 
specification.  You also didn't analyze a good bit of other popular main 
stream packages in the market, including none of the major MUAs which we 
have tested.

> It clearly indicates to me that at least the first and last reply codes
> need to be the same. If they are NOT the same, >>>CODE BREAKS<<<. This
> translates, to me, as indicating that we *NEED* to use "MUST" and not
> "SHOULD".

IMO, his is precisely the reason why it should be SHOULD and not a MUST.

A MUST encourages broken code behavior by falsely encouraging poor logic 
and pre 2821bis non-compliant logic that it is OK to read the first code 
because there is NEW presumption that the codes will be persistent.

By using a MUST you are saying that FUTURE clients should use a 
persistent reply code when in fact it is very conceivable that it might 
encounter a failure.

The solution is quite simple - follow the 25 year old spec of looking 
for the first non-continuation line.

Maybe Postels conservative/liberal axiom should apply:

     Servers SHOULD send persistent reply codes, however, clients
     SHOULD be aware for mixed reply codes scenarios and SHOULD
     only react to the first non-continuation reply code.

This is consistent with Postel's rule, it is consistent with past, 
present and future considerations, backward compatible and it is not a 
functional change on the specs. On the other hand, a MUST will attempt 
to enforce change. The MUST should not imply that CLIENT should begin 
enforcing persistent reply codes or just rely on the first line. It 
encourage incorrect designs of not ignoring continuation lines.

The point again is that it has been POSSIBLE for 25 years and a MUST 
would be changing that now and also encourage designs that may not be 
honored.

--
HLS

Date: Mon, 23 Apr 2007 07:17:10 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra  return-path headers in 4.4
Sender: [email-masked]

John,

My input, if I am following the issue right, is that this will mandate 
code change. Second, I never believe that SMTP software should alter 
headers or be responsible for cleaning it up.  I always felt this 
invites trouble - and it has and most likely will conflict with DKIM 
mail integrity issues if and when that protocol becomes widely adopted.

--
HLS

John C Klensin wrote:
> 
> 
> --On Monday, 23 April, 2007 03:34 +0200 Frank Ellermann
> <[email-masked]> wrote:
> 
>> Hector Santos wrote:
>>
>>> This changes CODE!!! My code!!
>> Pardon ?  Users of a server using your code certainly expect
>> that there is precisely one Return-Path, reflecting the MAIL
>> FROM as seen by the final delivery MTA.
>>
>> I'd bet that your code already does this, or do you keep crappy
>> Return-Paths inserted by prior hops in the header ?  What would
>> users do with that, where "user" can be anything from a stupid
>> vacation script and other auto-responders up to gateways.
>>
>>> What does this have to to with SMTP?
>> SMTP specifies who, when. what, and how for the Return-Path.
>> In RFC 2822 you also find that there's at most one
>> Return-Path, not more.
>> ...
> 
> Frank, Hector,
> 
> FWIW, let me give a different view of this issue, a view that is
> consistent with 2821.  For the purposes of this note, I'll
> ignore both whether it is consistent with the current state of
> 2821bis or with various perceptions of today's changing reality.
> 
> Along the transport path, multiple Return-path headers may
> happen and may happen regardless of whatever a standard says.
> They may happen because someone misbehaves.  They may happen
> because something thought it was a final delivery server and
> then ended up having to relay or gateway the mail because of
> something that happened downstream ... and did so at a point
> where the relevant code could not rescan the headers to be sure
> that the previously-inserted Return-path header was removed.
> 
> In any of those cases, a receiving MUA, or the delivery system
> immediately in front of it, has to be prepared to deal with a
> header collection that contains more than one Return-path and to
> figure out either
> 
> 	(i) how to ignore all but one of them or 
> 	
> 	(ii) to decide that a message with two (or more)
> 	Return-paths in it is such a heinous situation that the
> 	entire message should be immediately discarded without
> 	comment.  Even if one liked NDN messages, one can't
> 	return one in this case because making a choice of where
> 	to return it turns the case into (i).
> 
> I suggest to you that any real mail product that serves real
> customers who can choose other products will make the first
> choice --to figure out how to deliver the message if it can
> possibly be delivered -- because it is generally the only choice
> users will accept.   It may decide to deliver it with several
> demerit points for the excessive Return-path headers, or drop it
> in a "possibly junk" folder, but it will deliver it.    And, if
> the user decides to accept and process the thing, the MUA, or
> mail store, or delivery system will somehow figure out which
> Return-path header(s) to ignore.
> 
> Now, I think that is the reality, regardless of what any
> standard says.   The implications of that reality are the reason
> why 2821 says "you may take that out" rather than "you are
> really, really, evil if you don't take it out".  Even if the
> latter were there, it would not guarantee the receiver that
> multiple Return-paths would not show up.  And, if the receiver
> can't be guaranteed that, then it needs a coping strategy -- one
> that might fall anywhere on the spectrum from "simplistic" to
> "really clever heuristics".
> 
> In theory, one could assign "likelihood that a message is bogus"
> scores on the basis on whether the standard say MAY, SHOULD NOT,
> or MUST NOT.  But in practice, for this sort of situation (many
> others are different), that would be likely to be
> non-productive.  The dynamics of these situations change, the
> "badness" of an extra Return-path header varies with context
> (e.g., I'd be reluctant to get very excited about two of three
> Return-path headers with exactly the same content) and the life
> expectancy of a standard is just too long relative to changes in
> bad guy (or sloppy implementation) behavior to provide good
> guidance.
> 
> So while I'm happy to change the document in whatever way Tony
> says there is consensus, I am having a lot of trouble seeing
> what all the excitement is about.
> 
> regards,
>     john
> 
> 
> 
> 
> 
> 

Date: Mon, 23 Apr 2007 05:47:54 -0400
From: John C Klensin <[email-masked]>
To: "David F. Skoll" <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Sunday, 22 April, 2007 21:56 -0400 "David F. Skoll"
<[email-masked]> wrote:

>> Second, given the fact that 1yx are not currently defined for
>> current commands, it should be ignored.
> 
> Umm... no, I don't think so!
> 
> If you're writing a state machine (SMTP client) and it
> receives some unknown reply code from the server, the ONLY
> sane course of action is to QUIT or RSET.  The poor client has
> no idea what the server means; it could be expecting the
> client to alter its state in some way rendering subsequent
> commands useless or dangerous.

Let me suggest looking at this particular piece of the argument
a different way, because it is an area in which we could put
some words into 2821bis if there was agreement on those words.

Many codes are "not currently defined" in 2821.  2821 and its
predecessors are pretty clear about what to do about them: the
rule is "use the first digit if you don't recognize the specific
code".  That leaves us
   6yz, 7yz, 8yz, 9yz, and 0yz
which are clearly and unambiguously "not defined".  What is a
client to do if it receives one of them?  Ignore it?  Does the
client assume that, because 6 > 5 and the code system tends
toward "worse" as the first digits get higher, that  650 is even
more permanently fatal than 550 and that 750 is more fatal than
that?  One could, but I suggest that it is a real stretch.
Absent extensions, does one assume that it is harmless because
it is lower-numbered than 2yz and that it should be ignored?  Or
does one assume that a 0yz code arises from some sort of
wrapping and so is "worse" or "more fatal and permanent" than
9yz would be, whatever that means?  Whatever the answers, it is
clear to me that 2821 (and 821, and FTP) offer no guidance at
all about what to do with one of these undefined codes.

The strawman proposed text would be a provision that a client
SHOULD (or perhaps even MUST) send a RSET or QUIT in response to
a code whose first digit is undefined by either 2821bis or a
negotiated extension.

That brings us back to 1yz.  I suggest that there are two
possible interpretations of it and that logic doesn't permit
anything else.  It is either an undefined code, in which case
the same analysis that applies to 0yz, 6yz...9yz above applies
also to it.  Or it is, because of the language in 821 (and
carried forward) a defined code  with a definition of "client
must now issue another command to tell the server whether to
continue or abort".   There are no cases in 821 or 2821 that
justify an interpretation of "required to ignore this defined
code" and nothing there, at least that I can find, that makes
ignoring an undefined code a safe thing to do.

      john

Date: Mon, 23 Apr 2007 05:29:02 -0400
From: John C Klensin <[email-masked]>
To: "Peter J. Holzer" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Monday, 23 April, 2007 01:09 +0200 "Peter J. Holzer"
<[email-masked]> wrote:

> On 2007-04-22 16:46:06 -0400, Hector Santos wrote:
>> Just look at how it has been interpretation at:
>> 
>>    http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-
>>    2.htm
>> 
>> You can't just throw that out into the waste basket.
> 
> Why not? There are tons of bad books out there which belong
> into the waste basket. To be fair, a simple mixup of sender
> and recipient in a case which can only happen if the server
> uses an extension not defined in any RFC is probably easy to
> miss (I had to read the sentence several times to see what was
> wrong with it and I was specifically looking for an error).

I've written the author in the hope that he will clarify where
this came from but, having now read the several pages
surrounding that one and his FTP description, it is pretty clear
to me what happened.  With FTP, those 1yz codes definitely and
unambiguously mean that another, final/definitive, value is
coming and one doesn't need to do anything else --such as issue
a command-- to get it.   He assumes that the logic of the first
digit of the codes is identical in FTP and SMTP and says so.  

The problem is that they are not.   Even though most recent
implementations haven't recognized it, much less taken advantage
of it, FTP is asynchronous.   SMTP is not but, instead, is
modeled on one command - one response.  I am just guessing, but
I think it was exactly that difference that caused Jon Postel to
write the text that is causing all of this trouble.  For FTP, a
1yz response means "client should expect another response",
which is what that author says for both cases.  For SMTP, it
can't have that meaning because it would break the "one command
- one response" model.  So he wrote that a 1yz code would
require that the client issue a command --presumably to continue
or abort-- before the server could continue and send back
another response.  

Note that none of this has anything to do with multiline
responses of any flavor: in the FTP model, one issues a
non-continuation response with a 1yz code to convey those states
and, in SMTP, the 821 text doesn't say anything about
continuations, codes that can be ignored, etc.

If multiline replies with heterogeneous codes were permitted,
and permitted as a back-door keepalive mechanism, than one could
as easily have
   C:  RCPT TO:<[email-masked]>
   S:  250- looks ok, but stand by...
   S:  250- looks ok, but stand by...
   S:  250- looks ok, but stand by...
   S:  550  Sorry, we tried but will no deliver to that address

as anything involving insertion of a 1yz code and giving it
special treatment.
 
>...
>> What are you basically implying that we have a problem with
>> clients not  supporting the 25 year specification
> 
> No, we are concerned that clients which are supporting the 25
> year old specification will be rendered non-compliant if your
> interpretation is formalized.

Peter, while I agree, let's be a little careful.  Assume that
one reasonable person can read the text as saying both "yes, you
can invent 1yz codes and, despite the words about needing to
issue another command, treat them simply as a continuation
indicator" and "only the last code counts" and base an
implementation on it.  Assume that another reasonable person can
read the same text and conclude that there is no authorization
to use 1yz codes that way and no way to get from "In many cases"
to "in a continuation sequence, no reply line but the last one
counts" and, in addition, concludes that the codes must all be
the same, and base an implementation on that.  

Then we have a problem. The non-solution is to leave the text
exactly as it is, i.e., preserve the ambiguity that some would
claim has been there for a few months short of 25 years.
Either of the solutions -- to clearly support Hector's
interpretation or to clearly support yours -- results in some
implementations being declared invalid.  I don't see how to
avoid that.  

I also don't see that there is any difference between a SHOULD
NOT and a MUST NOT applied to sending out either mixed-code
multiline responses or to 1yz codes.  In either case, the
SMTP-sender needs to be prepared to cope and, if the multiline
response contains mixed codes, it will, probabilistically, screw
up.   Does it make any difference whether the problem is caused
by violation of a SHOULD NOT or a MUST NOT?  I think it does iff
the client is going to inspect all of the codes and, if they are
mixed, make a decision as to whether that is such a horrible
situation that it should immediately send a RSET or QUIT and
stop talking to that particular nasty, non-conforming,
ill-mannered, server _and_ it would do that if we said MUST NOT,
but would figure out how to cope and continue if we said SHOULD
NOT.  My experience with real systems in the field (as distinct
from what might happen in laboratories or with demonstration
systems) is that they don't get to behave that way: "nah, nah,
I'm not going to communicate with you until you clean up your
act" doesn't go over well with users who are trying to get mail
through unless the bad behavior is really so serious that there
is no reasonable way around it.

That turns my strongest argument for using "MUST NOT" into one
of not being able to identify the case that would justify the
exception that "SHOULD NOT" implies.  If, as Tony's recent note
points out, we can identify a significant number of real-world
situations in which some systems pick "first" and others pick
"last", then we invite client confusion if there are _any_
exceptions to a "no mixed codes" rule.  That says to me that
this must be a "MUST NOT" unless either 

	(i) we are prepared to identify the case(s) that would
	justify that confusion or
	
	(ii) we are prepared to put in text that tells a client
	that it _might_ receive mixed codes in spite of the
	SHOULD NOT prohibition and how it is expected to deal
	with that case.

While the second might not be a bad idea in practice, it would
require changing _everyone's_ code that picks either "first" or
"last" rather than inspecting all codes today.  And I haven't
seen any instances of the first if the measure is "having
confusion and  therefore having clients behave unpredictably is
ok".

      john

Date: Mon, 23 Apr 2007 05:06:41 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra  return-path         headers in 4.4
Sender: [email-masked]

--On Monday, 23 April, 2007 03:34 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> Hector Santos wrote:
> 
>> This changes CODE!!! My code!!
> 
> Pardon ?  Users of a server using your code certainly expect
> that there is precisely one Return-Path, reflecting the MAIL
> FROM as seen by the final delivery MTA.
> 
> I'd bet that your code already does this, or do you keep crappy
> Return-Paths inserted by prior hops in the header ?  What would
> users do with that, where "user" can be anything from a stupid
> vacation script and other auto-responders up to gateways.
> 
>> What does this have to to with SMTP?
> 
> SMTP specifies who, when. what, and how for the Return-Path.
> In RFC 2822 you also find that there's at most one
> Return-Path, not more.
>...

Frank, Hector,

FWIW, let me give a different view of this issue, a view that is
consistent with 2821.  For the purposes of this note, I'll
ignore both whether it is consistent with the current state of
2821bis or with various perceptions of today's changing reality.

Along the transport path, multiple Return-path headers may
happen and may happen regardless of whatever a standard says.
They may happen because someone misbehaves.  They may happen
because something thought it was a final delivery server and
then ended up having to relay or gateway the mail because of
something that happened downstream ... and did so at a point
where the relevant code could not rescan the headers to be sure
that the previously-inserted Return-path header was removed.

In any of those cases, a receiving MUA, or the delivery system
immediately in front of it, has to be prepared to deal with a
header collection that contains more than one Return-path and to
figure out either

	(i) how to ignore all but one of them or 
	
	(ii) to decide that a message with two (or more)
	Return-paths in it is such a heinous situation that the
	entire message should be immediately discarded without
	comment.  Even if one liked NDN messages, one can't
	return one in this case because making a choice of where
	to return it turns the case into (i).

I suggest to you that any real mail product that serves real
customers who can choose other products will make the first
choice --to figure out how to deliver the message if it can
possibly be delivered -- because it is generally the only choice
users will accept.   It may decide to deliver it with several
demerit points for the excessive Return-path headers, or drop it
in a "possibly junk" folder, but it will deliver it.    And, if
the user decides to accept and process the thing, the MUA, or
mail store, or delivery system will somehow figure out which
Return-path header(s) to ignore.

Now, I think that is the reality, regardless of what any
standard says.   The implications of that reality are the reason
why 2821 says "you may take that out" rather than "you are
really, really, evil if you don't take it out".  Even if the
latter were there, it would not guarantee the receiver that
multiple Return-paths would not show up.  And, if the receiver
can't be guaranteed that, then it needs a coping strategy -- one
that might fall anywhere on the spectrum from "simplistic" to
"really clever heuristics".

In theory, one could assign "likelihood that a message is bogus"
scores on the basis on whether the standard say MAY, SHOULD NOT,
or MUST NOT.  But in practice, for this sort of situation (many
others are different), that would be likely to be
non-productive.  The dynamics of these situations change, the
"badness" of an extra Return-path header varies with context
(e.g., I'd be reluctant to get very excited about two of three
Return-path headers with exactly the same content) and the life
expectancy of a standard is just too long relative to changes in
bad guy (or sloppy implementation) behavior to provide good
guidance.

So while I'm happy to change the document in whatever way Tony
says there is consensus, I am having a lot of trouble seeing
what all the excitement is about.

regards,
    john

Date: Mon, 23 Apr 2007 01:43:13 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

In order to help put this issue to bed, I'd like to get more input from
actual SMTP deployments.

For example, we've heard from one of the Exim developers that it uses
the first code returned. And sendmail's & Hector's code reportedly uses
the last.

So I decided to check out some of the accessible open source code bases,
focusing on the server code that *sends* email to the next MTA.

I verified that sendmail definitely uses the last code and the Exim code
definitely uses the *first* code of the response set.

AT&T's own MTA's code uses the last code, ignoring the code on all
previous continuation lines.

Smail uses the *first* code.

hMailServer appears to use the *first* code.

qmail uses the *first* code.

postfix appears to use the *last* code.

jsmtp uses the *last* code.

Courier generally uses the *first* code. In some cases it actually
treats 1yz, 2yz and 3yz as equivalent and will totally miss any
subsequent 4yz/5yz.

The use of the first response code appears to be used just as often as
the last response code. The codes in between seem to be universally ignored.

Other than ours and sendmail, I can't say much about how much all of
these code bases are used, but some of them are certainly used with
multiple millions of messages daily, and I think the sample is large
enough to draw some conclusions:

It clearly indicates to me that at least the first and last reply codes
need to be the same. If they are NOT the same, >>>CODE BREAKS<<<. This
translates, to me, as indicating that we *NEED* to use "MUST" and not
"SHOULD".

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> The ISSUE I have with all this what I think is being inconsistently
> applied here for this issue and Multiple response lines, is that for
> some ODD ball reason we don't think locking it down will break code
> because a few don't think it its being used the enough "superstars" and
> "common systems" when in fact it was always possible in the 25 year
> history of SMTP, following a formal specifications (see below), there
> was NEVER really a concern that it will break a system if you did use
> it.  The concern was simply never there following SMTP allowed to do
> this without any minor details that it will break a system.
> 
> But instead of at least making these two issues a SHOULD, it is
> incomprehensibly made a MUST - a lock down - which essentially does
> three things:
> 
> 1) It now makes any system that is using this in a published
>    and unpublished way, NON-COMPLIANT, including "superstars."
> 
> 2) If any new SMTP developer followed 2821bis or the new standard
>    the  would come from this work, inevitably, they will come across
>    a system where they will have a problem reading 1yx response lines.
>    The end result is that they need to CHANGE their code to support
>    these "legacy" systems,
> 
> 3) It literally makes it a SHOW STOPPER for current and future work in
>    this area - OPES is now completely dead - won't have any chance of
>    maximizing adoptability.
> 
> So never mind about changing a 25 year old written functionality, from
> an engineering and practical stand point, it makes absolutely NO sense
> to lock down these items simply on the basis that SOME believe it isn't
> being used - it isn't simply true.

Date: Mon, 23 Apr 2007 00:29:04 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra  return-path headers in 4.4
Sender: [email-masked]

Frank Ellermann wrote:
> Hector Santos wrote:
> 
>> This changes CODE!!! My code!!
> 
> Pardon ?  Users of a server using your code certainly expect that
> there is precisely one Return-Path, reflecting the MAIL FROM as
> seen by the final delivery MTA.

No.  Users historically and generally did not have that expectation, and 
  no MUA, including our own mail readers has that expectation.

Return-Path: is a network level consideration.

Your consideration now has to do with the growing need to supply 
off-line RFC based systems a greater say in EMAIL authentication concepts.

This is not a SMTP issue and should remain out of scope.  If a SMTP 
system wants to implement logic to keep the final MTA return-path, that 
is an implementation specific consideration.

Since it is NOT common practice to maintain the return-path for MDAs 
(for local recipient) your call for this change will CHANGE code.

> I'd bet that your code already does this, or do you keep crappy
> Return-Paths inserted by prior hops in the header ?  

No, and its none of my business what prior routing transactions took 
place.  I still adhere to the old age principle of "Thou shall not 
tamper with mail."

 > What would
> users do with that, where "user" can be anything from a stupid
> vacation script and other auto-responders up to gateways.

Thats outside the scope of SMTP. What you are basically implying is that 
all SMTP systems must implement 3834.  Thats changing code.

>> What does this have to to with SMTP?
> 
> SMTP specifies who, when. what, and how for the Return-Path.  In
> RFC 2822 you also find that there's at most one Return-Path, not
> more.

That still does not say I have to store one, nor does it say I have to 
massage the mail to clean it up.  Thats changing code.

>> If RFC 3834 requires it, thats fine. But you don't need 3834
>> to create bounces.
> 
> It's needed to create other auto-responses:
> 
> | If the response is to be generated after delivery, and there is no
> | Return-Path field in the subject message, there is an implementation
> | or configuration error in the SMTP server that delivered the message
> | or gatewayed the message outside of SMTP.  A Personal or Group
> | responder SHOULD NOT deliver a response to any address other than
> | that in the Return-Path field, even if the Return-Path field is
> | missing.  It is better to fix the problem with the mail delivery
> | system than to rely on heuristics to guess the appropriate
> | destination of the response.  Such heuristics have been known to
> | cause problems in the past.

That's all fine Frank, but you don't need 3834 to accomplish auto-responses.

But lets be clear, its a non-delivery auto-response A.K.A bounce, the 
return-path is required independently of 3834.

3834 is more about a STYLE of the message - not really what is already a 
requirement regarding the Return-Path.

> Are we trying to nail that the MAIL FROM = reverse path = envelope
> sender = return path = "bounces to" <shudder />  is "the real thing"
> everywhere, or do we need to discuss this emulation known as "PRA" ?

Well, we can get lost in that debate.  But what really interest me, is 
this, why is this being considered when this OBVIOUSLY it is something 
that changes functionality and hence code?

I'll tell you what, you give your VOTE for the reply code issues and 
I'll give you my vote for return-paths in local final destination? :-)

Jokes to the side, this changes code.  I think the compromise in what we 
both want is this:

    - its a valid format,

    - its verifiable, and

    - its ideally verifiable at the time it is presented, not
      in some belated future date and time and space.

Its a chicken and egg thing Frank. If you can't solidify the strong 
requirement for a valid return path, then any of the other EMAIL 
authentication considerations become a moot point.

Any HELO/EHLO domain validation idea is based on the idea that it is 
valid at the moment it is provided.

Similarly,  CBV and SPF is based on the idea that the return-path 
address and domain respectively, are valid at the moment it is provided.

Any MAIL content based technology will suffer in adoption if its 
MANDATED to be performed at the POST SMTP level only.  If applicable, it 
is already a proven concept that it best to do this at the DATA level.

But all this is really implementation specific and should be out of scope.

The only thing developers can hope for is that SMTP provides the 
endorsed framework that is already possible for new advances to continue.

--
HLS

Date: Sun, 22 Apr 2007 22:37:52 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Well, since you have no real mail system design or production 
experience, I think I will just skip replying to you.  Its ashame that 
vote counts has any weight at all in this critical design issue.  You 
are simply not the right person to comment on the matter.

--
HLS

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> The key difference in this text was the 1yz was literately interpreted
>> as a "PRELIMINARY code" not a COMPLETION code, which literally means
>> there more codes are pending.
> 
> The interpretation is wrong, as a reading of the RFC will show.
> 
>> Second, given the fact that 1yx are not currently defined for current
>> commands, it should be ignored.
> 
> Umm... no, I don't think so!
> 
> If you're writing a state machine (SMTP client) and it receives some
> unknown reply code from the server, the ONLY sane course of action is
> to QUIT or RSET.  The poor client has no idea what the server means;
> it could be expecting the client to alter its state in some way rendering
> subsequent commands useless or dangerous.
> 
>> As one of the developers in the community and market place, it has been
>> very clear.  As formally defined in section 4.3 "SMTP Replies" for the
>> reply code:
>>
>>    Formally, a reply is defined to be the sequence: a
>>    three-digit code, <SP>, one line of text, and <CRLF>, or a multiline
>>    reply (as defined in the same section).
> 
> RFC2821, sec. 4.2.1 (my emphasis):
> 
>    The format for multiline replies requires that every line, except the
>    last, begin with *THE* reply code, followed immediately by a hyphen,
>    "-" (also known as minus), followed by text.  The last line will
>    begin with *THE* reply code, followed immediately by <SP>, optionally
>    some text, and <CRLF>.
> 
> Note: all lines must begin with *THE* reply code.  Not "a" reply code, but
> *THE* one-and-only reply code.
> 
> Therefore, although your 1xy- server happens to work and to "solve"
> an engineering problem you're having, I contend that it violates
> RFC 2821.
> 
>> Coupled with the logical suggestive text for handling multiple lines -
>> look for the first "xyz[sp]" non-continuation line, I don't see how you
>> can screw that up and practice has shown it hasn't been an issue. Why
>> would a serious SMTP system go against this?  There is no reason why it
>> would.
> 
> I can see very good reasons why a serious SMTP client would *not* extend
> its timeout upon partial receipt of a reply code.  Since this is the only
> use case you've offered for the 1xy- hack, I think it's a bad idea.
> 
> --
> David.
> 
> 
> 

Date: Sun, 22 Apr 2007 21:56:42 -0400
From: "David F. Skoll" <[email-masked]>
To: Hector Santos <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> The key difference in this text was the 1yz was literately interpreted
> as a "PRELIMINARY code" not a COMPLETION code, which literally means
> there more codes are pending.

The interpretation is wrong, as a reading of the RFC will show.

> Second, given the fact that 1yx are not currently defined for current
> commands, it should be ignored.

Umm... no, I don't think so!

If you're writing a state machine (SMTP client) and it receives some
unknown reply code from the server, the ONLY sane course of action is
to QUIT or RSET.  The poor client has no idea what the server means;
it could be expecting the client to alter its state in some way rendering
subsequent commands useless or dangerous.

> As one of the developers in the community and market place, it has been
> very clear.  As formally defined in section 4.3 "SMTP Replies" for the
> reply code:
> 
>    Formally, a reply is defined to be the sequence: a
>    three-digit code, <SP>, one line of text, and <CRLF>, or a multiline
>    reply (as defined in the same section).

RFC2821, sec. 4.2.1 (my emphasis):

   The format for multiline replies requires that every line, except the
   last, begin with *THE* reply code, followed immediately by a hyphen,
   "-" (also known as minus), followed by text.  The last line will
   begin with *THE* reply code, followed immediately by <SP>, optionally
   some text, and <CRLF>.

Note: all lines must begin with *THE* reply code.  Not "a" reply code, but
*THE* one-and-only reply code.

Therefore, although your 1xy- server happens to work and to "solve"
an engineering problem you're having, I contend that it violates
RFC 2821.

> Coupled with the logical suggestive text for handling multiple lines -
> look for the first "xyz[sp]" non-continuation line, I don't see how you
> can screw that up and practice has shown it hasn't been an issue. Why
> would a serious SMTP system go against this?  There is no reason why it
> would.

I can see very good reasons why a serious SMTP client would *not* extend
its timeout upon partial receipt of a reply code.  Since this is the only
use case you've offered for the 1xy- hack, I think it's a bad idea.

--
David.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 22: Requiring removal of extra  return-path headers in 4.4
Date:  Mon, 23 Apr 2007 03:34:44 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 41
Sender: [email-masked]

Hector Santos wrote:

> This changes CODE!!! My code!!

Pardon ?  Users of a server using your code certainly expect that
there is precisely one Return-Path, reflecting the MAIL FROM as
seen by the final delivery MTA.

I'd bet that your code already does this, or do you keep crappy
Return-Paths inserted by prior hops in the header ?  What would
users do with that, where "user" can be anything from a stupid
vacation script and other auto-responders up to gateways.

> What does this have to to with SMTP?

SMTP specifies who, when. what, and how for the Return-Path.  In
RFC 2822 you also find that there's at most one Return-Path, not
more.

> If RFC 3834 requires it, thats fine. But you don't need 3834
> to create bounces.

It's needed to create other auto-responses:

| If the response is to be generated after delivery, and there is no
| Return-Path field in the subject message, there is an implementation
| or configuration error in the SMTP server that delivered the message
| or gatewayed the message outside of SMTP.  A Personal or Group
| responder SHOULD NOT deliver a response to any address other than
| that in the Return-Path field, even if the Return-Path field is
| missing.  It is better to fix the problem with the mail delivery
| system than to rely on heuristics to guess the appropriate
| destination of the response.  Such heuristics have been known to
| cause problems in the past.

Are we trying to nail that the MAIL FROM = reverse path = envelope
sender = return path = "bounces to" <shudder />  is "the real thing"
everywhere, or do we need to discuss this emulation known as "PRA" ?

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date:  Mon, 23 Apr 2007 03:06:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Peter J. Holzer wrote:

> To be fair, a simple mixup of sender and recipient in a case which
> can only happen if the server uses an extension not defined in any
> RFC is probably easy to miss (I had to read the sentence several
> times to see what was wrong with it and I was specifically looking
> for an error).

The RFC 821 text is:

| The sender-SMTP should send another command specifying whether to
| continue or abort the action.

Hector's source says:

| The SMTP sender should expect another reply before a new command
| may be sent.

Not only a simple mixup apparently.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 25: Conditions for non-delivery notification messages
Date:  Mon, 23 Apr 2007 02:15:40 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 122
Sender: [email-masked]

John C Klensin wrote:

> I thought we had closed that issue out

I thought you had just assigned issue 21 to 0b.  I think it's
really important, it's THE issue, but...  

> I'd like to assign an issue number

...I don't see why it should get a third issue number.

> My impression is that the general issue of prohibiting sending
> out an NDN message when it cannot be definitively rejected at
> SMTP envelope time has been ruled out of scope.

This is not THE issue.  Nobody *here* supports to *prohibit* all
NDNs.  If MTAs are confident that they know how to send an NDN
to the originator when necessary, or if they're confident that 
they don't have to send a NDN, they can simply accept the mail.

THE issue is that they must be really confident, because they
MUST send an NDN to the *originator*.  And no weasel words about
"as indicated in the reverse path" - in another article you just
proposed to move the old routing cruft into appendix F.2 with a
clear "DON'T DO THIS".

> examples have been given in which it is impossible to make
> the decisions while the server still has the connection
> open from the client

It's always possible to decide "accept or reject", if nothing
helps they could use a random number generator.  Or your "odd
IPs only on Tuesdays" recipe.  But in a standard readers would
look for some better guidance, and the standard must not only
offer "as indicated in the reverse path" - this is would be an
"821bis" design.  It also must not directly (or between the 
lines) say that "accept" is always a good decision, because 
sending an NDN later "as indicated by the reverse path" is no
problem.  It is only no problem for a hypothetical "821bis" or
its SPF PASS reincarnation.

In other words it is a serious problem for most mails today,
and 2821bis cannot pretend that "reverse paths" still work as
designed in RFC 821, or hide THE issue behind smoke screens.

> To take just one example

JFTR, I post such examples often on the spamcop list, I think
their FAQ is too simple.  Of course most "misdirected bounces"
are still clearly avoidable, and then it's IMO okay to punish
such "bouncers" as hard as possible.  The "bouncers" will then
(try to) quote RFC 2821, claiming that they MUST bounce, and
that RFC 2821 nowhere admits that its model is broken when the
reverse path has nothing to do with the originator.  As it's
the case for most mails today.

> Requiring that servers of either type have current knowledge
> of deliverable addresses is technically burdensome

Yes, and ?  Who should pay for such misdirected bounces, the
victims ?  Just move the crap into some poor end user mailbox
where they can POP it with V.90 doesn't fly.  Sooner or later
they'd be forced to reject or delete all bounces.  So by NOT 
trying to discourage this behaviour you'd get precisely what
you don't want, an SMTP with "de facto" prohibited bounces.

All participants have to share their part of this burden.  Not
only the receivers, as the spamcop FAQ suggests, but certainly
they are a key.  And for that 2821bis is important, because
receivers (like everybody else) are lazy and greedy, they will
say "MUST bounce" as long as RFC 2821 can be abused to justify
their misbehaviour.

The backup MX in question could temporarily block all mails
where the mail from + source IP is in any way suspicious.  It
will cause some backscatter, but it can limit this damage to
some degree.  Nobody *here* (as opposed to the spamcop list)
is expecting miracles.  Just some minimum of common sense and
good will based on a realistic description that the originator
from the POV of this backup MX is in most cases NOT indicated
in the reverse path.

> We have, historically, tried to encourage such intermediate
> MX-associated servers as a means of making the Internet 
> accessible to more people and keeping the mail system as 
> robust as possible.

Historically the spam ratio wasn't 80%.  And historically 
forging reverse paths was a taboo, I recall a mail abuse FAQ
published in 2002 saying that this is a "social" problem, and
that such spammers are scam.  At that time I couldn't nail it,
but I doubted that it's a _social_ problem, it was completely
against what I thought a reverse path is, _technically_

In 2003 I read an article about Hadmut Danisch's RMX draft, 
and suddenly it was all clear (minus one bad day when Tony 
told me that RFC 1123 is no nonsense, but a part of STD 3):

This is a BUG.  "They" - likely some "they" including you -
screwed up royally when "they" wrote 1123, and somebody has
to fix it.  IMHO 2821bis also has to share its part of this
collective burden.  

And that is how I interpret issue 0b or 21 or 26, THE issue.

> the consequences of going to a "never bounce" model are
> fairly large

Absolutely nobody with an SPF PASS or FAIL policy could wish
that, bounces are essential for its 551 emulation (i.e. when
receivers forward their mail to a third party without noting
this fact in the reverse path, and when this third party 
then rejects the unmodified reverse path resulting in FAIL)

> I don't think it is in scope for the 2821bis effort.

It's completely out of scope of anything remotely related to
SMTP, "never bounce" is as bad as "accept on probation", or
maybe worse.

Frank

Date: Sun, 22 Apr 2007 20:11:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Peter J. Holzer wrote:

>> You can't just throw that out into the waste basket.
> 
> Why not? There are tons of bad books out there which belong into the
> waste basket. To be fair, a simple mixup of sender and recipient in a
> case which can only happen if the server uses an extension not defined
> in any RFC is probably easy to miss (I had to read the sentence several
> times to see what was wrong with it and I was specifically looking for
> an error).

The key difference in this text was the 1yz was literately interpreted 
as a "PRELIMINARY code" not a COMPLETION code, which literally means 
there more codes are pending.

Second, given the fact that 1yx are not currently defined for current 
commands, it should be ignored.  That is something that isn't 
specifically stated but implied.   The 821/2821 specs simply "implies" 
that there is NO expectation for these codes to exist in any of the 
current commands and may be useful in an extended environment.

But in either interpretation, what do you do when your code does 
encounter it?

well you do what MOST systems will mostly like do - ignore it and when 
it coupled with a "-", it justifies it when coupled with the formal 
definition in section 4.3 "SMTP Replies" for "xyz[sp]"

>> The point is that it has possible and always been possible and by far 
>> clients have evolved to followed the 25 year old formal declaration of 
>> using only a "xyz[sp]" concept to the command response and ignoring 
>> continuation reply codes in lines with "xyz-".
> 
> Except that this has not been formally defined for 25 years. That is
> your interpretation of an (admittedly not perfectly clear) 25 year old
> specification.

As one of the developers in the community and market place, it has been 
very clear.  As formally defined in section 4.3 "SMTP Replies" for the 
reply code:

    Formally, a reply is defined to be the sequence: a
    three-digit code, <SP>, one line of text, and <CRLF>, or a multiline
    reply (as defined in the same section).

Coupled with the logical suggestive text for handling multiple lines - 
look for the first "xyz[sp]" non-continuation line, I don't see how you 
can screw that up and practice has shown it hasn't been an issue. Why 
would a serious SMTP system go against this?  There is no reason why it 
would.

>> What are you basically implying that we have a problem with clients not 
>> supporting the 25 year specification
> 
> No, we are concerned that clients which are supporting the 25 year old
> specification will be rendered non-compliant if your interpretation is
> formalized.

But my interpretation matches what is already written and already 
existed in practice. I'm not altering functionally as this new lock down 
text threatens to do with a greater possibility.

Can you show one system that will not use the first non-continuation 
reply line encountered?  How do you know there aren't systems out there 
that is using this?

What you are agreeing with has a greater chance of breaking new 
development when they do encounter it. It might even ABORT the 
transaction because it may enforce persistent reply code concept.

The chances the interoperability issues is less when you provide 
clarified insight that this is already possible and to emphasize that 
what is more important is the formal ABNF:

       Reply-line = Reply-code [ SP text ] CRLF

and not:

       Reply-line = Reply-code "-" [text ] CRLF

Which is what you are supporting that we still accommodate?  Why even 
bother with this?

  C: command
  S: 250-blah
  S: 250 done

when the following is sufficient in your view I presume?

  C: command
  S: 250-blah
  S: 250-done

All I am asking here is to clarify that is already existed - the formal 
reply code does not have a DASH in character #4, and the first 
non-continuation is the reply code per specification.

At the very least, the text should use "SHOULD" as a persistent reply 
code suggestion, not a MUST.

--
HLS

Date: Sun, 22 Apr 2007 20:02:31 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> Even after all the screaming that the SMTP server is now broken for what
> would was perfectly legal for 25 years but now is insanely made illegal
> in 2821bis, I'm willing to bet that you STILL write your CLIENT code, if
> not already, to accept this:

>    150-
>    150-
>    250

I've never written a real SMTP client (just some throwaway test code),
so maybe you're right.  Nevertheless, I don't believe that mixed reply
codes were *ever* legal.  The RFCs weren't 100% explicit, but I bet a
poll of random RFC readers would yield a huge majority who believe the
above behaviour is disallowed by the RFCs.

Since the whole point of this exercise is to clarify the RFCs, not to
change them in a major way, it makes sense to spell out what pretty
much everyone interprets to be the case.

Also, as I've stated before, the only use case you've presented
(to try to trick a client into extending its timeout) is a hackish "fix"
for a problem that should be solved, not patched over.

--
David.

Date: Sun, 22 Apr 2007 19:10:57 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 26: Source routes, especially	 reverse-paths
Sender: [email-masked]

--On Monday, 23 April, 2007 00:20 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> John C Klensin wrote:
>  
>> (2) Decide that source routes are ancient history and remove
>> Appendix C entirely, moving all discussion of source routes
>> into the present F.2, and shifting the rules for forward-
>> pointing source routes from "MAY ignore" (and use the Mailbox
>> information and MX records only) to "SHOULD ignore".
> 
> Good direction.  My proposal was "outsource all cruft to a
> separate document", but "put anything in F.2" is also fine.

And "put some bootstrap minimum in F.2 and then reference 821 or
2821" is pretty close to what I had in mind if we do that.  In
some sense, the "reference" part is more or less equivalent to
your "outsource" idea, but without the pain of writing an extra
doc.

> But don't remove the A-d-l syntax for paths, I think you want
> "MUST NOT generate, MUST ignore", but still also "MUST accept"
> for this cruft.  Why only "SHOULD ignore" ?  It can't work as
> designed in RFC 821 if it's generally not generated.  RFC 1123
> even went so far to say that it never worked as designed... :-(

Well, I can't remember whether Craig Partridge or I wrote that
particular text (there are other possibilities, but we are the
most likely), but that probably wasn't the intent.  The problem
--and sort of the last straw in the useful life of source
routes-- was that, if one had
   <@a,@b,@c:x@d>
821 said that the message should be routed to host "a" but there
were some systems that got it backwards and tried to route to
"c" first and a few that, following the same practices that got
UUCP routes into trouble, would say "aha, I know who 'b' is and
have a better path to it than through 'a', so I'll just go there
directly".  Combine those three variations with the requirement
that the reverse-path be constructed as an incremental set or
reverse source routings and, well, it would work as designed but
there was no way to predict just what would happen when a
message was sent. 
  
>> Reintroducing source routes and requiring support for them
>> would, of course, constitute adding functionality
> 
> Folks would know where to find RFC 4408, in a certain sense it
> reintroduces the lost reverse routes.  RFC 4408 also claims
> that source routes are "archaic", my appeal about this insult
> was rejected... ;-)

<gag choke/>

>> The current working version of -03 incorporates a version of
>> option (1) above.
> 
> I'd like option (2) better.  I could quote the "reverse path"
> design as it was in 821 when necessary no matter which option
> you pick in 2821bis - it's a philosophical thing about what
> "responsibility" means for forwarders to third parties, and
> what it means for MSAs accepting a MAIL FROM (4409 6.1).  And
> why the name "return-path" was NOT a stupid error that should
> have been "bounces-to".

I did that text before concluding that it might be better to
just pull it all out.  Let's see what others have to say -- I'm
certainly not attached to leaving it there.

     john

Date: Mon, 23 Apr 2007 01:09:16 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--neYutvxvOLaeuPCA
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-22 16:46:06 -0400, Hector Santos wrote:
> Just look at how it has been interpretation at:
>=20
>    http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-2.htm
>=20
> You can't just throw that out into the waste basket.

Why not? There are tons of bad books out there which belong into the
waste basket. To be fair, a simple mixup of sender and recipient in a
case which can only happen if the server uses an extension not defined
in any RFC is probably easy to miss (I had to read the sentence several
times to see what was wrong with it and I was specifically looking for
an error).

> The point is that it has possible and always been possible and by far=20
> clients have evolved to followed the 25 year old formal declaration of=20
> using only a "xyz[sp]" concept to the command response and ignoring=20
> continuation reply codes in lines with "xyz-".

Except that this has not been formally defined for 25 years. That is
your interpretation of an (admittedly not perfectly clear) 25 year old
specification.

> What are you basically implying that we have a problem with clients not=
=20
> supporting the 25 year specification

No, we are concerned that clients which are supporting the 25 year old
specification will be rendered non-compliant if your interpretation is
formalized.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--neYutvxvOLaeuPCA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--neYutvxvOLaeuPCA--

Date: Sun, 22 Apr 2007 19:02:25 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-02 Issue 25: Conditions for non-delivery notification messages
Sender: [email-masked]

John,

I'm not sure if this touches base with how this issue is stated or 
model, and maybe you will, as Dorothy Parker would of said, "throw it 
out with full force", I would to mention (again) my two non-functional 
change clarifications wish items:

1) Recognize the importance and the growing industry direction for
    SMTP level validation considerations,

2) Recognize the possible need to make a statement or clarification
    that the Return path (MAIL FROM:) must be valid at the MOMENT it
    is presented, not at some BELATED moment in time.  How
    that  validation is done is out of scope, but a emphasis of
    "validation vs time" should be clarified.

I hope you can understand #2 especially, since it is LOOPHOLE into the 
"must be valid" requirement and how BOUNCES are done or not done or 
discarded.

--
HLS

John C Klensin wrote:
> Hi.
> 
> A recent note from Frank raises the issue again about
> prohibitions on non-delivery notifications ("bouncing"
> messages), at least to unauthenticated sites.  I thought we had
> closed that issue out but, since it has come up again, I'd like
> to assign an issue number and see if we can have a clear and
> quick discussion.
> 
> For context, that note mentions...
> 
>> Unrelated, for some extremely important text see also
>> http://www.spamcop.net/fom-serve/cache/329.html#bounces
> 
> Some of the key sentences in that note are:
> 
>> Upgrade and/or configure your mail server software so that
>> this situation is never encountered. Configure your software
>> to either reject messages during delivery or accept them
>> permanently. Do not let your software make choices about
>> delivery after it has accepted a message.
> 
> My impression is that the general issue of prohibiting sending
> out an NDN message when it cannot be definitively rejected at
> SMTP envelope time has been ruled out of scope.  I also think
> that several examples have been given in which it is impossible
> to make the decisions while the server still has the connection
> open from the client, at least without significant changes to
> the SMTP model, some interesting extensions, or both.  
> 
> To take just one example, such a rule would require that a
> "backup MX" --a server configured to pick up mail for a
> destination site iff that site is temporarily broken or off-net)
> to have a definitive list of deliverable addresses on the target
> system and domain.   Similar issues arise with gateway servers
> for areas with poor online connectivity to the Internet
> backbone, such as many developing countries or at least rural
> areas in them.   Requiring that servers of either type have
> current knowledge of deliverable addresses is technically
> burdensome (especially since we have no protocols for securely
> exchanging such lists) and raises a number of security and
> privacy issues.   We have, historically, tried to encourage such
> intermediate MX-associated servers as a means of making the
> Internet accessible to more people and keeping the mail system
> as robust as possible.  And we have historically had an email
> model in which senders can reasonably expect that a message will
> either be delivered or an NDN will come back -- that "delivery
> notification" messages --which raise their own set of privacy
> concerns-- are not necessary.  
> 
> Thinking about these issues have even caused some people to wish
> that we had not deprecated the practice of relays constructing
> reverse-paths by adding their own identifiers to them --
> something that has not been a regular practice on the Internet
> since the mid- 1980s.  I can only suggest to them that this
> would not solve the problem: return paths are as easy to spoof
> as Received headers.  But more on that in my next note (issue
> 26).
> 
> The net result is that the consequences of going to a "never
> bounce" model are fairly large: it is not just a matter of
> saying "don't do that any more" despite the language in the
> quoted text and at the link above.  Partially because of the
> operational model changes needed to really get it right, I don't
> think it is in scope for the 2821bis effort.  But we need to
> settle that one way or the other and get on with our lives.
> 
>     john
> 
> 
> 

Date: Sun, 22 Apr 2007 18:51:40 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-02 Issue 21: Re: Section 6.2 - favoring         reject over bounce
Sender: [email-masked]

John C Klensin wrote:
> --On Sunday, 22 April, 2007 21:13 +0200 Frank Ellermann wrote:
>> Can we defer this until the ABNF is in good shape ?
> 
> Up to Tony.  My personal instinct is to get everything else
> fixed and then go back and clean up the ABNF.   But I'm flexible
> on that subject.

6 of one....

So far, the nits raised about the ABNF mostly haven't required changes
in the text. Nor have the textual changes been requiring changes in the
ABNF.

A serious overhaul of the ABNF would require verifying that things match
not only the text, but the previous pseudo-ABNF. And where there are
still issues involved with the text, I think it would behoove us to take
care of the issues that would affect the ABNF before tackling the ABNF.
I think tackling the ABNF as has been proposed would be worth doing;
however, in many senses it's really "low hanging fruit". My preference
would be to get the issues taken care of before cleaning up the ABNF.

	Tony

Date: Sun, 22 Apr 2007 18:49:10 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra	 return-path headers in 4.4
Sender: [email-masked]

Frank Ellermann wrote:
 > John C Klensin wrote:
 >
 >> Hi.  Issue 22 assigned to this.
 >
 > Any procedure resulting in precisely one Return-Path created by
 > the final delivery MTA is IMO fine.  Can you please smuggle in
 > an informative reference to RFC 3834 to stress this point ?

Not sure I follow, and you are now upsetting me.  This changes CODE!!! 
My code!!  What does this have to to with SMTP?  Are you asking that we 
have to now massage and clean up the headers?

Traditionally, the only time we add the Return-Path in the header at 
reception and for passthru mail (not local) and its not required once 
the delivery status has been determined and since we do a dynamic SMTP 
validation, it isn't necessary beyond that point.

If RFC 3834 requires it, thats fine. But you don't need 3834 to create 
bounces.

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Registration rules for extensions
Date:  Mon, 23 Apr 2007 00:43:10 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

John C Klensin wrote:
 
> not a 2821bis problem unless we want to try redefine the
> registry rules as part of 2821bis.  If anyone really wants
> to go down that path, supply draft text and I'll assign an
> issue number.   But I'd strongly recommend that this work
> be done separately if at all.

You already have BCP 90 for trace header fields, and some
"RFC required" variant for further via and with subclauses.
You also have "standardization" for any IPvFuture "tags".  

The first registry with the extensions requires standards
track or experimental, that's IMO good enough, the old VERB
and ONEX extension is apparently a grandfathered exception.

> if you ask me what I "want", we will be back to "RFC number"

Then chapter 8 "IANA considerations" is ready.  And anybody
is free to write a summary of one page in the bat book, and
submit this as an "experimental" RFC updating the registry.

Frank

Date: Sun, 22 Apr 2007 18:27:49 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:

 >> The CODE is broken if it reads a continuation line as
 >> the response code.

 > It's obvious to me that using DIFFERENT reply codes as part of the
 > SAME reply is "broken".

Even after all the screaming that the SMTP server is now broken for what 
would was perfectly legal for 25 years but now is insanely made illegal 
in 2821bis, I'm willing to bet that you STILL write your CLIENT code, if 
not already, to accept this:

    150-
    150-
    250

If you say you will not adjust your software to honor this, and choose 
instead to drop connections and then pass the buck to your operators 
and/or SMTP server vendor then good luck! <g>

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 26: Source routes, especially reverse-paths
Date:  Mon, 23 Apr 2007 00:20:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 38
Sender: [email-masked]

John C Klensin wrote:
 
> (2) Decide that source routes are ancient history and remove
> Appendix C entirely, moving all discussion of source routes
> into the present F.2, and shifting the rules for forward-
> pointing source routes from "MAY ignore" (and use the Mailbox
> information and MX records only) to "SHOULD ignore".

Good direction.  My proposal was "outsource all cruft to a
separate document", but "put anything in F.2" is also fine.

But don't remove the A-d-l syntax for paths, I think you want
"MUST NOT generate, MUST ignore", but still also "MUST accept"
for this cruft.  Why only "SHOULD ignore" ?  It can't work as
designed in RFC 821 if it's generally not generated.  RFC 1123
even went so far to say that it never worked as designed... :-(
 
> Reintroducing source routes and requiring support for them
> would, of course, constitute adding functionality

Folks would know where to find RFC 4408, in a certain sense it
reintroduces the lost reverse routes.  RFC 4408 also claims
that source routes are "archaic", my appeal about this insult
was rejected... ;-)

> The current working version of -03 incorporates a version of
> option (1) above.

I'd like option (2) better.  I could quote the "reverse path"
design as it was in 821 when necessary no matter which option
you pick in 2821bis - it's a philosophical thing about what
"responsibility" means for forwarders to third parties, and
what it means for MSAs accepting a MAIL FROM (4409 6.1).  And
why the name "return-path" was NOT a stupid error that should
have been "bounces-to".

Frank

Date: Sun, 22 Apr 2007 17:56:58 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
cc: [email-masked]
Subject: Re: RFC2821bis-02 Issue 22: Requiring removal of extra return-path headers in 4.4
Sender: [email-masked]

--On Sunday, 22 April, 2007 21:08 +0200 Frank Ellermann
<[email-masked]> wrote:

> John C Klensin wrote:
>  
>> Hi.  Issue 22 assigned to this.
 
> Any procedure resulting in precisely one Return-Path created by
> the final delivery MTA is IMO fine.  Can you please smuggle in
> an informative reference to RFC 3834 to stress this point ?

Hmm.  I'm not completely sure I see the applicability, so don't
know quite how to make such a reference.  I could certainly
indicate that automatic responses, such as those described in
3834, are dependent on having one, and only one, Return-path by
the time the message is delivered.  Would that do the job?

Keith, I don't know if you have been following the 2821bis
discussion on the ietf-smtp list, but I could use your advice on
this -- drop me a note off-list if you would like a quick
summary of the issue(s).

     john

Date: Sun, 22 Apr 2007 22:47:09 +0100
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: RFC2821bis-02 Issue 26: Source routes, especially reverse-paths
Sender: [email-masked]

On Sun, 22 Apr 2007, John C Klensin wrote:

> (2) Decide that source routes are ancient history and remove
> Appendix C entirely, moving all discussion of source routes into
> the present F.2, and shifting the rules for forward-pointing
> source routes from "MAY ignore" (and use the Mailbox information
> and MX records only) to "SHOULD ignore".

+1

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTHEAST FITZROY: NORTHERLY 4 OR 5 BECOMING VARIABLE 3 OR 4. SLIGHT OR
MODERATE. FOG PATCHES. GOOD, OCCASIONALLY VERY POOR.

Date: Sun, 22 Apr 2007 17:42:01 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 21: Re: Section 6.2 - favoring   reject over bounce
Sender: [email-masked]

--On Sunday, 22 April, 2007 21:13 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> John C Klensin wrote:
> 
>> Issue 21 assigned.
> 
> Fine, that could obsolete issue 0b and anything else related
> to <http://www.spamcop.net/fom-serve/cache/329.html#bounces>
> 
> I think this issue is not limited to section 6.2, it is THE
> issue. 

Whether it is "THE issue" or not, my reason for assigning issue
25 was to address the general topic.  As you say above, it isn't
just 6.2 -- there are issues wrt Security Considerations and
arguably the entire mail model buried in it.

> Can we defer this until the ABNF is in good shape ?

Up to Tony.  My personal instinct is to get everything else
fixed and then go back and clean up the ABNF.   But I'm flexible
on that subject.

     john

Date: Sun, 22 Apr 2007 17:40:59 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> The CODE is broken if it reads a continuation line as the response code.

Haven't we already settled this?

The RFC says that in many cases, a client can just take the response.
code from the last line.  This leaves open the possibility for the
client to use a different strategy.

It's obvious to me that using DIFFERENT reply codes as part of the
SAME reply is "broken".

--
David.

Date: Sun, 22 Apr 2007 14:28:38 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and "command"
Sender: [email-masked]

Hi,
At 08:05 22-04-2007, John C Klensin wrote:
>It has been pointed out that Section 2.3 contains a definition
>for "Reply" (in 2.3.11) but no parallel definition for
>"Command".  Is this worth fixing?

How about:

Commands

A SMTP command is sent from sender to receiver via the transmission channel.
SMTP commands and, unless altered by a service extension, message 
data, are transmitted in "lines".

2.3.7.  Lines

Lines consist of zero or more data  characters terminated by the 
sequence ASCII character ...

2.3.8.  Reply

The Reply paragraph (Section 2.3.11) is moved to after the Lines 
paragraph.  Maybe the "
Message Content and Mail Data" paragraph may be moved as well so that 
it follows the "Lines" paragraph.

Regards,
-sm 

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and	 "command"
Date:  Sun, 22 Apr 2007 23:31:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

John C Klensin wrote:

> Text specified below has been (very tentatively) incorporated
> into -03 with some small modifications.  Comments and additional
> suggestions welcome.

Looking again at it, maybe "message" would be less artificial than
"mail object".  Hector upset me again enough to check what RFC 821
said.  For commands it says in 4.1.1:

  The SMTP commands define the mail transfer or the mail system
  function requested by the user.  SMTP commands are character
  strings terminated by <CRLF>.  The command codes themselves are
  alphabetic characters terminated by <SP> if parameters follow
  and <CRLF> otherwise.

No "verb" in 821, you could take this text replacing "codes" by
"verbs".  Of course you already have almost literally the same
text in exaxctly the same section 4.1.1 in 2821bis-02, and maybe
the idea to repeat it near 2.3.11 is stupid.  

How about extending 2.3.11 to cover commands _and_ replies ?  And
to introduce "verb", otherwise this exercise would be a waste of
time.  Really "alphabetic" everywhere, if somebody uses digits or
hyphens they couldn't claim that it's 25 years old... (so far for
my "UTF8" verb proposed in EAI :-) 

Frank

Date: Sun, 22 Apr 2007 17:24:16 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Nitpicking typos (was: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

--On Sunday, 22 April, 2007 22:23 +0200 Frank Ellermann
<[email-masked]> wrote:

>...
> Nitpicking typos:
> 
> - 1yz  Positive Preliminary reply The command has been
> + 1yz  Positive Preliminary reply:  The command has been
> - 2yz  Positive Completion reply The requested action has
> + 2yz  Positive Completion reply:  The requested action has
> - 3yz  Positive Intermediate reply The command has been
> + 3yz  Positive Intermediate reply:  The command has been
> - 4yz  Transient Negative Completion reply The command was
> + 4yz  Transient Negative Completion reply:  The command was
> - 5yz  Permanent Negative Completion reply The command was
> + 5yz  Permanent Negative Completion reply:  The command was
> 
> Yeah, that's stupid, the RFC editor could fix it anyway, but
> I think that 2821bis should be a show case.  As you said 
> nobody will ever touch it again. 
>...

There is an interesting history as to why this got to be the way
it is in -02.   I have successfully changed the formatting to
that used in 2821 (and 821).   If people don't like that, I can
insert colons... or we can decide that getting to this level of
detail indicates that we are finished :-)

    john

Date: Sun, 22 Apr 2007 17:12:18 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 26: Source routes, especially reverse-paths
Sender: [email-masked]

Hi.

Source routes were implicitly deprecated from Internet mail with
RFC 975 at the beginning of 1986 and explicitly deprecated with
RFC 1123 in 1989.  The DRUMS work concluded that SMTP senders
should be prohibited from sending them out but that SMTP
receivers should be (i) required to accept the syntax and (ii)
either to follow the source routing rules for any routes they
accepted or to ignore the routing specifications and use the
mailbox address only.

These decisions were reinforced by knowledge of the nightmares
that explicit mail routing had caused in the UUCP environment,
where such routing persisted much longer than it did in the
Internet/ SMTP one.

To accomplish that goal, a slightly modified version of the
source routing rules from RFC 821 was incorporated into Appendix
C of 2821 and section F.2 of Appendix F was introduced to
deprecate the things.

It obviously was not sufficient, since we still have people
quoting text that specifies, e.g., that relays incorporate their
own domain names into reverse-paths, a feature that, as far as I
know, has not been actively used by anyone with SMTP for more
than 20 years (I suspect that there may still be pockets out
there, but they certainly aren't high-frequency).

To prevent further confusion, I propose that we do one of the
following:

(1) Modify Appendix C to clearly prohibit SMTP-senders from
inserting their own domain names into the return-path, thereby
construction source routes in violation of the existing
prohibition on clients sending out any such thing.  This is not
a change in functionality, just changing some appendix text that
can be read as contradicting some text in the body of the
specification.

This option requires some small tidying up elsewhere, but the
Appendix C changes are the core of the work.

(2) Decide that source routes are ancient history and remove
Appendix C entirely, moving all discussion of source routes into
the present F.2, and shifting the rules for forward-pointing
source routes from "MAY ignore" (and use the Mailbox information
and MX records only) to "SHOULD ignore".

Reintroducing source routes and requiring support for them
would, of course, constitute adding functionality, so that
option, which would otherwise be the third case above, is a
non-starter.

The current working version of -03 incorporates a version of
option (1) above.

     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 24: ABNF overhaul
Date:  Sun, 22 Apr 2007 22:59:57 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

John C Klensin wrote:

> We need to make a decision as to where to draw the line on this,
> if at all.

I want something that I can stuff into Bill's validator resulting
in a PASS.  It's not really difficult for 2821bis, all non-trivial
constructs are already in a form where that's possible (I didn't
check the results for -01 and 02 yet, but I did for -00 in 2005).

> Pro: Clearly, a complete ABNF cleanup, to make the set closed
> and consistent and, ideally, to eliminate and formalize as many
> of the comments as possible would be a nice thing.

Yes.

> Con: The odds of being able to do this in a reasonable amount
> of time and to do so without introducing new errors --either
> within 2821bis or between 2821bis and 2822[bis?] are, based on
> prior experience, not high.

NAK, it would take me at most a day, it's straight forward once
folks know what they want with special cases like TLDs, trailing
dots, IDs, or multiple paths in for-clauses.

It's also a good chance to find out what they really want, _that_
might take some time, but it's not the fault of an ABNF cleanup.

> my guess is that it represents an order of magnitude change in
> the amount of work we otherwise still need to get done

Adding the missing 'verb = "VERB" [etc]' to any 'VERB [etc]' with
a list command = verb / otherverb / moreverbs / [...] is really
trivial.  I've done that for the -00 check, and the simultaneous
2821 vs. 2821bis-00 comparison.  (I stumbled over the old files
yesterday in my "projects/spf" folder where I also have 4409bis
and "dkim" incl. Hector's drafts :-)

With that you could also create a "collected ABNF".  For a long
text making it even longer if that helps to read it is IMO okay.

Frank

Date: Sun, 22 Apr 2007 16:46:06 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Kari Hurtta <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

John C Klensin wrote:

> If they are not _all_ true, then we end up with either a
> procedural or a technical difficulty with this conversation and
> maybe with the document.  Partially because it seems so obvious
> to me that these uses of 0yz or 1yz codes should be supported by
> an extension to be sure that the client and server are in
> agreement about how to interpret what is being done, I continue
> to believe that we have a communications problem here rather
> than a substantive one, but I may be wrong and may not
> understand what is being proposed and why.

The problem as I see is that you are being uncharacteristically 
inconsistent in how you applying SHOULD or MUST for this issue and the 
persistent reply code issue.   You are using a premise there are NO 
systems in 25 years of possible operations or in your eyes not enough 
important systems are actually implementing, supporting or using  this 
in some form or fashion.  That is an incorrect presumption and basis to 
lock down these items.

The VERB issue is not the same. Here a CLIENT is compliant because it 
issued this VERBosity command. Whether its was a formal extension or 
proprietary, the CLIENT is in full control here and knows how to handle 
the this "05x " response line.

Now, the question here as I see it whether a SERVER has the formal 
specification blessing to issue "1xy " or "xyz-" as part of a multiple 
line response to clients.

The answer is a RESOUNDING yes and the many years of fine tuning, time 
tested, evolving, published, or unpublished behavior supporting this 
behavior beg the question as to why we would enforce a spec change to a 
MUST for both of these issues when there is a possibility that it made 
alter, perhaps uncommon, yet perfectly SMTP acceptable and supported 
behavior?

Common practice or not, you are basically saying

     "Well, I don't know anyone who is using this, so I guess we
      better close this loophole before it becomes a problem."

The point is is that it hasn't been a PROBLEM in practice because 
everyone is following the specification.

Again, you simply do not know whether its use in practice or not and to 
what levels because NO ONE has ever complain it was causing problems.

That is very uncharacteristic of how I have come to learn the IETF 
process which I have had depended on for common sense engineering.

This one completely goes against all common sense.  It shouldn't be a 
matter of playing the game of semantics but to clarify what is the 
actual possible practice evolved over 25 years.

Just look at how it has been interpretation at:

    http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-2.htm

You can't just throw that out into the waste basket.  It is important 
consideration on how the industry has evolved.  Whether it is common 
practice is insignificant.  What is common practice is that all clients 
handle it appropriately and if they do not, they are not following the 
formal and suggestive written specs.  It just isn't an issue.

> Application of the robustness principle as the usual
> disambiguator of such things says that the server ought to send
> homogeneous codes because that eliminates any sensitivity to
> "first", "middle", "last" or "all".  

Thats fine, and its good material when you are starting from scratch and 
it is also good advice for a very conservative and simple SMTP server to 
apply in its transactions.   But for that matter, using your 
conservative robustness principle, if it was really concern for the 
server, then it SHOULD NOT use Multiple Lines.  The 2821bis suggestion 
would be "avoid them."

The point is that it has possible and always been possible and by far 
clients have evolved to followed the 25 year old formal declaration of 
using only a "xyz[sp]" concept to the command response and ignoring 
continuation reply codes in lines with "xyz-".

What are you basically implying that we have a problem with clients not 
supporting the 25 year specification so therefore we better LOCK it down 
to a MUST (persistent reply codes) and a  MUST NOT use "1yz-" and thats 
simply not correct.

--
HLS

Date: Sun, 22 Apr 2007 16:46:02 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 25: Conditions for non-delivery notification messages
Sender: [email-masked]

Hi.

A recent note from Frank raises the issue again about
prohibitions on non-delivery notifications ("bouncing"
messages), at least to unauthenticated sites.  I thought we had
closed that issue out but, since it has come up again, I'd like
to assign an issue number and see if we can have a clear and
quick discussion.

For context, that note mentions...

> Unrelated, for some extremely important text see also
> http://www.spamcop.net/fom-serve/cache/329.html#bounces

Some of the key sentences in that note are:

> Upgrade and/or configure your mail server software so that
> this situation is never encountered. Configure your software
> to either reject messages during delivery or accept them
> permanently. Do not let your software make choices about
> delivery after it has accepted a message.

My impression is that the general issue of prohibiting sending
out an NDN message when it cannot be definitively rejected at
SMTP envelope time has been ruled out of scope.  I also think
that several examples have been given in which it is impossible
to make the decisions while the server still has the connection
open from the client, at least without significant changes to
the SMTP model, some interesting extensions, or both.  

To take just one example, such a rule would require that a
"backup MX" --a server configured to pick up mail for a
destination site iff that site is temporarily broken or off-net)
to have a definitive list of deliverable addresses on the target
system and domain.   Similar issues arise with gateway servers
for areas with poor online connectivity to the Internet
backbone, such as many developing countries or at least rural
areas in them.   Requiring that servers of either type have
current knowledge of deliverable addresses is technically
burdensome (especially since we have no protocols for securely
exchanging such lists) and raises a number of security and
privacy issues.   We have, historically, tried to encourage such
intermediate MX-associated servers as a means of making the
Internet accessible to more people and keeping the mail system
as robust as possible.  And we have historically had an email
model in which senders can reasonably expect that a message will
either be delivered or an NDN will come back -- that "delivery
notification" messages --which raise their own set of privacy
concerns-- are not necessary.  

Thinking about these issues have even caused some people to wish
that we had not deprecated the practice of relays constructing
reverse-paths by adding their own identifiers to them --
something that has not been a regular practice on the Internet
since the mid- 1980s.  I can only suggest to them that this
would not solve the problem: return paths are as easy to spoof
as Received headers.  But more on that in my next note (issue
26).

The net result is that the consequences of going to a "never
bounce" model are fairly large: it is not just a matter of
saying "don't do that any more" despite the language in the
quoted text and at the link above.  Partially because of the
operational model changes needed to really get it right, I don't
think it is in scope for the 2821bis effort.  But we need to
settle that one way or the other and get on with our lives.

    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date:  Sun, 22 Apr 2007 22:23:54 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 66
Sender: [email-masked]

Hector Santos wrote:

> check out this *published book" with an entirely different
> (and very logical) interpretation for 1yz:

> http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-2.htm

Maybe it's only me, but I've never before heard of this book
and the author.  Aren't we all trying to figure out how stuff
works, or how it's supposed to work, where the answers to both
questions are at best related ?

Nothing's wrong with your concept as new concept, we could
of course still discuss if there are better solutions.  But
it MUST NOT be imposed on existing clients and servers in
the SMTP standard when it breaks existing implementations.

You'd have real difficulties to find somebody who's more 
conservative than me wrt backwards compatibility.  I often
say "real SMTP" when talking about RFC 821 reverse paths,
as opposed to the postmodern 1123 5.3.6(a) spam universe.

If you find a source written by Jon Postel I'm all ears.
Sendmail's VERB is apparently compatible with 2821bis-02.

Nitpicking typos:

- 1yz  Positive Preliminary reply The command has been
+ 1yz  Positive Preliminary reply:  The command has been
- 2yz  Positive Completion reply The requested action has
+ 2yz  Positive Completion reply:  The requested action has
- 3yz  Positive Intermediate reply The command has been
+ 3yz  Positive Intermediate reply:  The command has been
- 4yz  Transient Negative Completion reply The command was
+ 4yz  Transient Negative Completion reply:  The command was
- 5yz  Permanent Negative Completion reply The command was
+ 5yz  Permanent Negative Completion reply:  The command was

Yeah, that's stupid, the RFC editor could fix it anyway, but
I think that 2821bis should be a show case.  As you said 
nobody will ever touch it again.  It goes straight to STD 
if they get issue 21 right.  And after all that's possible,
for the "IPv6 only" case I'm less confident.

> If makes you wonder why the AUTHOR wrote this

Not really, anybody is free to write whatever they like, and
nobody is forced to read it.  Maybe the author copied a piece
of text from ftp, see:

http://www.tcpipguide.com/free/t_FTPRepliesReplyCodeFormatandImportantReplyCodes-2.htm

Another author wrote this, easy to guess what I believe:

| The command has been accepted, but the requested action
| is being held in abeyance, pending confirmation of the
| information in this reply.  The sender-SMTP should send
| another command specifying whether to continue or abort
| the action.
|
| [Note: SMTP does not have any commands that allow this
|  type of reply, and so does not have the continue or
|  abort commands.]

Frank

Date: Sun, 22 Apr 2007 16:04:54 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 24: ABNF overhaul
Sender: [email-masked]

--On Sunday, 22 April, 2007 20:54 +0200 Frank Ellermann
<[email-masked]> wrote:
 
>...
> Yes.  We need it to convert all now still pseudo-ABNF lines
> like  
>  MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>
> 
> into proper ABNF like
> 
>  mail    = "MAIL FROM:" <reverse-path> [SP <mail-parameters> ]
> <CRLF>
> 
> after an introduction in the direction of
> 
>  command = hello / mail / rcpt / data / rset / quit ; etc.

We need to make a decision as to where to draw the line on this,
if at all.

Pro: Clearly, a complete ABNF cleanup, to make the set closed
and consistent and, ideally, to eliminate and formalize as many
of the comments as possible would be a nice thing.

However,
Con: The odds of being able to do this in a reasonable amount of
time and to do so without introducing new errors --either within
2821bis or between 2821bis and 2822[bis?] are, based on prior
experience, not high.  It is already clear to many people that I
can't do it and get it right so this would probably require
freezing the document, handing the source off to someone else
and having him or her make the changes, and then having everyone
do an in-depth check.  That is more or less what was done the
last time (during DRUMS) except that the "in-depth check" part
obviously failed to meet people's criteria today.

It is not clear how many of the current idiosyncrasies --as
distinct from outright errors, which we are fixing-- are
impeding consistent and interoperable implementations.  My
guess, based on the empirical evidence, is "very, very, few". 

So I will leave it up to Tony and Lisa to figure out how to find
the right balance here, but my guess is that it represents an
order of magnitude change in the amount of work we otherwise
still need to get done to produce a document that can be
advanced.

     john

Date: Sun, 22 Apr 2007 15:54:19 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and	 "command"
Sender: [email-masked]

(I've reordered the comments a bit for clarity)

--On Sunday, 22 April, 2007 20:54 +0200 Frank Ellermann
<[email-masked]> wrote:
 
> John C Klensin wrote:
> 
>> Is this worth fixing?
> 
> Yes.  We need it to convert all now still pseudo-ABNF lines
> like  
>  MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>
>...

Separate problem.  See separate note.

> There are 302 occurences of "command" in 286 lines (my editor
> says), from that POV explaining what it is is kind of
> pointless.
> 
> But there are also 15 ocurences of "verb", that's less obvious.
> Please insert a new section above the "reply" part (now
> 2.3.11).

Seems sensible.  Text specified below has been (very
tentatively) incorporated into -03 with some small
modifications.  Comments and additional suggestions welcome.

>> a "yes" answer accompanied by suggested text will weigh more
>> heavily on the editor than a "yes" answer without text
>...

> <section title="Commands and verbs"><t>
>  After an initial greeting from the server, a special case of
>  a reply discussed later, clients send either commands or when
>  permitted a mail object.  The server sends a reply for each
>  command.  All commands begin with what's known as "verb", 
>  followed by additional arguments as needed for the command
>  in question.
> </t><t>
>  The allowed commands depend on the state of the conversation.
>  The allowed replies depend on the state and the command.
> <!-- insert historical telnet/ftp note about this style ? -->
> </t></section>

> Unrelated, for some extremely important text see also
> http://www.spamcop.net/fom-serve/cache/329.html#bounces

See separate note... coming soon.

    john

Date: Sun, 22 Apr 2007 15:48:49 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Peter J. Holzer wrote:

>> VERB
>> 250 2.0.0 Verbose mode
>> MAIL FROM:<hurtta@localhost>
>> 250 2.1.0 <hurtta@localhost>... Sender ok
>> RCPT TO:<root>
>> 050 <root>... aliased to hurtta
>> 250 2.1.5 <root>... Recipient ok
> 
> But note an important difference: There is no "-" after "050", so the
> RCPT command returns two results (050 and 250) in this case, not a
> single result with conflicting return codes. LMTP and draft-hall-prdr
> extend the data command in a similar way.

I agree.  No one is saying multiple continuation codes should be 
supported because it goes against the 25 year old written formal 
specificaiton of BREAKING on the first non-continuation line.

In this case, the client issued the VERBosity command so it is full 
control of the situation. So this scenario is not the same issue here, 
but it does show that these NON-STANDARD reply codes are used in practice.

But as an important and related side note, please read the intepretation 
that been published in a TCP/IP Reference BOOK with a interesting 
sub-title:

        "The TCP/IP reference even you can understand"

which technically supports the usage of non-continuation PRELIMINARY codes:

http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-2.htm

--
HLS

Date: Sun, 22 Apr 2007 15:21:30 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> The ISSUE I have with all this what I think is being inconsistently 
> applied here for this issue and Multiple response lines, is that for 
> some ODD ball reason we don't think locking it down will break code 
> because a few don't think it its being used enough by "superstars" and 
> "common systems" when in fact it was always possible in the 25 year 
> history of SMTP, following a formal specifications (see below), there 
> was NEVER really a concern that it will break a system if you did use 
> it.  The concern was simply never there following SMTP allowed to do 
> this without any minor details that it will break a system.
> 

Forgot to write the "see below" to highlight formal declaration in 4.2 
for what some say is "suggestive:"

    ....  Formally, a reply is defined to be the sequence: a
    three-digit code, <SP>, one line of text, and <CRLF>, or a multiline
    reply (as defined in section 4.2.1).  Since, in violation of this
    specification, the text is sometimes not sent, clients which do not
    receive it SHOULD be prepared to process the code alone (with or
    without a trailing space character).  Only the EHLO, EXPN, and HELP
    commands are expected to result in multiline replies in normal
    circumstances, however, multiline replies are allowed for any
    command.

    In ABNF, server responses are:

       Greeting = "220 " Domain [ SP text ] CRLF
       Reply-line = Reply-code [ SP text ] CRLF

If any old or current system reads a line:

       Reply-Code "-" [text] CRLF

and return this as a REPLY code is 1) broken and 2) in violation of a 25 
year old specification. It will simply come across a transaction where 
this is a problem for them.

The reality if that most if not all systems do not behave like this.  It 
will be RARE to find one that does.

Why are we supporting this behavior by using a MUST for persistent reply 
codes or disallowing "1yz-" in attempt to support these broken systems?

The CODE is broken if it reads a continuation line as the response code.

--
HLS

Date: Sun, 22 Apr 2007 15:20:55 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Registration rules for extensions (was: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Sender: [email-masked]

(subject line changed since this is not a 2821bis problem unless
we want to try redefine the registry rules as part of 2821bis.
If anyone really wants to go down that path, supply draft text
and I'll assign an issue number.   But I'd strongly recommend
that this work be done separately if at all.)

--On Sunday, 22 April, 2007 20:20 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> Kari Hurtta wrote:
> 
>> Perhaps off topic, but it seems that sendmail uses 0xx
>> with similar way if VERB command is given
> [...]
>> VERB
>> 250 2.0.0 Verbose mode
>> MAIL FROM:<hurtta@localhost>
>> 250 2.1.0 <hurtta@localhost>... Sender ok
>> RCPT TO:<root>
>> 050 <root>... aliased to hurtta
>> 250 2.1.5 <root>... Recipient ok
> 
>> Seems that http://www.iana.org/assignments/mail-parameters
>> lists VERB
> 
> Interesting, apparently they had some rather odd rules about
> registering extensions (Ned 1, Frank 0).  But otherwise it
> it is a clear "opt-in" extension, both sides are supposed to
> know what the effect of VERB is, no problem for 2821(bis).

Yes.  See earlier note.

> Is it already clear that "buy the bat book" is not what we
> want for registered extensions ?

Personal opinion: I would prefer to have an RFC reference for
every one of these.  However, to avoid conflicts, I'd rather
have an extension registered than not, so, if the choice is "no
registration" and "ask Eric", I prefer "ask Eric".  And, yes, I
would prefer "buy the bat book and look at pages ..." to "ask
Eric" or even "read the code" although, if you ask me what I
"want", we will be back to "RFC number".

      john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 21: Re: Section 6.2 - favoring reject over bounce
Date:  Sun, 22 Apr 2007 21:13:12 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

John C Klensin wrote:

> Issue 21 assigned.

Fine, that could obsolete issue 0b and anything else related
to <http://www.spamcop.net/fom-serve/cache/329.html#bounces>

I think this issue is not limited to section 6.2, it is THE
issue.  Can we defer this until the ABNF is in good shape ?

Frank

Date: Sun, 22 Apr 2007 21:14:38 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--x+6KMIRAuhnl3hBn
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-20 18:33:41 +0300, Kari Hurtta wrote:
> John C Klensin <[email-masked]> writes in gmane.ietf.smtp:
> > --On Tuesday, 10 April, 2007 15:23 -0400 Hector Santos
> > <[email-masked]> wrote:
> > > Second, there is no restriction in using it as part of a
> > > continuation line.  Example:
> > >=20
> > >      C: DATA
> > >      S: 352 Begin sending your data
> > >         [client uploads data]
> > >      S: 150-DKIM signature found!
> > >      S: 150-Please wait while we process your DKIM message!
> > >      S: 150-Waiting for trust server to respond...
> > >      S: 150-Still waiting for trust server to respond...
> > >      C: 550-Sorry, The DKIM POLICY has failed this transaction
> > >      C: 550 Please see http://trust-r-us.com/msgid=3D123213123
[...]
>=20
> Perhaps off topic, but it seems that sendmail uses 0xx
> with similar way if VERB command is given
>=20
> VERB
> 250 2.0.0 Verbose mode
> MAIL FROM:<hurtta@localhost>
> 250 2.1.0 <hurtta@localhost>... Sender ok
> RCPT TO:<root>
> 050 <root>... aliased to hurtta
> 250 2.1.5 <root>... Recipient ok

But note an important difference: There is no "-" after "050", so the
RCPT command returns two results (050 and 250) in this case, not a
single result with conflicting return codes. LMTP and draft-hall-prdr
extend the data command in a similar way.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--x+6KMIRAuhnl3hBn
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--x+6KMIRAuhnl3hBn--

Date: Sun, 22 Apr 2007 15:12:45 -0400
From: John C Klensin <[email-masked]>
To: Kari Hurtta <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

(address list trimmed, ietf-smtp list added to close the loop
with the apparently non-working gmane gateway)

--On Friday, 20 April, 2007 18:33 +0300 Kari Hurtta
<[email-masked]> wrote:

> 
> The following message is a courtesy copy of an article
> that has been posted to gmane.ietf.smtp as well.
> 
> John C Klensin <[email-masked]> writes in gmane.ietf.smtp:
> 
>> --On Tuesday, 10 April, 2007 15:23 -0400 Hector Santos
>> <[email-masked]> wrote:
> 
>> > Second, there is no restriction in using it as part of a
>> > continuation line.  Example:
>> > 
>> >      C: DATA
>> >      S: 352 Begin sending your data
>> >         [client uploads data]
>> >      S: 150-DKIM signature found!
>> >      S: 150-Please wait while we process your DKIM message!
>> >      S: 150-Waiting for trust server to respond...
>> >      S: 150-Still waiting for trust server to respond...
>> >      C: 550-Sorry, The DKIM POLICY has failed this
>> >      transaction C: 550 Please see
>> >      http://trust-r-us.com/msgid=123213123
>> > 
>> > Who knows?  Is this not possible?  It is out of the whelm of
>> > real possibility?
>> 
>> No, it is is clearly not impossible.  However, it raises two
>> issues in addition to the one about the codes all being the
>> same which, IMO (and in the text of 2821), arises from one of
>> them.  
>> 
> 
> Perhaps off topic, but it seems that sendmail uses 0xx
> with similar way if VERB command is given
> 
> EHLo xx
> 250-Hurtta06k.keh.iki.fi Hello localhost [127.0.0.1], pleased
> to meet you 250-ENHANCEDSTATUSCODES
> 250-PIPELINING
> 250-EXPN
> 250-VERB
> 250-8BITMIME
> 250-SIZE
> 250-DSN
> 250-ETRN
> 250-AUTH DIGEST-MD5 CRAM-MD5
> 250-DELIVERBY
> 250 HELP
> VERB
> 250 2.0.0 Verbose mode
> MAIL FROM:<hurtta@localhost>
> 250 2.1.0 <hurtta@localhost>... Sender ok
> RCPT TO:<root>
> 050 <root>... aliased to hurtta
> 250 2.1.5 <root>... Recipient ok
> 
> ( Seems that http://www.iana.org/assignments/mail-parameters
>   lists VERB )

Well, this is, IMO, precisely on-topic because it illustrates
something else.

Note that, in this example, VERB is explicitly advertised as an
available extension in response to the EHLO command, then
invoked by a client-issued command.  While there is no RFC
describing VERB, we can deduce that it (or one of the other
advertised extensions) explicitly authorizes several things that
are not permitted by 821/ 2821/ 2821bis-02 SMTP:

	(1) Use "hurtta@localhost" in the MAIL command and
	"root" in the RCPT command.  Both are prohibited by
	unextended SMTP unless this is a submission server
	because they do not contain FQDNs.  If it is a
	submission server communicating with a local mailbox
	(which I would guess is the case), then all bets are off
	-- none of the rules that apply to SMTP on the public
	Internet are required to apply here even though applying
	them might well be a good idea.
	
	(2) The use of at least some reply codes with first
	digits that are not given clear definitions in 821 or
	2821 is permitted.  Here, 0yz is used, but the extension
	could have authorized use of 1yz, or even 9yz or Ayz,
   for its own purposes.  
	
	(3) Since 0yz is not being used as part of a
	continuation, there are actually two separate replies
	being returned in response to the RCPT command, the
	second one asynchronously (without another command being
	issued first).  So the extension is either authorizing
	asynchronous replies or, more likely, some class of
	reply codes that are simply ignored, with the client
	continuing to read reply lines, without issuing a new
	command, until a standard reply code appears.  An
	extension could, of course, authorize either one.

The key in both cases (2) and (3) is that the server announces
that it is able to support the extension and the client
announces that it can support it and wants it. 

That makes this a good example of how an extension can be used
to permit three-digit reply codes that are not specified in 2821
(or in 821) and how it can assign semantics to those codes that
are not specified in the standards.  It also shows that
extensions of this type can be made to work and, indeed, that
they are fairly established practice (I'm aware of other
implementations of VERB, including that in one SMTP gateway I
worked on, but my memory doesn't extend to whether their
semantics are identical or interoperable with what seems to be
going on here).

It doesn't tell us a thing about the use of 0yz codes, 1yz
codes, or 9yz codes in the absence of any extension.
 
> 1xx was not selected for that usage.   So 050 is really used,
> but is 150 used on somewhere?

Doesn't make any difference, IMO.  What this shows is that 050
is used _with an extension_.  Clearly 150 could be used with an
extension, either this way (presumably as a code that is ignored
for command-buffering / turnaround purposes), or as a keep-alive
code in a multiline response, or otherwise.   But, in this
context, only if there were an extension and the extension
definition specified it that way.   Our disagreement, if we have
one, is about whether some or all of

	(i) the use of 1yz, without specific authorizing
	extensions, represents long-term common practice without
	a "continue" command by the client to reestablish the
	command-response pairing.
	
	(ii) the use of 1yz responses at all without specific
	authorizing extensions, represents common practice at
	all.
	
	(iii) multiline responses with different codes and
	without specific authorizing extensions, (either in
	general or if all of the codes are identical except for
	zero or more 150 ones) represent long-term common
	practice that does not cause SMTP-sender problems.

If they are not _all_ true, then we end up with either a
procedural or a technical difficulty with this conversation and
maybe with the document.  Partially because it seems so obvious
to me that these uses of 0yz or 1yz codes should be supported by
an extension to be sure that the client and server are in
agreement about how to interpret what is being done, I continue
to believe that we have a communications problem here rather
than a substantive one, but I may be wrong and may not
understand what is being proposed and why.

In any event, (i) is problematic even if there were long-term
common practice because the only text in either 821 or 2821 that
discusses those codes appears to attribute fairly specific
semantics to them to preserve the command-response pairing.
Their use without the semantics is, IMO clearly, additional or
different functionality to what 2821 (and 821) specify.
Procedurally, such functionality cannot be added at this point
without bouncing 2821bis back to "proposed".

If we could get past (i), (ii) would require that the fans of
1yz codes demonstrate that their use was interoperably and
consistently implemented in a way that would be consistent with
the existing 2821 text.  I haven't seen that claim made yet in a
way that would support an interoperability report for Draft
Standard.  The fact that there is no RFC describing exactly how
they are used, to which an implementation can claim conformance,
makes such an interoperability claim doubly difficult but not
impossible: if there really are independent server, and
independent client, implementations out there that do the right
thing, are consistent with a clear definition in _some_
specification, and don't cause problems for other
implementations would, I think, be enough to get clarifying
words added to 2821bis.   But, again, the issue of (i) would
have to be satisfied first, i.e., it would also need to be
demonstrated that this was existing functionality in (or at
least clearly consistent with) 2821 and not an innovate wrt that
document, no matter how broadly deployed.

The third issue has been discussed at length but, to identify a
personal perspective, language in 2821 (and 821) that says that 
"in many cases" it is sufficient to look at the last code,
followed by a sentence that reads "The client will be able to
identify
these cases" (referring to the reply text) "from the current
context." neither proves nor disproves the "all codes must be
the same" theory.   It is clearly sloppy work (mea culpa as far
as 2821 is concerned), both because the "in many cases"
terminology doesn't give a clue about how to identify the
_other_ cases (which presumably exist) but also because that
third sentence doesn't give a clue about how the client is to
figure out how to identify applicable cases in which the text
may be important enough that some of the lines can be ignored.   

Application of the robustness principle as the usual
disambiguator of such things says that the server ought to send
homogeneous codes because that eliminates any sensitivity to
"first", "middle", "last" or "all".  It may be that the receiver
side of the robustness principle should cause us to suggest that
the client better be ready to handle a mixture of codes but, if
we do that, I think we need to either figure out what "in many
cases" actually means or specify a rule about which one is
significant.    If we make that "last" -- eliminating the "in
many cases" languages -- we had better be sure that there are no
"first" implementations out there (and I think we already have
evidence that there are) and we had better not be specifying new
functionality through the back door.  I don't see a way to meet
those conditions, but perhaps that is just lack of imagination.

        john

Date: Sun, 22 Apr 2007 15:08:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Frank Ellermann wrote:
> Kari Hurtta wrote:

>> Seems that http://www.iana.org/assignments/mail-parameters
>> lists VERB
> 
> Interesting, apparently they had some rather odd rules about
> registering extensions (Ned 1, Frank 0).  But otherwise it
> it is a clear "opt-in" extension, both sides are supposed to
> know what the effect of VERB is, no problem for 2821(bis).
> 
> Is it already clear that "buy the bat book" is not what we
> want for registered extensions ?
> 
> Frank (page 403, F=I)

If you think thats interesting, check out this *published book" with an 
entirely different (and very logical) interpretation for 1yz:

http://www.tcpipguide.com/free/t_SMTPRepliesandReplyCodes-2.htm

   1yz Positive Preliminary Reply

   An initial response indicating that the command has been
   accepted and processing of it is still in progress. The SMTP
   sender should expect another reply before a new command may be sent.

   Note that while this first digit type is formally defined in the
   SMTP specification for completeness, it is not currently actually
   used by any of the SMTP commands. That is to say, there are no
   reply codes between 100 and 199 in SMTP

WRITTEN AND PUBLISHED IN A TCP/IP GUIDE BOOK FOR PETE'S SAKE!

If makes you wonder why the AUTHOR wrote this and it makes you wonder 
the number of *unimportant" in-house SMTP implementator readers actually 
followed and implemented it depending on their needs.

The ISSUE I have with all this what I think is being inconsistently 
applied here for this issue and Multiple response lines, is that for 
some ODD ball reason we don't think locking it down will break code 
because a few don't think it its being used the enough "superstars" and 
"common systems" when in fact it was always possible in the 25 year 
history of SMTP, following a formal specifications (see below), there 
was NEVER really a concern that it will break a system if you did use 
it.  The concern was simply never there following SMTP allowed to do 
this without any minor details that it will break a system.

But instead of at least making these two issues a SHOULD, it is 
incomprehensibly made a MUST - a lock down - which essentially does 
three things:

1) It now makes any system that is using this in a published
    and unpublished way, NON-COMPLIANT, including "superstars."

2) If any new SMTP developer followed 2821bis or the new standard
    the  would come from this work, inevitably, they will come across
    a system where they will have a problem reading 1yx response lines.
    The end result is that they need to CHANGE their code to support
    these "legacy" systems,

3) It literally makes it a SHOW STOPPER for current and future work in
    this area - OPES is now completely dead - won't have any chance of
    maximizing adoptability.

So never mind about changing a 25 year old written functionality, from 
an engineering and practical stand point, it makes absolutely NO sense 
to lock down these items simply on the basis that SOME believe it isn't 
being used - it isn't simply true.

Finally, I've been ask to write an I-D that will literally propose to 
UPDATE and CHANGE the new specs to bring it back to the OLD spec!

Eh? There is something wrong with that approach.

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 22: Requiring removal of extra return-path headers in 4.4
Date:  Sun, 22 Apr 2007 21:08:49 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

John C Klensin wrote:
 
> Hi.  Issue 22 assigned to this.

Any procedure resulting in precisely one Return-Path created by
the final delivery MTA is IMO fine.  Can you please smuggle in
an informative reference to RFC 3834 to stress this point ?

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and "command"
Date:  Sun, 22 Apr 2007 20:54:41 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

John C Klensin wrote:

> Is this worth fixing?

Yes.  We need it to convert all now still pseudo-ABNF lines like
 
 MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>

into proper ABNF like

 mail    = "MAIL FROM:" <reverse-path> [SP <mail-parameters> ] <CRLF>

after an introduction in the direction of

 command = hello / mail / rcpt / data / rset / quit ; etc.

There are 302 occurences of "command" in 286 lines (my editor
says), from that POV explaining what it is is kind of pointless.

But there are also 15 ocurences of "verb", that's less obvious.
Please insert a new section above the "reply" part (now 2.3.11).

> a "yes" answer accompanied by suggested text will weigh more
> heavily on the editor than a "yes" answer without text

Unrelated, for some extremely important text see also
http://www.spamcop.net/fom-serve/cache/329.html#bounces

<section title="Commands and verbs"><t>
 After an initial greeting from the server, a special case of
 a reply discussed later, clients send either commands or when
 permitted a mail object.  The server sends a reply for each
 command.  All commands begin with what's known as "verb", 
 followed by additional arguments as needed for the command
 in question.
</t><t>
 The allowed commands depend on the state of the conversation.
 The allowed replies depend on the state and the command.
<!-- insert historical telnet/ftp note about this style ? -->
</t></section>

Frank
-- 
In any case, the SMTP adds its own identifier to the reverse-path.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date:  Sun, 22 Apr 2007 20:20:23 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Kari Hurtta wrote:

> Perhaps off topic, but it seems that sendmail uses 0xx
> with similar way if VERB command is given
[...]
> VERB
> 250 2.0.0 Verbose mode
> MAIL FROM:<hurtta@localhost>
> 250 2.1.0 <hurtta@localhost>... Sender ok
> RCPT TO:<root>
> 050 <root>... aliased to hurtta
> 250 2.1.5 <root>... Recipient ok

> Seems that http://www.iana.org/assignments/mail-parameters
> lists VERB

Interesting, apparently they had some rather odd rules about
registering extensions (Ned 1, Frank 0).  But otherwise it
it is a clear "opt-in" extension, both sides are supposed to
know what the effect of VERB is, no problem for 2821(bis).

Is it already clear that "buy the bat book" is not what we
want for registered extensions ?

Frank (page 403, F=I)

From: Kari Hurtta <[email-masked]>
To: "John C. Klensin" <[email-masked]>
cc: Kari Hurtta <[email-masked]>
Newsgroups: gmane.ietf.smtp
Subject: Received: -header field -- for -clause
Date: 31 Mar 2007 21:13:29 +0300
Lines: 43
Posted-To: gmane.ietf.smtp
Sender: [email-masked]

The following message is a courtesy copy of an article
that has been posted to gmane.ietf.smtp as well.

[ I guess that that this does not go to mailing
  list, what is behide of gmane.ietf.smtp ]

( Already mentioned in
  http://notabug.com/2002/drums-archive/6899
)

draft-klensin-rfc2821bis-01.txt: 

   Time-stamp-line  = "Received:" FWS Stamp <CRLF>

   Stamp          = From-domain By-domain Opt-info [CFWS] ";" FWS
                  date-time

   Opt-info       = [Via] [With] [ID] [For]

   For            = CFWS "FOR" 1*( FWS Path / Mailbox ))

I think that this allows several addresses.

RFC 2822 do not have provisions for several addresses:

received        =       "Received:" name-val-list ";" date-time CRLF

name-val-list   =       [CFWS] [name-val-pair *(CFWS name-val-pair)]

name-val-pair   =       item-name CFWS item-value

item-name       =       ALPHA *(["-"] (ALPHA / DIGIT))

item-value      =       1*angle-addr / addr-spec /
                         atom / domain / msg-id

( Is there update of RFC 2822 going? )

/ Kari Hurtta

From: Kari Hurtta <[email-masked]>
Newsgroups: gmane.ietf.smtp
cc: Hector Santos <[email-masked]>, Douglas Otis <[email-masked]>, Tony Finch <[email-masked]>, John C Klensin <[email-masked]>, Kari Hurtta <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date: 20 Apr 2007 18:33:41 +0300
Lines: 58
Posted-To: gmane.ietf.smtp
Sender: [email-masked]

The following message is a courtesy copy of an article
that has been posted to gmane.ietf.smtp as well.

John C Klensin <[email-masked]> writes in gmane.ietf.smtp:

> --On Tuesday, 10 April, 2007 15:23 -0400 Hector Santos
> <[email-masked]> wrote:

> > Second, there is no restriction in using it as part of a
> > continuation line.  Example:
> > 
> >      C: DATA
> >      S: 352 Begin sending your data
> >         [client uploads data]
> >      S: 150-DKIM signature found!
> >      S: 150-Please wait while we process your DKIM message!
> >      S: 150-Waiting for trust server to respond...
> >      S: 150-Still waiting for trust server to respond...
> >      C: 550-Sorry, The DKIM POLICY has failed this transaction
> >      C: 550 Please see http://trust-r-us.com/msgid=123213123
> > 
> > Who knows?  Is this not possible?  It is out of the whelm of
> > real possibility?
> 
> No, it is is clearly not impossible.  However, it raises two
> issues in addition to the one about the codes all being the same
> which, IMO (and in the text of 2821), arises from one of them.  
> 

Perhaps off topic, but it seems that sendmail uses 0xx
with similar way if VERB command is given

EHLo xx
250-Hurtta06k.keh.iki.fi Hello localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH DIGEST-MD5 CRAM-MD5
250-DELIVERBY
250 HELP
VERB
250 2.0.0 Verbose mode
MAIL FROM:<hurtta@localhost>
250 2.1.0 <hurtta@localhost>... Sender ok
RCPT TO:<root>
050 <root>... aliased to hurtta
250 2.1.5 <root>... Recipient ok

( Seems that http://www.iana.org/assignments/mail-parameters
  lists VERB )

1xx was not selected for that usage.   So 050 is really used,
but is 150 used on somewhere?

/ Kari Hurtta

From: Kari Hurtta <[email-masked]>
Newsgroups: gmane.ietf.smtp
cc: Tony Hansen <[email-masked]>, Kari Hurtta <[email-masked]>
Subject: "for" clause on Received: header field (Re: rfc2821bis-02 is out)
Date: 20 Apr 2007 18:38:43 +0300
Lines: 38
Posted-To: gmane.ietf.smtp
Sender: [email-masked]

The following message is a courtesy copy of an article
that has been posted to gmane.ietf.smtp as well.

Tony Hansen <[email-masked]> writes:

> The new draft can be found at your usual suspect locations, including
> 
> http://tools.ietf.org/html/draft-klensin-rfc2821bis
> 
> A side-by-side diff from rfc2821bis-01 can be seen at this URL:
> 
> http://tools.ietf.org/rfcdiff?url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-02.txt
> 
> Remember, if you want to discuss an issue with the draft, please use
> separate messages for each issue.
> 
> Also, please look at the open issues I pointed out in my recent posts.
> 
> 	Tony Hansen
> 	[email-masked]

For -clause on Received: allows some syntax which is not allowed
on RFC 2822:

 For            = CFWS "FOR" 1*( FWS Path / Mailbox ))

Compare to RFC 2822:

item-value      =       1*angle-addr / addr-spec /
                         atom / domain / msg-id

( that may also considered to be bug of RFC 2822 )

Is multiple mailboxes ever used on "for" -clause ?
Syntax for it is quite inconsistent between standards.

/ Kari Hurtta

Date: Sun, 22 Apr 2007 11:05:24 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-02 Issue 23: Definitions in Section 2.3 and "command"
Sender: [email-masked]

Hi.

It has been pointed out that Section 2.3 contains a definition
for "Reply" (in 2.3.11) but no parallel definition for
"Command".  Is this worth fixing?

Warning: At least absent controversy, a "yes" answer accompanied
by suggested text will weigh more heavily on the editor than a
"yes" answer without text or a "no" answer unless the latter can
demonstrate why such a definition would be harmful.

      john

Date: Sat, 21 Apr 2007 02:21:03 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Try next MX on a STARTTLS handshake error?
Sender: [email-masked]

At 17:56 -0700 on 04/20/2007, Carl S. Gutekunst wrote about Try next 
MX on a STARTTLS handshake error?:

>When a client MTA tries to perform TLS authentication on a server 
>MTA, and the authentication fails, should the client MTA move on to 
>the next MX host, or give up?
>
>[snip]
>
>I'm struggling with trying to decide if this is correct behavior or 
>not; for some reason it seems intuitively wrong, although I can make 
>a good argument for it being right.
>[snip]

Which way you go is (at least partly) how you classify the failure to 
authenticate.

If you take the tack that you have succeeded if you establish a 
connection to the MX, then the failure to authenticate should be a 
show-stopper and going on to the next MX should not be done.

OTOH, if you regard the failure to authenticate as a failure to 
connect (ie: As if the attempt to connect to the MX itself failed), 
then running the MX chain until all MXs are checked or you get one 
where there is no failure to authenticate, then keep checking.

IOW: When do you regard the connection as successful - When the MX 
responds or when you succeed in doing an authentication of the MX's 
certificate (and thus have established a secure session/connection)?

Date: Fri, 20 Apr 2007 17:56:32 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: [email-masked]
Subject: Try next MX on a STARTTLS handshake error?
Sender: [email-masked]

When a client MTA tries to perform TLS authentication on a server MTA, 
and the authentication fails, should the client MTA move on to the next 
MX host, or give up?

I have an OpenSSL-based SMTP MTA client that attempts to authenticate 
the server whenever offered the STARTTLS EHLO keyword. It gets the 
server's certificate, verifies that the certificate is valid, follows 
and verifies the trust chain, and then verifies that the hostname on the 
cert matches the hostname to which the client connected (normally the MX 
host). If any part of the authentication fails, the client drops the 
connection and then moves on to the next MX host, repeating until it has 
tried them all.

I'm struggling with trying to decide if this is correct behavior or not; 
for some reason it seems intuitively wrong, although I can make a good 
argument for it being right. I've got one case where the receiver has a 
phalanx of MTAs all sharing the same certificate; the client happily 
runs through all of them, failing them all. I've got another case, 
though, where the receiver has to MX hosts configured as primary and 
backup; the higher-precedence primary has an expired cert, but the 
secondary has a valid one. So the mail is delivered successfully.

Any RTFMs or pointers to other lists are welcome.

<csg>

Date: Fri, 20 Apr 2007 12:12:32 -0400
From: John C Klensin <[email-masked]>
To: SM <[email-masked]>
cc: [email-masked]
Subject: RFC2821bis-02 Issue 22: Requiring removal of extra return-path headers in 4.4
Sender: [email-masked]

Hi.  Issue 22 assigned to this.

Editor's note: My recollection of the DRUMS discussion is that
these suggests should be weak, rather than stronger, for two
reasons:

(i) We wanted to generally discourage relay MTA from inspecting
message headers.  Requiring that they remove return-path headers
is implicitly a requirement that they inspect headers, which is
inconsistent with the design of other parts of the specification.

(ii) Regardless of what we specify, delivery MTAs must, in
practice, be prepared to receive messages with return-path
headers already in place and hence must be really to cope with
them.  Making these requirements stronger (MAY-> SHOULD,
SHOULD-> MUST) doesn't help much with that other than, possibly,
creating a false sense of security.  

The reasons for changing our minds about that are described
below.

     john

--On Wednesday, 18 April, 2007 14:10 -0700 SM <[email-masked]>
wrote:

> 
> In Section 4.4:
> 
> I suggest changing the MAY to SHOULD in the following
> paragraph:
> 
>     It is sometimes difficult for an SMTP server to determine
> whether or
>     not it is making final delivery since forwarding or other
> operations
>     may occur after the message is accepted for delivery.
> Consequently,
>     any further (forwarding, gateway, or relay) systems SHOULD
> remove the
>     Return-path and rebuild the MAIL command as needed to
> ensure that
>     exactly one such line appears in a delivered message.
> 
> I'm inclined to change the following SHOULD NOT to MUST NOT.
> However a well-known MTA sends a message with a Return-path
> header.  If the SMTP servers performing a relay function
> cannot inspect the message data to determine if Return-path
> headers are present, we can reach a situation where there is
> more than one Return-path header.
> 
>     A message-originating SMTP system SHOULD NOT send a
> message that
>     already contains a Return-path header.  SMTP servers
> performing a
>     relay function MUST NOT inspect the message data, and
> especially not
>     to the extent needed to determine if Return-path headers
> are present.
>     SMTP servers making final delivery MAY remove Return-path
> headers
>     before adding their own.
> 
> Quoting the draft:
> 
>     "The primary purpose of the Return-path is to designate
> the address to
>     which messages indicating non-delivery or other mail
> system failures
>     are to be sent."
> 
> This is best achieved by ensuring that there is only one
> Return-path header.  I suggest changing the SHOULD to MUST in:
> 
>     For this to be unambiguous, exactly one return path
>     MUST be present when the message is delivered.
> 
> I suggest changing the SHOULD to MUST in the following
> paragraph:
> 
>     o  a gateway from elsewhere -> SMTP MUST delete any
> Return-path
>        header present in the message, and either copy that
> information to
>        the SMTP envelope or combine it with information
> present in the
>        envelope of the other transport system to construct the
> reverse
>        path argument to the MAIL command in the SMTP envelope.
> 
> I did not suggest changing the SHOULD for "a gateway from SMTP
> -> elsewhere" as there may be local considerations in the
> destination environment which require it.
> 
> Regards,
> -sm 

Date: Fri, 20 Apr 2007 12:05:11 -0400
From: John C Klensin <[email-masked]>
To: SM <[email-masked]>, [email-masked]
Subject: RFC2821bis-02 Issue 21: Re: Section 6.2 - favoring reject over bounce
Sender: [email-masked]

Issue 21 assigned.

--On Wednesday, 18 April, 2007 14:19 -0700 SM <[email-masked]>
wrote:

> 
> In Section 6.2.  Unwanted, unsolicited, and "attack" messages,
> I suggest favoring reject over bounce by a rewording of the
> sentence:
> 
>    They should be "bounced" as described above only if they
> cannot be delivered,
>    and cannot be rejected by the SMTP server during the SMTP
> transaction.
> 
> Regards,
> -sm
> 

Date: Wed, 18 Apr 2007 14:19:49 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Section 6.2 - favoring reject over bounce
Sender: [email-masked]

In Section 6.2.  Unwanted, unsolicited, and "attack" messages, I 
suggest favoring reject over bounce by a rewording of the sentence:

   They should be "bounced" as described above only if they cannot be 
delivered,
   and cannot be rejected by the SMTP server during the SMTP transaction.

Regards,
-sm

Date: Wed, 18 Apr 2007 14:10:27 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Section 4.4 - return-path
Sender: [email-masked]

In Section 4.4:

I suggest changing the MAY to SHOULD in the following paragraph:

    It is sometimes difficult for an SMTP server to determine whether or
    not it is making final delivery since forwarding or other operations
    may occur after the message is accepted for delivery.  Consequently,
    any further (forwarding, gateway, or relay) systems SHOULD remove the
    Return-path and rebuild the MAIL command as needed to ensure that
    exactly one such line appears in a delivered message.

I'm inclined to change the following SHOULD NOT to MUST NOT.  However 
a well-known MTA sends a message with a Return-path header.  If the 
SMTP servers performing a relay function cannot inspect the message 
data to determine if Return-path headers are present, we can reach a 
situation where there is more than one Return-path header.

    A message-originating SMTP system SHOULD NOT send a message that
    already contains a Return-path header.  SMTP servers performing a
    relay function MUST NOT inspect the message data, and especially not
    to the extent needed to determine if Return-path headers are present.
    SMTP servers making final delivery MAY remove Return-path headers
    before adding their own.

Quoting the draft:

    "The primary purpose of the Return-path is to designate the address to
    which messages indicating non-delivery or other mail system failures
    are to be sent."

This is best achieved by ensuring that there is only one Return-path 
header.  I suggest changing the SHOULD to MUST in:

    For this to be unambiguous, exactly one return path
    MUST be present when the message is delivered.

I suggest changing the SHOULD to MUST in the following paragraph:

    o  a gateway from elsewhere -> SMTP MUST delete any Return-path
       header present in the message, and either copy that information to
       the SMTP envelope or combine it with information present in the
       envelope of the other transport system to construct the reverse
       path argument to the MAIL command in the SMTP envelope.

I did not suggest changing the SHOULD for "a gateway from SMTP -> 
elsewhere" as there may be local considerations in the destination 
environment which require it.

Regards,
-sm 

Date: Wed, 18 Apr 2007 16:31:46 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: rfc2821bis-02 is out
Sender: [email-masked]

The new draft can be found at your usual suspect locations, including

http://tools.ietf.org/html/draft-klensin-rfc2821bis

A side-by-side diff from rfc2821bis-01 can be seen at this URL:

http://tools.ietf.org/rfcdiff?url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-02.txt

Remember, if you want to discuss an issue with the draft, please use
separate messages for each issue.

Also, please look at the open issues I pointed out in my recent posts.

	Tony Hansen
	[email-masked]

From: [email-masked]
Cc: [email-masked]
Date: Tue, 17 Apr 2007 14:42:49 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> >> If folks get used to register their new header fields specified in I-Ds
> >> it can help to avoid collisions.  And if they don't use it it won't help,
> >> with effects as noted in the subject of this thread.

> > Exactly. And this means that the goal of any registry is first and
> > foremost to encourage its use, even if that involves constructing the
> > registry in a way that may admit race conditions. And the only
> > meaningful measure of the success of any registry is whether or not it
> > gets used.

> Maybe it wasn't clear, I'm not against the status code registry, quite
> the contrary.  With thousands of RFCs it's very important to organize
> this huge repository for ordinary folks who have read less than say 100
> RFCs.  Maybe RFC 5000 will be a new STD 1, I'd appreciate it.

No, that was quite clear.

> I only don't share your optimism about this registry as improvement for
> more potential collisions.  IFF avoiding collisions is your main goal,
> and if you want the "expert review" model (two IFs), then I'd say "copy
> 3864".

My optimism for using a conventional registry to address this set of problems
is based on numerous past cases where registries have helped eliminate
collisions or duplications.

I don't share your optimism regarding the provisional registry approach simply
because the (admittedly fairly) small amount of running code we have says the
approach isn't working. Copying proven approaches is one thing, copying
approaches that appear to be in trouble is quite another.

> Otherwise, if you think that a good "collision detection" already helps
> to avoid most collisions, then I'd say "stay away from 'expert review',
> use the simpler 'RFC required' model" (or maybe 'IETF review').

I could certainly live with any of these, however, I don't share your pessimism
towards expert review. While there have been some problem cases (the charsets
registry is the obvious apps-related example), there are plenty of other cases
where the approach has worked (I may be biased, but IMO both the current media
types and language code registries have been handled reasonably well).

In the specific case of media types, the problem historically was essentially
the same as the one the current header registration process has - people
didn't/don't use it. It took a major revision of the media type registration
process to make it _much_ more user-friendly to get people to use it. I suspect
the same is going to be needed to have a header registration process that
actually works - my guess is we'll need to streamline the entire process to get
adoption, likely eliminating the entire provisionaal registry mechanism as we
go. However, I'm willing to wait a little longer before suggesting that we do
this.

But this brings up another point, which is that processes and procedures have
changed dramatically over time, making comparisons to how things worked more
than a few years back quite dubious in a lot of cases.  Many if not
most of the past problems with IANA registry processing, including but not
limited to management of expert reviews, have been eliminated by IANA's
adoption of a ticket system. Things just don't slip through the cracks the way
they did 3-4 years ago. And recent changes (some only a few months old) have
tightened things up even more.

For the problems that remain, the really interesting question is why expert
review works for some things and fails for others. From what I've seen the
problem arises when the review process is infrequently used and is for whatever
reason difficult to perform. There is no indication that enhanced status code
review is going to be difficult, and while there's no chance it will reach the
frequency of media type registrations, it doesn't appear to be on the "once in
a blue moon" track either.

> > only actual results matter, and we're not seeing those, at least not yet.

> One obstactle with RFC 3864 was to initialize the bureaucracy, convince
> IANA that they're supposed to handle this, let IANA convince the IESG
> that the IESG has to appoint an expert, and let the IESG convince the
> author that he's the expert also on this review list.

Some of this may have been valid criticism in the past, but almost none  of it
is relevant now, due in large part to the way IANA works with the RFC Editor
during the publication process. To the extent there are problems in the future,
they're going to be with unusual procedures, which likely still applies to the
provisional registry approach.

> For the "expert
> review" model there has to be a volunteer for this job, and the author
> might be considered as the obvious choice. Maybe my extrapolation based
> on Graham and you as experts isn't correct.  Otherwise Tony can take it
> as fair warning, if he picks this model instead of "RFC required" or the
> IETF variation of this theme.

The author is indeed the obvious choice.

				Ned

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Date:  Tue, 17 Apr 2007 21:06:24 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 58
Sender: [email-masked]

[email-masked] wrote:

>> If folks get used to register their new header fields specified in I-Ds
>> it can help to avoid collisions.  And if they don't use it it won't help,
>> with effects as noted in the subject of this thread.

> Exactly. And this means that the goal of any registry is first and
> foremost to encourage its use, even if that involves constructing the
> registry in a way that may admit race conditions. And the only
> meaningful measure of the success of any registry is whether or not it
> gets used.

Maybe it wasn't clear, I'm not against the status code registry, quite
the contrary.  With thousands of RFCs it's very important to organize
this huge repository for ordinary folks who have read less than say 100
RFCs.  Maybe RFC 5000 will be a new STD 1, I'd appreciate it.

I only don't share your optimism about this registry as improvement for
more potential collisions.  IFF avoiding collisions is your main goal,
and if you want the "expert review" model (two IFs), then I'd say "copy
3864".

Otherwise, if you think that a good "collision detection" already helps
to avoid most collisions, then I'd say "stay away from 'expert review',
use the simpler 'RFC required' model" (or maybe 'IETF review').

> only actual results matter, and we're not seeing those, at least not yet.

One obstactle with RFC 3864 was to initialize the bureaucracy, convince
IANA that they're supposed to handle this, let IANA convince the IESG
that the IESG has to appoint an expert, and let the IESG convince the
author that he's the expert also on this review list.  For the "expert
review" model there has to be a volunteer for this job, and the author
might be considered as the obvious choice.  Maybe my extrapolation based
on Graham and you as experts isn't correct.  Otherwise Tony can take it
as fair warning, if he picks this model instead of "RFC required" or the
IETF variation of this theme.

 [name squatters]
> What about them? It is axiomatic that registries only work when they are
> used.  Squatters by definition aren't following the process.

Yes, but that wasn't what I meant, I had (domain) name squatters in mind.
Folks intentionally reserving names because they hope to that somebody
else might need them in the future.  With "first come first served" (and
neither "RFC required", nor "expert review") it could be possible to
grab names or numbers, and cause intentional conflicts.  Likely too
pessimistic, "DSN status codes" don't play in the same league as URNs or
"URI schemes", they're more in the direction of "SASL mechanisms" (that
would be neither RFC nor expert, IMO too permissive).

> As usual we're letting the best be the enemy of the good.

Not really.  You've good reasons why you insist on "IETF consensus", and
I've (less good) reasons why I like John's carefully hidden loopholes.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: recap of 2821bis-01 closed issues
Date:  Tue, 17 Apr 2007 20:06:36 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Tony Hansen wrote:

> 0a)     rewrite section 7.1 to remove terms like "ignorant"
>         consensus to do

Section 7.1 has to be rewritten from scratch, it's unacceptable.
But IMO that can wait until most less controversial issues are
settled.

> 0b)     clarify MAIL FROM validation and EHLO validity tests
>         no consensus to change

The duties of a border MTA, i.e. either "accept and eat your
spam" or "if in doubt reject", are not yet clear in the draft.
We can discuss it later in conjunction with a new section 7.1.

> 12)     IPv6 issues with MX records
>         proposed wording received from ADs

Please post a Message-ID, I missed that.  The "IPv6 only" case
is related to 0a and 0b:  No IPv4 only user can be interested
in spam from "IPv6 only", let alone legit mail from "IPv6 only".
It's also related to UTF8SMTP.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Date:  Tue, 17 Apr 2007 19:31:43 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

John C Klensin wrote:

> I also received a couple of long messages from a usual suspect
> in these matters

<sigh />.  That's a one-man-process-experiment, or actually he's
a set of experiments not limited to 3683.  I bet he likes TLDs
operated as (SMTP or NNTP) hosts...

> claiming to advise and/or represent several ccTLDs

...he's not necessarily talking about TLDs under the ICANN root.

> Bottom line, based on responses received so far, is that no one
> is using mail to TLD names.  Some think they would like to.

> My impression, based on those remarks but, more important, on my
> reading of the comments on the list, is that we shouldn't ban
> mail to TLD domains, but should clarify and strengthen the "only
> FQDNs on the wire" rule.

If you change your mind again:  I still like your 2821 "one-dot"
rule disqualifying TLDs, already mirrored in 4408 <domain-spec>
and in RFC.usefor-usefor-12 <path-nodot>.

Besides I don't like "host museum.", and I hated "host com." when
it was around.  Nobody needs TLDs as host (apart from localhost).

AFAIK ICANN tries to guarantee that TLD operators always get SLDs
noc.tld and whois.tld, isn't that good enough for all TLDs ?

TLD operators are also always free to create any SLD they like as
mail host (e.g. mail2.tld, "mail2" is a popular label for hosts
offering POP3, MSA, and webmail services).

Frank

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Date: Tue, 17 Apr 2007 10:25:08 -0700
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

On Apr 17, 2007, at 3:36 AM, Arnt Gulbrandsen wrote:

>
> John C Klensin writes:
>> Bottom line, based on responses received so far, is that no one is  
>> using mail to TLD names. Some think they would like to.
>
> Ah. I'm sure Verisign would like to _be_ .com ;)

Concerns could be more along the lines of market advantages created  
by DKIM.  As currently specified, keys within any parent domain can  
validate messages for any sub-domain.  This allows TLDs to publish  
keys where their outbound MX service can "DKIM sign" email from  
anywhere within the TLD.

-Doug

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: 2821bis-01 Issue 5: Sect 4.4. Syntax of ID in trace: needs confirmation
Date:  Tue, 17 Apr 2007 18:48:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 48
Sender: [email-masked]

Tony Hansen wrote:

> I'd like to reconfirm the consensus between these two choices:

> (i) change "string" to "atom"
> (ii) change "string" to "dot-string"

With the examples of <Dot-string>s shown here I tend to (II).

In <http://permalink.gmane.org/gmane.ietf.smtp/5496> I proposed
to replace 2821 <Dot-string> by 2822 <dot-atom-text> because
they are syntactically equivalent.

There are only six uses of <Atom> in 2821bis-01, two of them in
the <Dot-string>.  Confusingly a 2821 <Atom> isn't the same as
a 2821 <atom>, the latter is [CFWS] 1*atext [CFWS], but in 2821
it's only 1*atext.

I propose to replace the remaining four 2821 <Atom> by 1*atext
eliminating this potential cause of confusion:

- String         = Atom / Quoted-string
+ String         = 1*atext  / Quoted-string

- Addtl-Link     = Atom
+ Addtl-Link     = 1*atext

- Attdl-Protocol = Atom
+ Attdl-Protocol = 1*atext

- Argument       = Atom
+ Argument       = 1*atext

The uses of <String> in 2821bis, ignoring the settled "ID" case,
have to be checked:  Are the arguments for EXPN, VRFY, HELP,
and NOOP really a <Quoted-string> if they're no <Atom> ?

In other words, is 'HELP me now' a syntax error, should it be
'HELP "me now"' ?  Comparing it with RFC 821, apparently these
strings used to be *_unquoted_* strings of any <c> (= <atext>)
or "\" <x> (= <quoted-pair>).  Is 'HELP me\ now' acceptable ?

RFC 821 allowed no argument for NOOP.  Please add this to the
todo list for an "implementation and interoperability report":
Are NOOP arguments supported, or do they cause havoc ?

Frank

Date: Tue, 17 Apr 2007 07:08:35 -0400
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
cc: Tony Hansen <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Sender: [email-masked]

--On Tuesday, 17 April, 2007 12:36 +0200 Arnt Gulbrandsen
<[email-masked]> wrote:

> John C Klensin writes:
>> Bottom line, based on responses received so far, is that no
>> one is  using mail to TLD names. Some think they would like
>> to.
> 
> Ah. I'm sure Verisign would like to _be_ .com ;)
> 
>> My impression, based on those remarks but, more important, on
>> my  reading of the comments on the list, is that we shouldn't
>> ban mail to  TLD domains, but should clarify and strengthen
>> the "only FQDNs on the  wire" rule.
> 
> That would satisfy me, although I remain jittery about highly
> unusual syntax. (If 99.99% of email is to non-TLDs, the last
> 0.01% will attract more than its share of bugs. Is that 0.01%
> worth the bugs? But all this is just a parenthetical remark.)

My guess is that, regardless of what we do, the situation will,
in practice, continue more or less as it has been.  In other
words, many or most MUAs and Submission servers will not make
the special provisions for identifying the difference between a
TLD name and an abbreviated domain-part that 2821 calls for, and
certainly will not make them in a consistent way.  The result
will be that access to mailboxes with TLD domain-parts will not
be usefully and consistently accessible to users and that will
make the "service" hard to market, whether as a user@TLD
directory or subdomain@TLD as an alternative form for what has
traditionally been [email-masked].  That situation
will continue until and unless at least as few of the large
gTLDs adopt the model... and that will require an ICANN PDP or
other approval process that would, I predict, take many, many,
years.

But I think the general trend of the discussions is that any
restrictions should not come from the protocol itself.

     john

Date: Tue, 17 Apr 2007 12:36:52 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Cc: Tony Hansen <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> Bottom line, based on responses received so far, is that no one is 
> using mail to TLD names. Some think they would like to.

Ah. I'm sure Verisign would like to _be_ .com ;)

> My impression, based on those remarks but, more important, on my 
> reading of the comments on the list, is that we shouldn't ban mail to 
> TLD domains, but should clarify and strengthen the "only FQDNs on the 
> wire" rule.

That would satisfy me, although I remain jittery about highly unusual 
syntax. (If 99.99% of email is to non-TLDs, the last 0.01% will attract 
more than its share of bugs. Is that 0.01% worth the bugs? But all this 
is just a parenthetical remark.)

>>  When 2821bis-02 comes out, please review the text in section 2.3.5.
>
> please

Of course.

Arnt

Date: Tue, 17 Apr 2007 05:56:39 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Sender: [email-masked]

--On Monday, 23 April, 2007 23:48 -0400 Tony Hansen
<[email-masked]> wrote:

> 
> The following issue with 2821bis-01 is still open, but we have
> a tentative conclusion that needs to be affirmed after
> 2821bis-02 comes out.
> 
> 1)	Format of domain name - does it allow/not-allow/require
> period/trailing-period in TLD?
> 
> The discussion on this issue was all over the place with no
> totally clear consensus. The trend was toward no dot, but
> adding some explanations about top level domains.
> 
> An email was sent to the ICANN to find out why some country
> code TLDs have MX records, but no response has been received.

Not quite true ... sorry for not noting the responses to the
list when they arrived, but I've been trying to avoid adding to
the clutter.

The final action at ICANN was to send a copy of my note out to
the ccTLD list.  So far, I've received about a half-dozen
responses.  They varied from one domain that was not using
these, had noticed that they often didn't work, and considered
them dumb to someone who has an MX record but wanted to start a
debate about "services" of this variety and maybe making
Verisign support it in .COM so that MTA/MUA vendors would be
forced to support it.

I also received a couple of long messages from a usual suspect
in these matters (and from an organization that may or may not
consistent entirely of him), claiming to advise and/or represent
several ccTLDs, indicating that SMTP was not the right situation
(although he still cares about MX records although I don't
presume he knows what they are) and the DNS isn't either and
trying to put this all in context with the nationally-based
"multilingual Internet".  I assume those comments apply to the
well-known X.400(2008) and not to SMTP or RFC2821bis.

Bottom line, based on responses received so far, is that no one
is using mail to TLD names.  Some think they would like to.

My impression, based on those remarks but, more important, on my
reading of the comments on the list, is that we shouldn't ban
mail to TLD domains, but should clarify and strengthen the "only
FQDNs on the wire" rule.  That impression is instantiated in the
-02 draft although, of course, it could be changed again if it
isn't right.

> When 2821bis-02 comes out, please review the text in section
> 2.3.5.

please

   john

Date: Tue, 17 Apr 2007 08:19:27 +0100
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: SMTP Interest Group <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 5: Sect 4.4. Syntax of ID in trace: needs confirmation
Sender: [email-masked]

On Mon, 23 Apr 2007, Tony Hansen wrote:
>
> 5)	Sect 4.4. Syntax of ID in trace
>
> The discussion on this issue gave a fairly clear consensus forming
> around changing "string" to "atom". However it was pointed out that this
> needs to be "dot-string" instead of "atom".

A dot-string has never before been permitted there by the syntax. However
I see a number of dot-string style ID fields in my email so this change
would be blessing existing practice, and I think it's fairly harmless.

        by imo-m22.mx.aol.com (mail_out_v38_r8.1.) id p.d0d.e1b18c6 (52371)

        by p101m061.symantecmail.net (mxl_mta-2.14.0-15)
        with SMTP id 915fed54.1636494256.133251.p101m061.symantecmail.net

	by lon-mail-3.gradwell.net with esmtp (Gradwell gwh-smtpd 1.232)
	id 4529adbb.7c5.54f

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOLE: NORTHEASTERLY 4 OR 5, OCCASIONALLY 6 IN EAST. SLIGHT OR MODERATE. FOG
PATCHES CLEARING. MODERATE OR GOOD, OCCASIONALLY VERY POOR.

Date: Tue, 24 Apr 2007 00:12:28 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap: 2821bis-01 Postponed Issues
Sender: [email-masked]

The following issues with 2821bis-01 are still open, and a solution to
them is being postponed to after 2821bis-02 has been posted.

2)	Check VRFY/EXPN text in Section 3.5.3
7)	Switch conformity language to 2119 form
10)	Add discussion of mail rerouting via 251/551 to Security
	Considerations
16)	General use of "is permitted", etc
19)	Comments on literals in EHLO - remove either "Extended-literal"
	or the text in 4.1.1.1 that describes it.
20)	RFC3552 and Security Considerations

Thanks.

	Tony Hansen
	[email-masked]

Date: Tue, 24 Apr 2007 00:09:37 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap: 2821bis-01 Issue 8: text on transactions/recipients per connection
Sender: [email-masked]

The following issue with 2821bis-01 is still open, but we have a
tentative conclusion that needs to be confirmed.

8)	Should there be text added regarding limits on number of
transactions/recipients per connection

After this issue was split out, there were clarifications
differentiating this issue from recipients per message and a number of
other issues raised, but not much posted on this specific question.

At this point, we think this issue is dead and nothing should be added
to or removed from the draft.

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 23:58:31 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap: 2821bis-01 Issue 5: Sect 4.4. Syntax of ID in trace: needs confirmation
Sender: [email-masked]

The following issue with 2821bis-01 is still open, but we have a
tentative conclusion that needs to be confirmed.

5)	Sect 4.4. Syntax of ID in trace

The discussion on this issue gave a fairly clear consensus forming
around changing "string" to "atom". However it was pointed out that this
needs to be "dot-string" instead of "atom".

I'd like to reconfirm the consensus between these two choices:

(i) change "string" to "atom"
(ii) change "string" to "dot-string"

Thanks.

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 23:53:35 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap: 2821bis-01 Issue 3: EHLO validation in sect 4.1.4, para 6
Sender: [email-masked]

The following issue with 2821bis-01 is still open, but we have a
tentative conclusion that needs to be affirmed after 2821bis-02 comes out.

3)	the use and validation of the domain name value in EHLO or HELO

The discussion on this issue was all over the place with no totally
clear consensus. Some of the text in section 4.1.4 has been rewritten to
eliminate redundancies and to add more cross references. When 2821bis-02
comes out, please review the text in section 4.1.4.

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 23:48:46 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap: 2821bis-01 Issue 1: Format of domain - trailing period
Sender: [email-masked]

The following issue with 2821bis-01 is still open, but we have a
tentative conclusion that needs to be affirmed after 2821bis-02 comes out.

1)	Format of domain name - does it allow/not-allow/require
period/trailing-period in TLD?

The discussion on this issue was all over the place with no totally
clear consensus. The trend was toward no dot, but adding some
explanations about top level domains.

An email was sent to the ICANN to find out why some country code TLDs
have MX records, but no response has been received.

When 2821bis-02 comes out, please review the text in section 2.3.5.

	Tony Hansen
	[email-masked]

Date: Mon, 23 Apr 2007 13:28:28 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: recap of 2821bis-01 closed issues
Sender: [email-masked]

This is a recap of where we stand on issues that I consider to be closed
with respect to 2821bis-01. John is getting ready to submit -02 and
these will be included. I will send out separate messages on other
issues that are not yet closed.

Thank you everyone for your help.

	Tony Hansen
	[email-masked]

The following issues have reached concensus and can be considered closed:

0a)	rewrite section 7.1 to remove terms like "ignorant"
	consensus to do
0b)	clarify MAIL FROM validation and EHLO validity tests
	no consensus to change
4)	improve Sect 4.2.1 description of Reply Codes 4yz 5yz
	consensus to improve, changed to SHOULD and SHOULD NOT
6)	Precedence error -- ABNF technical problem w/ grouping
	fixed
9)	Add requirements and limits table or equivalent, ala 1123
	4.5.3.1 and 4.5.3.2 lists clarified, converted to sections
11)	Change instances of "A RR" to language that is IPv6-inclusive
	done
12)	IPv6 issues with MX records
	proposed wording received from ADs
13)	Syntax for parameters that contain email addresses.
	fixed to use xtext from RFC 3461 section 4
14)	Continuation lines for 220 Greeting?
	consensus to fix ABNF; no dissenting implementations found
15)	Syntax change for reply lines to allow for continuation?
	consensus of clear bug in ABNF
17)	What to do with multiple codes in continuations
	consensus to use single code
18)	What to do with 1yz codes
	consensus to only allow 1yz with extension

From: [email-masked]
Cc: [email-masked]
Date: Sun, 15 Apr 2007 14:02:44 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> >> BTW, RFC 3864 is an _excellent_ model for an "expert review" registry

> > The jury is still out on that, but things aren't looking good. RFC 3864
> > has been out for two and a half years, and in that time there has
> > apparently been exactly one non-RFC provisional registration: Martin's
> > Archived-at: proposal.

> Meanwhile three, it took some time to kick IANA that they kick the IESG
> to appoint an expert reviewer.  For JabberID the registration was aborted
> after Bruce found that the syntax had issues, IMO related to a bug in the
> RFC 4622 URI scheme.  Indirectly the JabberID review helped to identify
> the 4622 bug.

Which is all well and good but has little to do with the intended goal of the
provisional registry.

> > I have a very hard time beleving the world has stood still in regards
> > to header field name use for all this time.  A far more elaborate
> > registration procedure than most that's rarely is ever used is NOT an
> > "excellent" model for anything IMO.

> If folks get used to register their new header fields specified in I-Ds
> it can help to avoid collisions.  And if they don't use it it won't help,
> with effects as noted in the subject of this thread.

Exactly. And this means that the goal of any registry is first and foremost to
encourage its use, even if that involves constructing the registry in a way
that may admit race conditions. And the only meaningful measure of the success
of any registry is whether or not it gets used.

> > in the fullness of time all of this stuff may prove to be useful and
> > therefore justified. But little if any of it has so far.

> It's one of the many things that can work if folks want it to work.  I'm
> no big fan of the "expert review" model, experts tend to be MIA or AWOL,
> or the review lists stop to work, but when it's anyway used, then adding
> "provisional registrations" as a feature where collisions are possible
> makes sense.

Again, it may make theoretical sense, but practically speaking only actual
results matter, and we're not seeing those, at least not yet.

> > There are all sorts of ways to define a collision-avoiding registry
> > that don't involve a provisional registration step. An obvious one is
> > the "IANA assigns number during the publication process".

> Does that work for ideas published only in drafts ?

Of course not. But that begs the question of whether or not you want people
implementing and deploying ideas that only appear in drafts. And the answer to
that is sometimes you don't care, but other times you actually want to actively
discourage such behavior, and one way to do that is to deny allocation until
RFC publication.

>  Without an "expert review", what about name squatters ?

What about them? It is axiomatic that registries only work when they are used.
Squatters by definition aren't following the process.

> > I don't think it is the right approach for extended status codes even
> > though they are numeric and hence amenable to being assigned this
> > way. But it is simply not the case that the elaboration of provisional
> > registration is necessary to solve this problem

> I don't see how the draft under discussion solves the collision problem
> that triggered its creation.

There are actually two collision problems here, one IMO major and the other IMO
fairly minor. The major one is that if Chris Newman hadn't done a careful
review we would have ended up with a conflict between two published RFCs. It's
unacceptable to depend on this sort of careful review to avoid such problems
and that's the problem we really need to solve.

The issue that draft-siemborski-rfc2554bis.txt sat around for a good long while
with a conflict is far less important. And while it can theoreticallly be
solved either through the use of a provisional registry or an
assign-at-publication-time approach, that only works if the construct is used,
and we're not seeing the running code elsewhere indicating the provisionaal
registry approach will be used if it is adopted.

I would also argue that the real problem here is that 2554bis sat around for
literally years without forward progress and despite apparent need for the
revision. I really don't think it is a good idea to effectively encourage
implementation of security services from drafts no matter what the registry
mechanism for error codes happens to be.

> > _in practice_ simply having a registry, even one whose procedues don't
> > prevent race conditions, eliminates most collisions. The enhanced status
> > code problem is the result of not having a registry period, as opposed
> > to not having the right sort.

> It would be the same mess if the long forgotten I-D had a proper "IANA
> considerations" section requesting a registration.

No it wouldn't. IANA would have caught the conflict before publication and we
would not have had to depend on a review happening across it.

> If an expired I-D
> is implemented, but the name or number isn't registered, unrelated I-Ds
> could run into the same trap.  For any enumeration the next (allegedly)
> free number is an obvious choice.

And if the draft fails to use the provisional registry - as will likely be the
case - the same problem will exist. As usual we're letting the best be the
enemy of the good.

				Ned

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Date:  Sun, 15 Apr 2007 21:48:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 64
Sender: [email-masked]

[email-masked] wrote:

>> BTW, RFC 3864 is an _excellent_ model for an "expert review" registry

> The jury is still out on that, but things aren't looking good. RFC 3864
> has been out for two and a half years, and in that time there has
> apparently been exactly one non-RFC provisional registration: Martin's
> Archived-at: proposal.

Meanwhile three, it took some time to kick IANA that they kick the IESG
to appoint an expert reviewer.  For JabberID the registration was aborted
after Bruce found that the syntax had issues, IMO related to a bug in the
RFC 4622 URI scheme.  Indirectly the JabberID review helped to identify
the 4622 bug.

> I have a very hard time beleving the world has stood still in regards
> to header field name use for all this time.  A far more elaborate
> registration procedure than most that's rarely is ever used is NOT an
> "excellent" model for anything IMO.

If folks get used to register their new header fields specified in I-Ds
it can help to avoid collisions.  And if they don't use it it won't help,
with effects as noted in the subject of this thread.

> in the fullness of time all of this stuff may prove to be useful and
> therefore justified. But little if any of it has so far.

It's one of the many things that can work if folks want it to work.  I'm
no big fan of the "expert review" model, experts tend to be MIA or AWOL,
or the review lists stop to work, but when it's anyway used, then adding
"provisional registrations" as a feature where collisions are possible
makes sense.

> There are all sorts of ways to define a collision-avoiding registry
> that don't involve a provisional registration step. An obvious one is
> the "IANA assigns number during the publication process".

Does that work for ideas published only in drafts ?  Without an "expert
review", what about name squatters ?

> I don't think it is the right approach for extended status codes even
> though they are numeric and hence amenable to being assigned this
> way. But it is simply not the case that the elaboration of provisional
> registration is necessary to solve this problem

I don't see how the draft under discussion solves the collision problem
that triggered its creation.

> _in practice_ simply having a registry, even one whose procedues don't
> prevent race conditions, eliminates most collisions. The enhanced status
> code problem is the result of not having a registry period, as opposed
> to not having the right sort.

It would be the same mess if the long forgotten I-D had a proper "IANA
considerations" section requesting a registration.  If an expired I-D
is implemented, but the name or number isn't registered, unrelated I-Ds
could run into the same trap.  For any enumeration the next (allegedly)
free number is an obvious choice.

If the procedure doesn't help with unintentional collisions I dare not
think about intentional collisions.

Frank

From: [email-masked]
Cc: [email-masked]
Date: Sun, 15 Apr 2007 08:11:07 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> > there's plenty of "running code" in this space - for example, there are
> > heaven knows how many nonstandard header field in use, but aside from the
> > attempts by Jacob Palme and other third parties leading up to the current
> > header registration model, how many independent RFC submissions have
> > there been by authors seeking to document their header usage?

> BTW, RFC 3864 is an _excellent_ model for an "expert review" registry:

The jury is still out on that, but things aren't looking good. RFC 3864 has
been out for two and a half years, and in that time there has apparently been
exactly one non-RFC provisional registration: Martin's Archived-at: proposal.
And the only use of it in an RFC has apparently been the initial "bulk load" of
the HTTP header registry. I have a very hard time beleving the world has stood
still in regards to header field name use for all this time.

A far more elaborate registration procedure than most that's rarely is ever
used is NOT an "excellent" model for anything IMO.

> It allows to "reserve" header fields with provisional registrations
> (example: Archived-At) while they're not yet standardized, it allows
> to deprecate non-standard header fields (example: Errors-To), and if
> a header field finally makes into an RFC (or similar, but we can ignore
> "similar" for status codes), it's added to the "permanent" registry.

Yes, and in the fullness of time all of this stuff may prove to be useful and
therefore justified. But little if any of it has so far.

> With that rules in place the old x.7.8 defined in an I-D could have
> been registered provisionally, avoiding the current mess.

> Models without a "provisional registry" don't avoid such collisions in
> practice, they only avoid the collisions in standards published years
> later.

Nonsense. There are all sorts of ways to define a collision-avoiding registry
that don't involve a provisional registration step. An obvious one is the "IANA
assigns number during the publication process". Of course this doesn't work for
field names and I don't think it is the right approach for extended status
codes even though they are numeric and hence amenable to being assigned this
way. But it is simply not the case that the elaboration of provisional
registration is necessary to solve this problem - and unless something changes
in the header registry world it may prove not to be sufficient either.

In fact _in practice_ simply having a registry, even one whose procedues don't
prevent race conditions, eliminates most collisions. The enhanced status code
problem is the result of not having a registry period, as opposed to not having
the right sort.

				Ned

Date: Fri, 13 Apr 2007 17:57:26 -0400
From: John Leslie <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Sender: [email-masked]

Peter J. Holzer <[email-masked]> wrote:
> On 2007-04-13 10:07:19 -0400, John Leslie wrote:
>> Peter J. Holzer <[email-masked]> wrote:
>>> On 2007-04-12 16:44:01 -0400, Robert A. Rosenberg wrote:
>>>> At 15:20 -0400 on 04/11/2007, Tony Hansen wrote...
>>>>>	(iii) Prohibit different codes and, optionally, suggest
>>>>>	that it is ok for a client to select one of them and
>>>>>	assume that all of the others are the same.
>> 
>> We're discussing a bug (IMHO) in 2821 which allows, in the case of
>> multiline replies to DATA (phase 2), different clients to have
>> different views of whether the message was accepted for delivery.
> 
> The bug is IMHO that it isn't completely clear whether all codes in a
> multiline reply must be the same. Hector interpretes the current text to
> mean that they can be different, while several other (including me)
> interprete it to mean that they must be the same.

   The question isn't whose interpretation is "right".

   There isn't any clear statement in RFC2821 that different codes are
an error: thus, that spec allows Hector's behavior in a server. The
bug (as I said) is that such allowable behavior can lead to different
clients having different views of whether the message was accepted
for delivery.

>> I, at least, would like to say something like, "Absent agreement
>> between server and client on the meaning of different reply codes
>> within a multiline reply, the server SHOULD make all the codes the
>> same, and the client SHOULD generate an error if it receives
>> different codes."
> 
> I would go for "the server MUST make all the codes the same" here. Is
> there anything to be gained from this ambiguity? 

   It's a smaller change; thus it increases our chances of getting to
Proposed Standard.

> At most I would add a warning that an extension could allow different
> codes in multiline-replies.

   I would go for MUST language myself if we were starting from scratch.
Different codes strikes me as a bad idea. (I wouldn't like it even if an
extension were available.)

--
John Leslie <[email-masked]>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2181bis-01 Issue 20: General security considerations overhaul
Date:  Fri, 13 Apr 2007 22:34:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

John C Klensin wrote:

> Instructions?

Let's do the security consideration when we know what we're
talking about, starting with a clean uncontroversial (some
kind of "rough consensus") ABNF published in at least one
I-D passing Bill's validator.

We also should have it clear what the outcome of the not yet
existing "implementation and interoperability report" means
for  2821bis.  It makes no sense to discuss the security of
say EXPN when it's anyway removed because nobody suports it.
Or those address literal explanations.

But posting -02 very soon now is fine.  I'm starting to get
confused what it will be.  The IPv6 stuff could be another
major obstacle, discussing security considerations is IMO
premature.

Frank

Date: Fri, 13 Apr 2007 16:14:35 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:
> 
> 
> --On Friday, 13 April, 2007 14:07 -0400 Hector Santos
> <[email-masked]> wrote:
> 
>> I am not proposing anything else but a "150-" as a PRELIMINARY
>> reply code for this purpose. No mixing of two or more
>> different 2xx, 3xx, 4xx, 5xx lines.  If the server finds
>> itself where it needs to "keep the channel" alive, or show it
>> is still "working" it could issue the PRELIMINARY code 150-
>> before any other COMPLETION code is finally determined and
>> issued.
> 
 > ..
 >
> So, ironically, the more strongly you argue that the present
> text permits multiline replies with different codes as long as
> those codes follow one particular scheme, the stronger the
> arguments get for insisting that all continuation lines use the
> same code.

Sorry John, I prematurely hit the SEND button when I was going to add 
the following. I can easily see how you can come to your summary.

BTW, as I indicated a few times, starting with the very first message in 
this thread (as a reply to Frank) I am not proposing any new text 
defining usage, nor new functionality as described above as one possible 
*implementation* method.

--
HLS

Date: Fri, 13 Apr 2007 15:28:48 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Sender: [email-masked]

--On Friday, 13 April, 2007 14:07 -0400 Hector Santos
<[email-masked]> wrote:

> I am not proposing anything else but a "150-" as a PRELIMINARY
> reply code for this purpose. No mixing of two or more
> different 2xx, 3xx, 4xx, 5xx lines.  If the server finds
> itself where it needs to "keep the channel" alive, or show it
> is still "working" it could issue the PRELIMINARY code 150-
> before any other COMPLETION code is finally determined and
> issued.

Hector,

Once again, if we were in the middle of the original DRUMS work,
it would have been be perfectly reasonable to propose:

	(i) Define 150 as a code to be used only in multiline
	responses as a placeholder and
	
	(ii) Permit mixed codes in multiline replies as long as
	all of them are the same except for zero or more
	intermediate lines starting with 150.

But it is just too late to do that in 2821bis since it involves
new definitions and new functionality.

In my view of what is going on here, we are supposed to be
eliminating ambiguities and dropping unimplemented features (if
any).  We can get rid of the apparent ambiguity about mixed
codes in multiline replies by prohibiting them or by explicitly
permitting them.  But, if we do the latter, we need to specify
what happens if they are different _in the general case_.  It
would be unambiguous to say, "well, in the general case, you
can't use multiline replies with different codes, but there is
this specific case involving '150' lines in particular
contexts....", but it involves new functionality for the
standard (i.e., regardless of what you have been doing in your
implementation and no matter how successful it has been), doubly
so because 2821 and its predecessors have never assigned any
semantics to a 150 code and the comment you take as authorizing
it at all is (IMO) clearly about final-position 1yz codes, not
their use as placeholders intermediate in multiline replies.

I just don't see any way out of this box.   We either (i) retain
the ambiguities that you believe exist, or (ii) we add new ones,
or (iii) we add or change functionality.  Or we end up
prohibiting what you ware asking for.    We can't do (iii).
(ii) strikes me as a bad idea, inconsistent with our mandate.
And, once it has been persuasively argued that an ambiguity
exists, I see a rational way to leave it there if we know how to
fix it.

So, ironically, the more strongly you argue that the present
text permits multiline replies with different codes as long as
those codes follow one particular scheme, the stronger the
arguments get for insisting that all continuation lines use the
same code.

     john

Date: Fri, 13 Apr 2007 14:07:05 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Peter J. Holzer wrote:

> The bug is IMHO that it isn't completely clear whether all codes in a
> multiline reply must be the same. Hector interpretes the current text to
> mean that they can be different, while several other (including me)
> interprete it to mean that they must be the same.

No Peter.  I could easily be among the same group to have a view that 
suggest that it is "common sense" to presume it is all the same.

However, in the name of looking for a solution, and there is no 
normative text that indicates it MUST be the same, but instead has a 25 
year old suggestive text, a MAY concept, which implies only the last one 
reply code is necessary and a 25 years wide practice to back it up,  I 
was proposing that we use this unique opportunity to embrace it for 
maximum impact.

There is no doubt, if it wasn't a concept that I could not envision as a 
real possibility and useful for developers now and down the road, I 
could easily see myself voting for a LOCK DOWN.

I am not proposing anything else but a "150-" as a PRELIMINARY reply 
code for this purpose. No mixing of two or more different 2xx, 3xx, 4xx, 
5xx lines.  If the server finds itself where it needs to "keep the 
channel" alive, or show it is still "working" it could issue the 
PRELIMINARY code 150- before any other COMPLETION code is finally 
determined and issued.

--
HLS

Date: Fri, 13 Apr 2007 13:57:56 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2181bis-01 Issue 20: General security considerations overhaul
Sender: [email-masked]

Hi.

Another issue from digging through the document.... This is
prior to trying to trying to post -02, but I don't expect
responses to make it into -02 unless they are very quick, very
clear, and represent a clear consensus.

RFC 3552 contained a fairly extensive analysis of what it
thought the Security Considerations section of RFC 2821 ought to
look like.   Some of those changes have already been
incorporated in 2821bis, but a number of the suggestions are
controversial or possibly just plain wrong.  An extended
discussion of some of the issues appears in the long-expired
I-D, draft-klensin-rfc2821-security-00.txt, which is now
referenced in the 2821bis-01 text.

I can repost that I-D if needed, but that seems worthwhile only
if we are going to turn every point it raises into an issue and
discuss it.  Otherwise, I need to delete the discussion that now
appears at the beginning of the Security Considerations section
and the reference.

Instructions?

    john

Date: Fri, 13 Apr 2007 18:24:09 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--zbGR4y+acU1DwHSi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-13 10:07:19 -0400, John Leslie wrote:
> Peter J. Holzer <[email-masked]> wrote:
> > On 2007-04-12 16:44:01 -0400, Robert A. Rosenberg wrote:
> >> At 15:20 -0400 on 04/11/2007, Tony Hansen wrote...
> >>>	(iii) Prohibit different codes and, optionally, suggest
> >>>	that it is ok for a client to select one of them and
> >>>	assume that all of the others are the same.
> >>=20
> >> Also, I do not like the optional "Pick One" authority in iii.
> >=20
> > That follows directly from the requirement that all codes MUST be the
> > same. If they are the same, it obviously doesn't matter which one the
> > client uses. If they are not the same, the server is violating the spec
> > and the client can do what it wants.
>=20
>    The server would indeed be violating the spec.
>=20
>    It does not follow that a cient may do what it wants.

If one side violates a MUST clause, we are IMHO out of the scope of
defined behaviour, unless the spec explicitely defines some error
behaviour. (Which RFC 2821 in general doesn't, unless I've missed it:
For example, what happens if a client sends a command with more than 512
characters, or a bare LF in data? The RFC doesn't say, some servers will
issue an error message, some will accept it, and some (badly written)
servers will crash or behave weirdly in some other way)

>    We're discussing a bug (IMHO) in 2821 which allows, in the case of
> multiline replies to DATA (phase 2), different clients to have different
> views of whether the message was accepted for delivery.

The bug is IMHO that it isn't completely clear whether all codes in a
multiline reply must be the same. Hector interpretes the current text to
mean that they can be different, while several other (including me)
interprete it to mean that they must be the same.

>    I, at least, would like to say something like, "Absent agreement
> between server and client on the meaning of different reply codes
> within a multiline reply, the server SHOULD make all the codes the
> same, and the client SHOULD generate an error if it receives
> different codes."

I would go for "the server MUST make all the codes the same" here. Is
there anything to be gained from this ambiguity?=20

At most I would add a warning that an extension could allow different
codes in multiline-replies.

> > Also, must the reply codes be exactly the same?
>=20
>    Yes, IMHO.
>=20
> > Consider something like:
> >=20
> > 250-Accepting message
> > 251 Forwarded to <[email-masked]>
>=20
>    IMHO, the server should know it will forward by the time the first
> line is issued.

I wasn't suggesting that it's a particularly bright idea for a server to
return that kind of message, but it's one of the edge cases which should
be considered once one starts to standardize behaviour in case of
protocol errors.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--zbGR4y+acU1DwHSi
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--zbGR4y+acU1DwHSi--

Date: Fri, 13 Apr 2007 10:07:19 -0400
From: John Leslie <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Sender: [email-masked]

Peter J. Holzer <[email-masked]> wrote:
> On 2007-04-12 16:44:01 -0400, Robert A. Rosenberg wrote:
>> 
>> At 15:20 -0400 on 04/11/2007, Tony Hansen wrote...
>> 
>>>	(iii) Prohibit different codes and, optionally, suggest
>>>	that it is ok for a client to select one of them and
>>>	assume that all of the others are the same.
>> 
>> Also, I do not like the optional "Pick One" authority in iii.
> 
> That follows directly from the requirement that all codes MUST be the
> same. If they are the same, it obviously doesn't matter which one the
> client uses. If they are not the same, the server is violating the spec
> and the client can do what it wants.

   The server would indeed be violating the spec.

   It does not follow that a cient may do what it wants.

   We're discussing a bug (IMHO) in 2821 which allows, in the case of
multiline replies to DATA (phase 2), different clients to have different
views of whether the message was accepted for delivery.

   I, at least, would like to say something like, "Absent agreement
between server and client on the meaning of different reply codes
within a multiline reply, the server SHOULD make all the codes the
same, and the client SHOULD generate an error if it receives
different codes."

>> you should verify that all the codes are the same and error out if
>> they are different.

   To that extent, I agree with Robert: it should not be legal for
clients to behave non-deterministically in the presence of known
server (mis)behavior -- especially in the case where there's no
opportunity to re-synchronize.

> If such a requirement is written into the RFC, it should also describe
> what the the client is supposed to do in case of different codes.

   Here, I agree with Peter: I'm just not sure what to recommend.

> Permanent error? Temporary error?

   At first blush, a permanent error seems more likely to help.

> Also, must the reply codes be exactly the same?

   Yes, IMHO.

> Consider something like:
> 
> 250-Accepting message
> 251 Forwarded to <[email-masked]>

   IMHO, the server should know it will forward by the time the first
line is issued.

   (YMMV...)

--
John Leslie <[email-masked]>

Date: Fri, 13 Apr 2007 10:46:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Finch wrote:

>On Wed, 11 Apr 2007, Tony Hansen wrote:
>  
>
>>	(iii) Prohibit different codes and, optionally, suggest
>>	that it is ok for a client to select one of them and
>>	assume that all of the others are the same.
>>    
>>
>http://www.imc.org/ietf-smtp/mail-archive/msg03247.html
>
>My reasons are that (iia) and (iib) are unacceptable because of running
>code, and (i) has lead to interoperability problems which I understand
>this revision effort aims to minimise.
>  
>
+1.

Date: Fri, 13 Apr 2007 09:35:47 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--7AUc2qLy4jB3hD7Z
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-12 16:44:01 -0400, Robert A. Rosenberg wrote:
>=20
> At 15:20 -0400 on 04/11/2007, Tony Hansen wrote about Re: recap=20
> rfc2821bis-01 Issue 17: all contination lines mus:
>=20
> >	(i) Do nothing, leaving the text as is
> >	(ii-a) Make it clear that the codes may be different and
> >	that clients are expected to respect the *last* value listed
> >	(ii-b) Make it clear that the codes may be different and
> >	that clients are expected to respect the *first* value listed
> >	(iii) Prohibit different codes and, optionally, suggest
> >	that it is ok for a client to select one of them and
> >	assume that all of the others are the same.
>=20
[...]
>=20
> Also, I do not like the optional "Pick One" authority in iii.

That follows directly from the requirement that all codes MUST be the
same. If they are the same, it obviously doesn't matter which one the
client uses. If they are not the same, the server is violating the spec
and the client can do what it wants.

> If you are going to ignore the LAST result (the only valid one in a
> "Here is my result" case when the prior replies are intermediate
> status updates [as in the 150 case] as opposed to a single
> continuation reply such as the EHLO 250)

(iii) doesn't allow intermediate status updates, so this is moot.

> you should verify that all the codes are the same and error out if
> they are different.

If such a requirement is written into the RFC, it should also describe
what the the client is supposed to do in case of different codes.
Permanent error? Temporary error? Also, must the reply codes be exactly
the same? Consider something like:

250-Accepting message
251 Forwarded to <[email-masked]>

A client looking at either the first or the last code sees a 2xx code
and concludes that the message has been accepted. A client which
compares the codes must decide whether 250 and 251 are "equal enough"
and if they aren't what kind of error it should report.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--7AUc2qLy4jB3hD7Z
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--7AUc2qLy4jB3hD7Z--

Date: Thu, 12 Apr 2007 16:44:01 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use  same code
Sender: [email-masked]

At 15:20 -0400 on 04/11/2007, Tony Hansen wrote about Re: recap 
rfc2821bis-01 Issue 17: all contination lines mus:

>Please state your preference for one of the following. (I've broken
>John's (ii) into two parts.)
>
>	(i) Do nothing, leaving the text as is
>	(ii-a) Make it clear that the codes may be different and
>	that clients are expected to respect the *last* value listed
>	(ii-b) Make it clear that the codes may be different and
>	that clients are expected to respect the *first* value listed
>	(iii) Prohibit different codes and, optionally, suggest
>	that it is ok for a client to select one of them and
>	assume that all of the others are the same.

While I can see a reason for splitting ii into ii-a and ii-b so as to 
document the two options, I do feel that a usage example for ii-b 
should provided to should a valid case where ii-b is justified (as 
was done for the 150-I'm_working_on_it case that was associated with 
ii-a). If none can be provided, I think the option list should be 
trimmed back to i/ii/iii with the ii having the proposed ii-a text 
(IOW: The codes can differ BUT the last one is the one to use) 
possibly saying that the continued codes should be parsed not just 
ignored.

Also, I do not like the optional "Pick One" authority in iii. If you 
are going to ignore the LAST result (the only valid one in a "Here is 
my result" case when the prior replies are intermediate status 
updates [as in the 150 case] as opposed to a single continuation 
reply such as the EHLO 250) you should verify that all the codes are 
the same and error out if they are different.

IOW: I vote for a wording that makes EXPLICATE that the last reply 
(in a status vs. list case) is the only VALID reply and that prior 
replies must not be substituted for it in any case where different 
codes are permitted. I thus vote for either ii-a or iii (but iii only 
with the "optional" permission REQUIRING that the others be validated 
for consistency - IOW: If you pick the non-Last code, you must not 
ignore the others by not checking for consistency and just ASSUME 
they are/will-be the same).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Explained-literal
Date:  Thu, 12 Apr 2007 22:16:56 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

John C Klensin wrote:

> I'm happy to assign an issue number if others believe the
> decision needs revisiting.

Please add it to the list of things that have to be checked
thoroughly in the implementation and interoperability report.
I hope that EXPN and VRFY are already on this list.  

If the result is that <explained-literal> isn't used or is
in practice unusable it could be removed.  That would be sad.

Frank

Date: Thu, 12 Apr 2007 15:33:35 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Lisa Dusseault <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Hi Lisa,

Thank you for the clarification here and in your post regarding I-Ds.

It is my engineering opinion, that we can have new "clarification" 
statements for existing 2821 statements which will not alter its meaning 
(as it is currently stated), nor change 25 years of wide practice based 
on the statement that is currently stated, yet provide the opportunity 
for new development to continue with legitimacy, and also not force 
certain undesirable operations (post smtp processing with accept/bounce 
exploit potential).

A lock down will pretty much force POST-SMTP processing for the new 
augmenting technologies currently under way.

While an extension is conceivable, it will not have provide the maximum 
impact for legacy operations as would adding new clarifications will have.

With a lock down, the extension will come in one of two flavors:

   1) "break" (update) the new lock down statements being
       considered for 2821bis:

        - All reply codes in multi-line responses must be the same,
        - "1yz-" must not be used in standard 2821 operations.

   2) Introduce new response codes which implies new client support
      (no legacy support).

#1 will have the highest impact, but I don't think it is desirable to do 
break or change existing protocol (if it was written in 2821bis).

#2 would be the more practical approach, but will not provide the 
highest impact.

So at this point, I agree with Hansen the "do nothing" conservative 
approach is probably the best approach.  It will at least allow for 
extension #1 to have a better change of being proposed.

Yes, if need be, I would be more than happy to author an extension. How 
it is structure will depend on whats decided here.

Thanks for your consideration.

--
HLS

Lisa Dusseault wrote:
> Hi Hector,
> 
> John's got my support and Tony's to reject new capabilities, extensions, 
> etc in this round of changes, and he's not enforcing that 
> preferentially.  In fact there's strong process pressure to reject any 
> additions in order to achieve the status of Draft Standard, so these are 
> IETF rules, not John's rules.
> 
> Do you disagree with John on the basis that your proposed change is too 
> important to be put off ? or that it's not a new capability but merely 
> a  clarification?
> 
> If it's a new feature, please do consider writing up SMTP extension 
> proposals in I-D form -- the fact that John's working on 2821bis should 
> not prevent people from working on useful extensions.  The two efforts 
> can and should be separated as far as we can tell.
> 
> Lisa
> 
> On Apr 10, 2007, at 7:04 AM, Hector Santos wrote:
> 
>>> You might be able to overcome some of these difficulties in an
>>>
>>> extension, but it certainly does not belong in the base spec.
>>>
>>
>> I disagree but I defer to your rules. Thats too bad because I can't 
>> help but feel if someone else would had brought this up, you would 
>> take a more open minded consideration. I am sorry to say that only 
>> because I find it extremely odd that technical clarity in this case is 
>> not taken serious when in reality the less serious nits are being 
>> debated else where. The odds are very good systems you will be 
>> confronted with this ISSUE more so than some of the other issues that 
>> essentially have little to no impact in improving the system.
>>
> 

Date: Thu, 12 Apr 2007 20:02:19 +0100
From: Tony Finch <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
cc: [email-masked]
Subject: Re: Explained-literal
Sender: [email-masked]

On Thu, 12 Apr 2007, Mark E. Mallett wrote:
>
> I'm guessing the choice was either remove it from the narrative or
> add it to the BNF.  Flipping the coin the other way might be better.

Yes, I think the ABNF should stay the same as 2821 and the "optionally
followed by information that will help to identify the client system"
clause should be removed.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT: VARIABLE 2 OR 3, BECOMING EASTERLY 4. SLIGHT OCCASIONALLY
MODERATE. FOG PATCHES. MODERATE OR GOOD, OCCASIONALLY VERY POOR.

Date: Thu, 12 Apr 2007 14:53:23 -0400
From: John C Klensin <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>, Tony Finch <[email-masked]>
cc: [email-masked]
Subject: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO (was:  Explained-literal)
Sender: [email-masked]

p.s. the reason why there isn't any syntax or explicit
requirement to incorporate that text into the relevant trace
field is because, as far as I recall, no one suggested it, much
less convinced others that it was important.

I'm sometimes just the clerk/editor here.

    john

Date: Thu, 12 Apr 2007 14:50:37 -0400
From: John C Klensin <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>, Tony Finch <[email-masked]>
cc: [email-masked]
Subject: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO (was:  Explained-literal)
Sender: [email-masked]

--On Thursday, 12 April, 2007 14:28 -0400 "Mark E. Mallett"
<[email-masked]> wrote:

> 
> On Thu, Apr 12, 2007 at 07:08:43PM +0100, Tony Finch wrote:
>> 
>>    ehlo              = "EHLO" SP ( Domain / Explained-literal
>>    ) CRLF
>> 
>>    Explained-literal = address-literal [string]
>> 
>> Um, boggle. Why was this new syntax introduced?
> 
> I'm guessing the choice was either remove it from the
> narrative or add it to the BNF.

yes, probably that too, since the relevant narrative hasn't
changed significantly from 2821.

>  Flipping the coin the other way might be better.

No opinion, but you are the second person to speak up, so I'm
assigning Issue 19 to this.

   john

Date: Thu, 12 Apr 2007 14:44:37 -0400
From: John C Klensin <[email-masked]>
To: Tony Finch <[email-masked]>, [email-masked]
Subject: Re: Explained-literal
Sender: [email-masked]

--On Thursday, 12 April, 2007 19:08 +0100 Tony Finch
<[email-masked]> wrote:

> 
>    ehlo              = "EHLO" SP ( Domain / Explained-literal
> ) CRLF
> 
>    Explained-literal = address-literal [string]
> 
> Um, boggle. Why was this new syntax introduced? Clients that
> use it are going to fail to negotiate ESMTP even though both
> the client and the server support it, because existing servers
> (sendmail, exim, exchange) will reject it as an error. This
> new feature is also incomplete - there's no syntax for
> including the string explanation in a Received: trace field.

This was introduced after a lengthy discussion that, according
to my notes, occurred in late 2003.  I apparently sent a note to
this list confirming it on 20040103.  The issue is discussed
early in 4.1.1.1 and has to do with the desire to permit the
client to attach some free text explaining why a literal is
being used rather than a domain name.

I'm happy to assign an issue number if others believe the
decision needs revisiting.

    john

Date: Thu, 12 Apr 2007 14:35:56 -0400
From: John C Klensin <[email-masked]>
To: Jeff Macdonald <[email-masked]>
cc: [email-masked]
Subject: Use of code 421 (was: Re: rfc2821bis-01 Issue 17: all contination lines must use same code)
Sender: [email-masked]

--On Thursday, 12 April, 2007 14:02 -0400 Jeff Macdonald
<[email-masked]> wrote:

> On Tue, Apr 10, 2007 at 11:24:18AM -0400, Jeff Macdonald wrote:
>> 
>> On Tue, Apr 10, 2007 at 10:40:52AM -0400, John C Klensin
>> wrote:
>> > 	(iii) Prohibit different codes and, optionally, suggest
>> > 	that it is ok for a client to select one of them and
>> > 	assume that all of the others are the same.
>> 
>> +1
>> 
>> While some may believe the current spec is clear enough, I've
>> seen individuals outside of this group that have been
>> confused as to what is valid and I've seen multi-line
>> responses were different codes have been used. So, the
>> current spec isn't clear enough IMO. I'll try to dig up some
>> actual examples.
> 
> I have found only 1 live example of this:
> 
> 550-Unknown user
> 421 clean.daytoncreative.net: SMTP command timeout - closing
> connection
> 
> It is unclear to me what is happening here. 421 can happen at
> any time, but is it ok as part of a multi-line response? I
> think not.

Again, just personal opinion, but, if I'm designing a server, I
never want to issue a 421 code (or any other 4yz code), in a
situation in which I want the sender to take that particular
message and go away and not come back with it -- at least
without making some significant changes.  421 is effectively a
"try again later" request, which I wouldn't want to send out
unless I really want to see that message, with that content and
recipient list, again.

In addition, if the client does look only at the code on the
last line of the multiline response, the "unknown user"
information is going to get lost entirely.

> I also have examples of multiple responses to a command. This
> is not directly related to multi-line responses, but I'd like
> to point it out. I believe the RFC allows this behaviour, but
> there seems to be two ways folks handle rejecting content:
> 
> C: \r\n.\r\n <-- end of data
> S: 5xx
> S: 421

Well, this would, as Tony Finch pointed out, be construed by a
client that wasn't aware of the games that were being played as
a 5xx response to DATA and a 421 response to whatever command
was issued next.  As such, it is a courtesy before the server
closes the connection -- if the client sends out a command or
otherwise opens a read and gets the 421, it at least knows that
the server tried to warn it before closing.  If it doesn't, not
much harm done.

There is specific language in 2821 that authorizes asynchronous
issuing of a 421 if necessary and it incorporates exactly that
logic.

Note that using 421 that way is pushing the boundaries of 2821
fairly hard: unless the server thinks that "bad content"
constitutes an attack against which it must be defended --and
thinks that this defends it -- a server close without a QUIT
violates a strong prohibition in the spec.
 
> and
> 
> C: \r\n.\r\n <-- end of data
> S: 421-<long explanation>
> S: 421
> 
> I think it is out of scope, but should the RFC make a
> recommendation either way?

Speaking personally, I'd prefer not.  What we have now is a
general prohibition on server closes and a fairly general
loophole about self-defense.  If we start making
recommendations, we are going to need to make one or the other
(or both) much more specific.  I suspect that trying to do that
would lead us through a whole sequence of previously-unexplored
ratholes.

    john

From: "Mark E. Mallett" <[email-masked]>
Date: Thu, 12 Apr 2007 14:28:42 -0400
To: Tony Finch <[email-masked]>
Cc: [email-masked]
Subject: Re: Explained-literal
Sender: [email-masked]

On Thu, Apr 12, 2007 at 07:08:43PM +0100, Tony Finch wrote:
> 
>    ehlo              = "EHLO" SP ( Domain / Explained-literal ) CRLF
> 
>    Explained-literal = address-literal [string]
> 
> Um, boggle. Why was this new syntax introduced?

I'm guessing the choice was either remove it from the narrative or
add it to the BNF.  Flipping the coin the other way might be better.

mm

Cc: Hector Santos <[email-masked]>, "David F. Skoll" <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Date: Thu, 12 Apr 2007 11:14:37 -0700
To: John C Klensin <[email-masked]>
Sender: [email-masked]

--Apple-Mail-4-525982201
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

On Apr 11, 2007, at 11:19 AM, John C Klensin wrote:

>> If I understand the question, you are saying that an I-D
>> proposal can update and even go as far as change an RFC and my
>> "assumption" that it can not is an incorrect assumption?
>
> An I-D cannot.

Just to be clear, an I-D can *propose* to change an RFC.  If it's  
accepted as an RFC, then the new RFC is marked as updating the old  
RFC.  It's not necessary to replace an RFC completely in order to  
change some detail in it.

For example, RFC 3501 -- IMAP4 -- obsoletes RFC 2060.  We no longer  
recommend to read or implement 2060 at all, we recommend to read and  
implement RFC3501.  This kind of replacement is what John is doing.

The other option is to update an older RFC. For example, RFC4466  
(collected ABNF changes) updates RFC3501, and the RFC Index says so  
(http://www.ietf.org/iesg/1rfc_index.txt).  An implementor working on  
RFC3501 also has to read RFC4466 to see what has been changed.   
Sometimes extensions are forced to make changes in the protocol  
they're extending and these changes are registered as updates so  
people can tell.

Lisa
--Apple-Mail-4-525982201
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=ISO-8859-1

<HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode: space; =
-khtml-line-break: after-white-space; "><BR><DIV><DIV>On Apr 11, 2007, =
at 11:19 AM, John C Klensin wrote:</DIV><BR =
class=3D"Apple-interchange-newline"><BLOCKQUOTE type=3D"cite"><BLOCKQUOTE =
type=3D"cite"><P style=3D"margin: 0.0px 0.0px 0.0px 10.0px"><FONT =
face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px Helvetica">If I =
understand the question, you are saying that an I-D</FONT></P> <P =
style=3D"margin: 0.0px 0.0px 0.0px 10.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">proposal can update and even =
go as far as change an RFC and my</FONT></P> <P style=3D"margin: 0.0px =
0.0px 0.0px 10.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: =
12.0px Helvetica">"assumption" that it can not is an incorrect =
assumption?</FONT></P> </BLOCKQUOTE><P style=3D"margin: 0.0px 0.0px =
0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px"><BR></P> <P =
style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">An I-D cannot.<SPAN =
class=3D"Apple-converted-space">=A0</SPAN></FONT></P> =
</BLOCKQUOTE></DIV><BR><DIV>Just to be clear, an I-D can *propose* to =
change an RFC.=A0 If it's accepted as an RFC, then the new RFC is marked =
as updating the old RFC.=A0 It's not necessary to replace an RFC =
completely in order to change some detail in it.=A0</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>For example, RFC 3501 -- =
IMAP4 -- obsoletes RFC 2060.=A0 We no longer recommend to read or =
implement 2060 at all, we recommend to read and implement RFC3501.=A0 =
This kind of replacement is what John is doing.</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>The other option is to =
update an older RFC. For example, RFC4466 (collected ABNF changes) =
updates RFC3501, and the RFC Index says so (<A =
href=3D"http://www.ietf.org/iesg/1rfc_index.txt">http://www.ietf.org/iesg/=
1rfc_index.txt</A>).=A0 An implementor working on RFC3501 also has to =
read RFC4466 to see what has been changed.=A0 Sometimes extensions are =
forced to make changes in the protocol they're extending and these =
changes are registered as updates so people can tell.</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>Lisa</DIV></BODY></HTML>=

--Apple-Mail-4-525982201--

Date: Thu, 12 Apr 2007 19:12:29 +0100
From: Tony Finch <[email-masked]>
To: Jeff Macdonald <[email-masked]>
cc: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Thu, 12 Apr 2007, Jeff Macdonald wrote:

> S: 5xx
> S: 421

In this case the 421 is effectively an early response to the client's next
command (whatever that may be - the server clearly doesn't care).

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH FITZROY SOLE: VARIABLE 2 OR 3, OCCASIONALLY 4. SLIGHT OR MODERATE. FOG
PATCHES. GOOD, OCCASIONALLY VERY POOR.

Date: Thu, 12 Apr 2007 19:08:43 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: Explained-literal
Sender: [email-masked]

   ehlo              = "EHLO" SP ( Domain / Explained-literal ) CRLF

   Explained-literal = address-literal [string]

Um, boggle. Why was this new syntax introduced? Clients that use it are
going to fail to negotiate ESMTP even though both the client and the
server support it, because existing servers (sendmail, exim, exchange)
will reject it as an error. This new feature is also incomplete - there's
no syntax for including the string explanation in a Received: trace field.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTH FITZROY: NORTHERLY 4 OR 5, OCCASIONALLY 6 AT FIRST. SLIGHT OR MODERATE.
THUNDERY SHOWERS. GOOD.

Date: Thu, 12 Apr 2007 14:02:46 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, Apr 10, 2007 at 11:24:18AM -0400, Jeff Macdonald wrote:
> 
> On Tue, Apr 10, 2007 at 10:40:52AM -0400, John C Klensin wrote:
> > 	(iii) Prohibit different codes and, optionally, suggest
> > 	that it is ok for a client to select one of them and
> > 	assume that all of the others are the same.
> 
> +1
> 
> While some may believe the current spec is clear enough, I've seen
> individuals outside of this group that have been confused as to what is
> valid and I've seen multi-line responses were different codes have been
> used. So, the current spec isn't clear enough IMO. I'll try to dig up
> some actual examples.

I have found only 1 live example of this:

550-Unknown user
421 clean.daytoncreative.net: SMTP command timeout - closing connection

It is unclear to me what is happening here. 421 can happen at any time,
but is it ok as part of a multi-line response? I think not.

I also have examples of multiple responses to a command. This is not
directly related to multi-line responses, but I'd like to point it out.
I believe the RFC allows this behaviour, but there seems to be two ways
folks handle rejecting content:

C: \r\n.\r\n <-- end of data
S: 5xx
S: 421

and

C: \r\n.\r\n <-- end of data
S: 421-<long explanation>
S: 421

I think it is out of scope, but should the RFC make a recommendation
either way?

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Cc: John C Klensin <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Date: Thu, 12 Apr 2007 10:56:49 -0700
To: Hector Santos <[email-masked]>
Sender: [email-masked]

--Apple-Mail-2-524914030
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

Hi Hector,

John's got my support and Tony's to reject new capabilities,  
extensions, etc in this round of changes, and he's not enforcing that  
preferentially.  In fact there's strong process pressure to reject  
any additions in order to achieve the status of Draft Standard, so  
these are IETF rules, not John's rules.

Do you disagree with John on the basis that your proposed change is  
too important to be put off ? or that it's not a new capability but  
merely a  clarification?

If it's a new feature, please do consider writing up SMTP extension  
proposals in I-D form -- the fact that John's working on 2821bis  
should not prevent people from working on useful extensions.  The two  
efforts can and should be separated as far as we can tell.

Lisa

On Apr 10, 2007, at 7:04 AM, Hector Santos wrote:

>> You might be able to overcome some of these difficulties in an
>> extension, but it certainly does not belong in the base spec.
>
> I disagree but I defer to your rules. Thats too bad because I can't  
> help but feel if someone else would had brought this up, you would  
> take a more open minded consideration. I am sorry to say that only  
> because I find it extremely odd that technical clarity in this case  
> is not taken serious when in reality the less serious nits are  
> being debated else where. The odds are very good systems you will  
> be confronted with this ISSUE more so than some of the other issues  
> that essentially have little to no impact in improving the system.

--Apple-Mail-2-524914030
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=ISO-8859-1

<HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode: space; =
-khtml-line-break: after-white-space; "><DIV>Hi Hector,</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>John's got my support and =
Tony's to reject new capabilities, extensions, etc in this round of =
changes, and he's not enforcing that preferentially.=A0 In fact there's =
strong process pressure to reject any additions in order to achieve the =
status of Draft Standard, so these are IETF rules, not John's =
rules.</DIV><DIV><BR class=3D"khtml-block-placeholder"></DIV><DIV>Do you =
disagree with John on the basis that your proposed change is too =
important to be put off ? or that it's not a new capability but merely =
a=A0 clarification?</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>If it's a new feature, =
please do consider writing up SMTP extension proposals in I-D form -- =
the fact that John's working on 2821bis should not prevent people from =
working on useful extensions.=A0 The two efforts can and should be =
separated as far as we can tell.</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>Lisa</DIV><BR><DIV><DIV>On =
Apr 10, 2007, at 7:04 AM, Hector Santos wrote:</DIV><BR =
class=3D"Apple-interchange-newline"><BLOCKQUOTE type=3D"cite"><BLOCKQUOTE =
type=3D"cite"><P style=3D"margin: 0.0px 0.0px 0.0px 10.0px"><FONT =
face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px Helvetica">You might =
be able to overcome some of these difficulties in an</FONT></P> <P =
style=3D"margin: 0.0px 0.0px 0.0px 10.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">extension, but it certainly =
does not belong in the base spec.</FONT></P> </BLOCKQUOTE><P =
style=3D"margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; =
min-height: 14.0px"><BR></P> <P style=3D"margin: 0.0px 0.0px 0.0px =
0.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px =
Helvetica">I disagree but I defer to your rules. Thats too bad because I =
can't help but feel if someone else would had brought this up, you would =
take a more open minded consideration. I am sorry to say that only =
because I find it extremely odd that technical clarity in this case is =
not taken serious when in reality the less serious nits are being =
debated else where. The odds are very good systems you will be =
confronted with this ISSUE more so than some of the other issues that =
essentially have little to no impact in improving the system.</FONT></P> =
</BLOCKQUOTE></DIV><BR></BODY></HTML>=

--Apple-Mail-2-524914030--

Date: Thu, 12 Apr 2007 13:38:01 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue  17: multiline reply codes
Sender: [email-masked]

--On Thursday, 12 April, 2007 11:20 -0400 Tony Hansen
<[email-masked]> wrote:

>...
> This thread is supposed to be about the wording of 1yz's
> description. It has wandered way off course back to Issue 17
> (different reply codes in multiline responses).
>...
> Issue 17, the possibility of different reply codes being used
> in multiline responses and the question of which one should be
> believed, is an example of such an ambiguity. Some people have
> taken the sentence beginning "in many cases..." to be
> normative and others not. To me, the wording of "in many
> cases" is equivalent to a MAY, which is also equivalent to a
> MAY NOT. Given that there *are* implementations that have
> chosen both interpretations, and we can't remove multiline
> responses, I'm now convinced that we're going to need to take
> the conservative approach.

I think one thing that is pushing me toward a hard line on this
is that I can take "in many cases..." to mean, not MAY (or MAY
NOT), but that there are some cases in which that would be an
unsafe strategy.  In other words, it is (possibly or probably)
wise for a client to not do this unless it somehow knows,
presumably out of band, whether the strategy is safe, even if it
would be ok in many (or most) cases.

This is why it is important to remember that the text in
question is original 821 material, not something that was
introduced by DRUMS into 2821.   RFCs were written very
differently in 1981-82 than they were even a decade later, much
less nearly two decades later.  In that period, the community
made the shift between a design specification for comment,
implementation, and more comment to standards with conformance
criteria.  We had little systematic history of talking regularly
about conformance and mandatory features prior to RFC 1122 and
1123.

And it just makes little sense to try to split hairs over the
exact conformance meaning or interpretation of a comment in an
1982 document that says "in many cases".  Had that text been
written in 2001, the situation would be different (and, in all
likelihood, that language would have been changed to a 2119-like
statement during Last Call if not earlier).

best,
   john

From: "Mark E. Mallett" <[email-masked]>
Date: Thu, 12 Apr 2007 13:18:22 -0400
To: [email-masked]
Subject: Explained-literal
Sender: [email-masked]

Just to strike a different note in here:

4.1.1.1 says

   Explained-literal  = address-literal [string]

Capitalization aside, shouldn't this be

   Explained-literal  = address-literal SP [string]

or

   Explained-literal  = address-literal 1*SP [string]

?  Because I think 

EHLO [127.0.0.1]"a horse with no name"

is messy.

This may have been discussed already, but I didn't find it if so.

(Not that I can find many implementations in the wild, other than those
that do no syntax checking at all, that support explained-literal, even
though it was described in the narrative in 2821.)

Unrelatedly, I have a handful of other nitpicky comments that seem
inappropriate for the "one message per issue" request.  Maybe somebody
would rather see and vet them off-list?

Yours,
-mm-

Date: Thu, 12 Apr 2007 12:52:05 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Window of opportunity
Sender: [email-masked]

<pseudo chair hat on>

The ship for new features to *821 sailed with 2821. It's not going to
happen within 2821bis.

</pseudo chair hat off>

	Tony Hansen
	[email-masked]

Frank Ellermann wrote:
> Hector Santos wrote:
> 
>> We are not going to see another 2821bis for a long time. The window of
>> opportunity is now with 2821bis.
> 
> Hi Hector, I don't see it as window of opportunity for the introduction
> of NEW features.  

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Window of opportunity (was: rfc2821bis-01 Issue 18: Usability of 1yz replies)
Date:  Thu, 12 Apr 2007 18:23:09 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 54
Sender: [email-masked]

Hector Santos wrote:

> We are not going to see another 2821bis for a long time. The window of
> opportunity is now with 2821bis.

Hi Hector, I don't see it as window of opportunity for the introduction
of NEW features.  

Otherwise I'd propose to move in a mandatory FAIL rejection at the MX, 
considering an emulation of the RFC 821 reverse path as more important
than anything else, TLDs trying to be hosts, servers trying to convince
broken clients to stay online, or other less important features.

The window of opportunity is IMO limited to mitigating the worst effects
of killing the reverse path while keeping 1123 5.3.6(a).  If the new 
2821bis has it clear that border MTAs cannot accept mail if they've no
clue how to return it if forwarding it fails, I'm fine with it.  And of
course 2821bis is an opportunity to fix known errors and ambiguities.

The 1yz and multiline stuff is a good example for this.  Maybe you've
seen that I supported (2a), in the form of a "SHOULD NOT be different",
that's very close to the almost unanimous winner (3).

And that was before somebody posted the very plausible recipe "collect
all reply lines into a buffer and take the first three digits as result"
corresponding to (2b).

As John wrote your concept should actually get a new "000- ignore me"
class of error codes, not 1yz, and adding this 000- trick would be a
NEW feature. 

IMO 2821bis should be a PS if it doesn't make every compatible effort to
explain drastically what "originator as indicated in the reverse path"
actually means for a border MTA.  Certainly _not_ "accept and discard",
IIRC proposed as possible "receiver policy" in the same (2b) article.

But I also think that this is the ONLY issue justifying a "recycle at
PS" strategy, if it's not addressed adequatly in a language understood
by any postmaster.  

For the 000- idea (or any variant of it you pick), that's really a job
for a separate RFC, like the I-Ds about "selective reject" (indirectly
related to the bogus bounce issue), a status code registry, UTF8SMTP,
Alexey's LANG proposal, the quickstart QHLO, etc.

When you propose to add 000- (or similar) to the core spec. without an
extension resulting in a mutual agreement about it, it would break all
clients doing something with different codes in a mutiline reply that
you don't expect.  That would be no "opt-in", it would be brute force.

I'd have better ideas than 000- if brute force would be acceptable...

Frank

Date: Thu, 12 Apr 2007 11:20:42 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies, and Issue 17: multiline reply codes
Sender: [email-masked]

<pseudo chair hat on>

This thread is supposed to be about the wording of 1yz's description. It
has wandered way off course back to Issue 17 (different reply codes in
multiline responses).

Time to jump up a few thousand feet.

When moving a protocol from Proposed Standard to Draft Standard, one of
the things you need to do is look for, find and stamp out ambiguities in
the spec that have led to different implementations. When such an
ambiguity is found, either the offending portion of the protocol must be
removed from the spec or the wording changed to clarify things. A goal
is that an implementation of the old RFC should work the same under the
new RFC. Sometimes this requires a very conservative approach.

Issue 17, the possibility of different reply codes being used in
multiline responses and the question of which one should be believed, is
an example of such an ambiguity. Some people have taken the sentence
beginning "in many cases..." to be normative and others not. To me, the
wording of "in many cases" is equivalent to a MAY, which is also
equivalent to a MAY NOT. Given that there *are* implementations that
have chosen both interpretations, and we can't remove multiline
responses, I'm now convinced that we're going to need to take the
conservative approach.

If you wish to continue discussing Issue 17 (different reply codes in
multiline responses), please respond in that thread or at least change
the subject.

</pseudo chair hat off>

	Tony Hansen
	[email-masked]

Date: Thu, 12 Apr 2007 11:03:26 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

[...]

> All I am saying is closing it, locking it down, will hurt new
> development, not help it.

How will it hurt new development?  Closing this loophole will spur
development of *correct* keepalive and/or checkpointing extensions.  It
will prevent dubious practices from being enshrined as acceptable.

http://www.martinfowler.com/bliki/TechnicalDebt.html

Regards,

David.

Date: Thu, 12 Apr 2007 10:34:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Look David,

I am not disagreeing with any points being made here, including some of 
yours.  Foremost, I am 100% for SMTP technical protocol consistency and 
compliance.

In this case, we have an unique opportunity to embrace and legitimize 
what you call a "hack" to not only clarify what is already inherently a 
25 year old wide practice (last reply code) but to also use it to 
establish the necessary framework for inevitable pending new technology 
that will, without a doubt, confront this very issue.

So we can either embrace it or close what you call a "hack."

All I am saying is closing it, locking it down, will hurt new 
development, not help it.  The philosophy differences between the 
dynamic vs post smtp processing people is a non-starter. So need to 
bring that up.

--
HLS

David F. Skoll wrote:
> Hector Santos wrote:
> 
> [...]
> 
>> But when you are confronted with a problem, you need to sometimes to
>> think out of the box, to explore, and then, see, as in this case, if you
>> can make it work with the current system.  That would be ideal.
> 
> Your solution to the SMTP timeout problem was certainly creative and
> interesting, but nevertheless, it is a hack.  By "hack", I mean something
> clever that seems to work well in practice, but is inelegant and unworthy
> of being raised to the status of an Internet standard.
> 
>> Hence, IMV, I believe using 150- for SMTP to resolve a the potential
>> TIMEOUT issue with backward compatibility is a valuable and viable
>> solution.
> 
> Actually, the problem is being approached incorrectly.  If a client is
> timing out frequently, it either means (1) the client's timeout is too
> short, or (2) the server is too slow or is ignoring the RFC's
> statement that it "MUST seek to minimize the time required to respond
> to the final <CRLF>.<CRLF>"  Note that there's a MUST in there, not
> a SHOULD.
> 
> So I don't believe hacking in a fix to alleviate the symptoms is the
> right approach; the proper approach is to fix the problem.  If online
> scanning is taking too long, then IMO the server SHOULD accept the mail
> with a 2xx code and do offline scanning.  It's then a matter of policy
> whether the server should generate an NDN or simply discard unacceptable
> messages.
> 
> I suspect the primary motivation for a keepalive is to prevent MUAs
> like Outlook from timing out when doing message submission.  If
> timeouts are a problem in this situation, then the correct solution is
> to rearchitect the network so MUAs submit to a dedicated (and fast)
> submission machine, with any content-scanning happening offline or on
> a separate content-scanning relay.  In this constrained environment
> (especially if you're using SMTP AUTH), it makes sense to generate
> bounce messages without fearing backscatter.
> 
> Regards,
> 
> David.
> 
> 
> 

Date: Thu, 12 Apr 2007 08:57:03 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

[...]

> But when you are confronted with a problem, you need to sometimes to
> think out of the box, to explore, and then, see, as in this case, if you
> can make it work with the current system.  That would be ideal.

Your solution to the SMTP timeout problem was certainly creative and
interesting, but nevertheless, it is a hack.  By "hack", I mean something
clever that seems to work well in practice, but is inelegant and unworthy
of being raised to the status of an Internet standard.

> Hence, IMV, I believe using 150- for SMTP to resolve a the potential
> TIMEOUT issue with backward compatibility is a valuable and viable
> solution.

Actually, the problem is being approached incorrectly.  If a client is
timing out frequently, it either means (1) the client's timeout is too
short, or (2) the server is too slow or is ignoring the RFC's
statement that it "MUST seek to minimize the time required to respond
to the final <CRLF>.<CRLF>"  Note that there's a MUST in there, not
a SHOULD.

So I don't believe hacking in a fix to alleviate the symptoms is the
right approach; the proper approach is to fix the problem.  If online
scanning is taking too long, then IMO the server SHOULD accept the mail
with a 2xx code and do offline scanning.  It's then a matter of policy
whether the server should generate an NDN or simply discard unacceptable
messages.

I suspect the primary motivation for a keepalive is to prevent MUAs
like Outlook from timing out when doing message submission.  If
timeouts are a problem in this situation, then the correct solution is
to rearchitect the network so MUAs submit to a dedicated (and fast)
submission machine, with any content-scanning happening offline or on
a separate content-scanning relay.  In this constrained environment
(especially if you're using SMTP AUTH), it makes sense to generate
bounce messages without fearing backscatter.

Regards,

David.

Date: Thu, 12 Apr 2007 08:22:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Thanks for your comment.

First let me say, as a problem solver and modern developer, we are 
always looking for solutions to today issues, problems and future issues 
as you can come about.

Clearly, one would naturally assume the prefix numbers would be the 
same. Sorta born from a EDI or PUNCH code mentality with a 5 digit 
numeric column, with a 6th column, if punched, relates to a 
continuation.  Anyone with development experience will know that in 
principle.

But when you are confronted with a problem, you need to sometimes to 
think out of the box, to explore, and then, see, as in this case, if you 
can make it work with the current system.  That would be ideal.

To put this into context, the "idea" was born from a real timeout issue 
with FTP client software.  We needed a solution PRONTO to solve large 
file upload backend processing with our FTP hosting system specializing 
in the Medical, Insurance, Payroll business.

The problem was solved with a similar "150-" continuation line concept 
to keep the FTP client alive which has a DOCUMENTED behavior of using a 
~30 second SOCKET timeout to wait on the completed file upload server 
response.  That was nearly 6 years ago.  Keep in mind that making this a 
server driven solution was vital and the customer client base was wide 
and had no control over the FTP client (i.e., we could not mandate a 
client change).

Move to SMTP, with the advent of AVS solution adventures and other SMTP 
level backend processing needs, it was clearly envisioned similar 
timeout issues would be very possible.

The idea was tested for SMTP but not in a full blown effort.

I believe it was IETF OPES WG where I brought up the potential need for 
a timeout solution for these new "HOOKS" or "SHIMS" or "PROXY" 
processing ideas for SMTP.  It was the first time where we had an 
"official IETF" incentive to research a similar solution for SMTP.  That 
was done and the proof of concept was tested and confirmed with various 
MUAs and a few MTAs, including our own.  The "150-" was also a solution 
for SMTP to prevent client timeouts.

I should note, that I checked using the "same code" idea first but found 
that didn't work well because some of the clients were looking for 2xx-, 
4xx-, 55x- and ignoring 150-.   But that was ok, because it didn't make 
sense to use 2xx-, 4xx-, 5xx- since the whole point is that you don't 
know what is the final result until it reaches that point.

Those experiences were expressed in OPES and here.

What does that mean?

Well, we have two choices:

1) Ignore it, and if someone needs this, you can suggest extensions.
    That doesn't solve the compatibility problem.

2) We can embrace it and use the idea to solve a potential problem
    and to help new developments currently going on.

There is really nothing in 821 or 2821 that prevents the idea. There is 
the natural presumption that all codes must be the same, but that isn't 
confirm 100% and also 821 give you the 25 year old suggestion to 
consider just the last line and that is very important.

The 1yz description has a "view" that it is only usable in extensions 
only, but that isn't clear either.

Hence, IMV, I believe using 150- for SMTP to resolve a the potential 
TIMEOUT issue with backward compatibility is a valuable and viable solution.

I can understand the technical documentation consistency issues and I 
can understand the affinity for documentation people to get it all right.

If that is where it must go, fine.  If it reduces confusion, fine.

But consider whose is being confused?  Not me.  Not people who see the 
potential problem, not people who are looking for a solution.  We 
understand the issue. We know what we are facing.  It will hurt the 
efforts to improve the system, and in this particular case, I believe 
very strongly the *proper and responsible* solution is to embrace it and 
legitimize it.  Not close the "Loophole, the Hack, or Trick" as it has 
been improperly labeled.

Thanks

--
HLS

Peter J. Holzer wrote:
> On 2007-04-11 15:07:22 -0400, Hector Santos wrote:
>> John C Klensin wrote:
>>> But, as several people have pointed out, there is ample reason
>>> to infer an assumption that all codes in a multiline reply will
>>> be the same from the text.
>> You know thats fine, and its a natural assumption.  But its doesn't 
>> represent reality and that reality has evolved from the fact it has been 
>> possible all along to have inconsequential and different reply codes in 
>> continuation lines.
>>
>> Remember the ONLY thing that a developer has to go by is a 25 year old 
>> suggestion and it is the ONLY thing to go by for implementing a 
>> extremely no brainer concept that the only ultimate reply code is the 
>> last one, hence why it is in place in wide practice.  It is not by 
>> coincidence.  Why would any system not USE the only logical suggestion 
>> for a reply code design is beyond me.
> 
> One reason why a system might use the first instead of the last code is
> that it just collects the whole reply before looking for a return code.
> In this case it is a lot easier to just use the first three bytes of the
> reply than scan backwards for the penultimate CRLF sequence and use the
> next 3 bytes after that. In just about any programming language except C
> and Fortran that approach sounds at least as "logical" to me as using
> the last code.
> 
>> I honestly don't grasp the resistance.
> 
> My understanding of the RFC has always been that an SMTP command can have only
> one result code and that in multiline replies this (singular!) code must
> be repeated on every line. Others seem to have interpreted this the same.
> You are the first person I know who interpretes this differently. 
> 
> Therefore I think your suggestion is a change in the RFC as it is
> commonly understood, and not just a clarification. Such a change would
> move the RFC back to "proposed standard".
> 
>> If this has all to to do with your RFC time-line, thats one thing, but
>> lets not try to continue to look for reasons to not legitimize what is
>> really a wide practice.
> 
> Is it wide practice? I never encountered it, but I do admit that I don't
> habitually scan SMTP connections.
> 
> 	hp
> 

Date: Thu, 12 Apr 2007 12:13:26 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--cWoXeonUoKmBZSoM
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-04-11 15:07:22 -0400, Hector Santos wrote:
> John C Klensin wrote:
> >But, as several people have pointed out, there is ample reason
> >to infer an assumption that all codes in a multiline reply will
> >be the same from the text.
>=20
> You know thats fine, and its a natural assumption.  But its doesn't=20
> represent reality and that reality has evolved from the fact it has been=
=20
> possible all along to have inconsequential and different reply codes in=
=20
> continuation lines.
>=20
> Remember the ONLY thing that a developer has to go by is a 25 year old=20
> suggestion and it is the ONLY thing to go by for implementing a=20
> extremely no brainer concept that the only ultimate reply code is the=20
> last one, hence why it is in place in wide practice.  It is not by=20
> coincidence.  Why would any system not USE the only logical suggestion=20
> for a reply code design is beyond me.

One reason why a system might use the first instead of the last code is
that it just collects the whole reply before looking for a return code.
In this case it is a lot easier to just use the first three bytes of the
reply than scan backwards for the penultimate CRLF sequence and use the
next 3 bytes after that. In just about any programming language except C
and Fortran that approach sounds at least as "logical" to me as using
the last code.

> I honestly don't grasp the resistance.

My understanding of the RFC has always been that an SMTP command can have o=
nly
one result code and that in multiline replies this (singular!) code must
be repeated on every line. Others seem to have interpreted this the same.
You are the first person I know who interpretes this differently.=20

Therefore I think your suggestion is a change in the RFC as it is
commonly understood, and not just a clarification. Such a change would
move the RFC back to "proposed standard".

> If this has all to to do with your RFC time-line, thats one thing, but
> lets not try to continue to look for reasons to not legitimize what is
> really a wide practice.

Is it wide practice? I never encountered it, but I do admit that I don't
habitually scan SMTP connections.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--cWoXeonUoKmBZSoM
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--cWoXeonUoKmBZSoM--

Date: Thu, 12 Apr 2007 10:14:12 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Cc: "David F. Skoll" <[email-masked]>, Hector Santos <[email-masked]>
Sender: [email-masked]

Hector Santos writes:
> Arnt Gulbrandsen wrote:
>
>> Hector Santos writes:
>>> The fact is, it hasn't shown to be a problem.  Now is the time to 
>>> make it OFFICIAL.
>>
>> So write an internet-draft, get it published as proposed standard, 
>> and go on from there.
>
> Hi Arnt,
>
> Yes, I might just do that.

Good. I look forward to reading it.

> But I ask, which at this point I fully realized it is merely rethorical:
>
> Why waste the time when 1 or 2 "clarification" sentences of probably 
> twenty words or less will accomplish the same and have a much more 
> greater benefit to all and IMV, it would reside exactly where it 
> belongs - the base protocol?
>
> We are not going to see another 2821bis for a long time. The window of 
> opportunity is now with 2821bis.

No, it isn't. 2821 was the window, and it closed long ago.

2821 is/was a proposed standard. 2821bis advances that to draft - it 
cannot add anything new.

Arnt

Date: Thu, 12 Apr 2007 10:13:12 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Cc: Tony Hansen <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> That is a long-winded way of saying that I've gradually become 
> convinced that (iii) is the only plausible and responsible option.

+1

Arnt

Date: Wed, 11 Apr 2007 22:52:00 -0400
From: John C Klensin <[email-masked]>
To: Jeff Macdonald <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

--On Wednesday, 11 April, 2007 21:30 -0400 Jeff Macdonald
<[email-masked]> wrote:

> On Wed, Apr 11, 2007 at 06:21:46PM -0400, John C Klensin wrote:
> 
> shouldn't this
> 
>> >> 		The SMTP client should send another command specifying
>> >> 		whether to continue or abort the action.
> 
> be this
> 
>  		The SMTP client MUST send another command specifying
>  		whether to continue or abort the action.
> 
> due to this?
> 
>> (2) SMTP is designed as strictly a synchronized one-command /
>> one-response protocol.  Pipelining and some other extension
>> ideas that are floating around may permit delaying the
>> responses or otherwise changing the synchronization, but the
>> model remains.  So, once the server issues a reply (or the
>> last line of a multiline reply), it needs to get a new
>> command from the client before it can send out another
>> response code/line.  

Probably yes.  I seem to still be suffering a little reluctance
to change Jon Postel's text unless it is clearly necessary.

    john

Date: Wed, 11 Apr 2007 22:50:31 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Wednesday, 11 April, 2007 16:03 -0400 Hector Santos
<[email-masked]> wrote:

>> or with its symmetric relative
>>    550- Don't think I can reach that one
>>    250  Ah, I found a path
> 
> No, that is illogical and that was discovered with FTP as
> well. You want to use a inconsequential code that is would not
> be interpretation as either negative or positive.
> 
> You need to use a code that is not typically used to interpret
> an initial negative or positive condition. It would be
> illogical do to so.
>...
 
> Doesn't apply because it wouldn't be how it be used.  It needs
> to be an inconsequential code not a "heads up" code that may
> change status as you point out.

But, Hector, it is exactly this sort of comment that has
convinced me that we need to change the text to prohibit
mixed-code multiline replies.  What I think you are saying is
that mixed-code multiline replies should be permitted iff the
intermediate codes are "inconsequential".   But there is no
concept of an "inconsequential code" in 2821 so, if we were to
write such a rule, we would need to then put in text defining a
new class of codes, presumably with semantics of "ignore" or
"ignore and wait", at least in multiline replies.  But then we
run into the problem that there are no codes which can be use
only in the non-terminal lines of multilines and not as replies,
so we would need to deal with that case, presumably identifying
them and prohibiting their use.  By the time we got that far, we
would be in big trouble because:

(i) Whatever 821 (and 2821) are interpreted as saying about 1yz
codes, it isn't "ignore", "ignore and wait", "inconsequential",
or 
"can't use in final code of multiline reply or in a reply by
itself".   Instead, it assigns at least the outline of semantics
to those codes -- the client has to issue either a "continue" or
"abort" command.   So that would be a clear and incompatible
change to the current standard definition, even if one
interprets the text as permitting 1yz in the first place.

(2) We would need to add several new concepts ("inconsequential
codes", etc.) to the document, concepts that we couldn't
possibly justify adding in at this stage even if they
represented established practice... which I suggest that they do
not in any general way, regardless of what you may be able to
show about one particular case.

   john

Date: Wed, 11 Apr 2007 21:30:11 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

On Wed, Apr 11, 2007 at 06:21:46PM -0400, John C Klensin wrote:

shouldn't this

> >> 		The SMTP client should send another command specifying
> >> 		whether to continue or abort the action.

be this

 		The SMTP client MUST send another command specifying
 		whether to continue or abort the action.

due to this?

> (2) SMTP is designed as strictly a synchronized one-command /
> one-response protocol.  Pipelining and some other extension
> ideas that are floating around may permit delaying the responses
> or otherwise changing the synchronization, but the model
> remains.  So, once the server issues a reply (or the last line
> of a multiline reply), it needs to get a new command from the
> client before it can send out another response code/line.  

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Wed, 11 Apr 2007 17:19:34 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions
Sender: [email-masked]

At 12:09 11-04-2007, Frank Ellermann wrote:
>Strictly speaking it's not allowed to use <[email-masked]>
>in Internet messages or SMTP envelopes in the public Internet
>as defined by STD 10, because we have no way to send a reply
>or error report to <[email-masked]> if we're not in the
>same local net.

Yes.

>There MUST be a gateway fixing this before it reaches us.  If
>not it's a standards violation.  Very likely it's also spam

Yes.

>IPv6 only addresses are not better than UTF8SMTP addresses,

UTF8SMTP is a SMTP extension.  The main issue with IPv6 is 
reachability which a SMTP extension won't solve.

>For starters the MAIL FROM has to work.  That's essential for
>SMTP.  And it's enforced by receivers employing some kind of
>"call back verification".  Even if it's as little as checking
>that the domain in question has an associated IPv4 address.
>
>And a plausible (receiver's POV) MAIL FROM isn't enough, the
>end user getting the mail would also like to click a "reply"
>button, expecting that the Reply-To (or From) address work.

Interestingly, one might read "alternate address functions" and 
"intermediate relays" as a means to get the above working.

>There can be numerous cases where it won't work in practice,
>that's just the "S" in SMTP, but if it systematically doesn't
>work it's a violation of the spec.

Prohibiting "IPv6 only" would get this draft shot down.  I don't 
think we want that to happen.

Regards,
-sm 

Date: Wed, 11 Apr 2007 16:15:29 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions
Sender: [email-masked]

At 12:19 11-04-2007, Frank Ellermann wrote:
>Addition:  John's proposed changes match your suggestions.
>That won't solve the "IPv6 only" problem, but it fixes all
>missing AAAA cases.

The aim was to fix the "AAAA" case which you pointed out. Solving the 
"IPv6 only" problem is a whole new ballgame. :)

Regards,
-sm

Date: Wed, 11 Apr 2007 19:22:44 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:
 >
 > That is a long-winded way of saying that I've gradually become
 > convinced that (iii) is the only plausible and responsible
 > option.

And its the worst decision you can make. You will hurting progress, not 
helping it and I strong say that as one of the long time commercial SMTP 
  developer.  You gain absolutely nothing here with (iii) and it will 
goes against 25 years of time tested last line reply code logic.

Terrible decision, I'm completely flabbergasted by all this.

--
HLS

Date: Wed, 11 Apr 2007 16:02:59 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

At 12:20 11-04-2007, Tony Hansen wrote:
>         (iii) Prohibit different codes and, optionally, suggest
>         that it is ok for a client to select one of them and
>         assume that all of the others are the same.

+1

That avoids any misinterpretation.

Regards,
-sm 

Date: Wed, 11 Apr 2007 18:29:20 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

--On Wednesday, 11 April, 2007 15:20 -0400 Tony Hansen
<[email-masked]> wrote:

> Please state your preference for one of the following. (I've
> broken John's (ii) into two parts.)
> 
> 	(i) Do nothing, leaving the text as is
> 	(ii-a) Make it clear that the codes may be different and
> 	that clients are expected to respect the *last* value listed
> 	(ii-b) Make it clear that the codes may be different and
> 	that clients are expected to respect the *first* value listed
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

Speaking personally only, I've become convinced that, if the
current text can engender this much discussion and confusion, it
needs to be fixed in some way.  I've also become convinced that
both versions of (ii) would cause far more problems than they
are worth.  That is partially because I don't see any good use
for mixed codes --especially if all but a specific one are to be
ignored-- that does not require semantics that are not defined
in 2821 and hence would either require an extension or create
even more confusion.  I also share Ned's concern that there are
probably clients out there that pick any code in sequence to
look at that we can imagine but, for me, that is actually a
secondary concern.

That is a long-winded way of saying that I've gradually become
convinced that (iii) is the only plausible and responsible
option.

    john

Date: Wed, 11 Apr 2007 18:21:46 -0400
From: John C Klensin <[email-masked]>
To: Jeff Macdonald <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

--On Wednesday, 11 April, 2007 16:31 -0400 Jeff Macdonald
<[email-masked]> wrote:

> On Wed, Apr 11, 2007 at 09:25:04AM -0400, John C Klensin wrote:
>> Proposed text:
>> 
>> 		The command has been accepted, but the requested action
>> 		is being held in abeyance, pending confirmation of the
>> 		information in this reply.  The SMTP client should send
>> 		another command specifying whether to continue or abort
>> 		the action.  
> 
> Does this allow the following?
> 
> s: 150
> <time goes by>
> s: 250

Not without either an extension or other changes, for two
reasons:

(1) 1yz replies are not now permitted _at all_ except by
extension. As I understand a slightly extended form of Hector's
argument, if one can have different codes on different
continuation lines and only the last one counts, then one can
have _anything_ (or at least any three-digit code) on the
previous lines with no semantic impact as far as the standard is
concerned other than the hope that the client will wait until
the sequence of continuation lines is complete.  The more I
think about that argument, the less I buy it, largely because of
longstanding personal opposition to standardizing tricks and
hacks.  But it is not applicable to this case in any event.

(2) SMTP is designed as strictly a synchronized one-command /
one-response protocol.  Pipelining and some other extension
ideas that are floating around may permit delaying the responses
or otherwise changing the synchronization, but the model
remains.  So, once the server issues a reply (or the last line
of a multiline reply), it needs to get a new command from the
client before it can send out another response code/line.  

     john

Date: Wed, 11 Apr 2007 18:01:04 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], "David F. Skoll" <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> Hector Santos writes:
>> The fact is, it hasn't shown to be a problem.  Now is the time to make 
>> it OFFICIAL.
> 
> So write an internet-draft, get it published as proposed standard, and 
> go on from there.

Hi Arnt,

Yes, I might just do that. But I ask, which at this point I fully 
realized it is merely rethorical:

Why waste the time when 1 or 2 "clarification" sentences of probably 
twenty words or less will accomplish the same and have a much more 
greater benefit to all and IMV, it would reside exactly where it belongs 
- the base protocol?

We are not going to see another 2821bis for a long time. The window of 
opportunity is now with 2821bis.

--
HLS

Date: Wed, 11 Apr 2007 14:50:32 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Wed, Apr 11, 2007, Tony Hansen wrote:

> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

This gets my vote.

Date: Wed, 11 Apr 2007 17:44:58 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

>> (i)   Two different reasonable people could interpret the reply
>> differently.

> More like 1 in a million. :-)

No; see John's example:

    250-OK
    550 Not OK.

>> (ii)  It seems like a confusing hack.

> eh, hence why we want to legitimate it.

We do NOT want to legitimize hacks.  As I said, your use-case relies on
accidents of implementation.

>> (iii) The only use-case proposed so far relies on client implementation
>>       details upon which the server SHOULD NOT rely.

> Then why bother with with 45x and 55x in tying to control clients?  In
> reality, you can't rely on that either.

You can rely on the fact that an RFC-Compliant client will react as
expected to 4xx and 5xx codes.  You can't predict at all what an
RFC-Compliant client will do with a multiline response with different
codes.

(Obviously, if the client is evil ratware, then you cannot predict how
it will react to 4xx and 5xx codes, but that's outside the scope of the RFC.)

> The fact is, it hasn't shown to be a problem.  Now is the time to make
> it OFFICIAL.  Who want to wait until 5-10 years?

I don't think evil hacks should be baked into RFCs.  That's the process
that leads to abominations like the Win32 API.

Regards,

David.

From: [email-masked]
Cc: [email-masked], Frank Ellermann <[email-masked]>, [email-masked]
Date: Wed, 11 Apr 2007 14:37:58 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Wednesday, 11 April, 2007 08:25 -0700
> [email-masked] wrote:

> >...
> >> While I agree with you in principle, I think we need to be a
> >> bit careful here.  If the goal is to prevent the same code
> >> from being used for different purposes, I think it is an
> >> unpleasant necessity to recognize and register codes acquired
> >> by squatting based on non-conforming code that has become
> >> deployed widely enough to be significant.
> >
> > If so, that is NOT an argument for the "RFC Required" model -
> > I see essentially no chance that a code-squatter is going to
> > write an RFC of any sort, let alone write one but then send it
> > to the RFC Editor instead of putting it through the IETF
> > process.

> Agreed.  See below.

> >> Have a look at the registration-specification language in
> >> draft-klensin-smtp-code-registry-00 (a parallel development to
> >> Tony's draft that I hope will be quickly consolidated with it)
> >> and let us know what you think.
> >
> > You have gone with the "specification required" approach. This
> > is in one sense more lenient than the "RFC Required" since it
> > allows for non-RFC specificaitons and would avoid the problem
> > that code-squatters rarely if ever seem to be willing to write
> > RFCs to document their usage.

> That was, more or less, the intent.
 
> > However, in another sense it is less lenient since it requires
> > approval by a designated expert. I think this is sufficient
> > (barely) to address my concerns about overlapping or poorly
> > defined codes getting into the registry.

> One could, in principle, combine the two, allowing registration
> by RFC if a document could make it through either the IESG or
> the RFC Editor, but a specification for other cases.  I don't
> know if that would be helpful or not.

> > However, one of the requirements for expert review is that the
> > review criteria be clearly spelled out in the registration
> > document. I don't think what you have is sufficient in this
> > regard. I think it needs to have a list of requirements, such
> > as:
> >
> > (1) The meaning of the code must be clearly defined.
> > (2) The code most not duplicate the functionality of a
> > previously defined     code.
> > (3) Whenever possible the use-cases new codes should not
> > overlap the use-cases     for existing codes, except when a
> > new code is intended to allow     identification of a more
> > specific subcase of an existing code.

> This is definitely an improvement.   I'm going to leave the
> merging and word-smithing to Tony unless he asked for help, but
> would consider adding this worthwhile.
 
> > Finally, the document should say that exceptions to the rules
> > can be made when an unregistered code has achieved significant
> > deployment since the main purpose of the registry is to
> > document all code use in order to avoid confusion and prevent
> > multiple different definitions for the same code.

> My bottom line is that, as long as some provision such as the
> above is included  -- I tried to treat it not as an exception
> but as a nearly-sufficient justification for approval  -- I can
> live with almost any plausible "normal case" language.   I am
> concerned about getting into situations in which a code that is
> in wide use cannot make it into the registry because a
> capricious RFC Editor or IESG decide they don't like the idea
> behind the code or the tone of the description and would prefer
> to make it clear that would be an undesirable outcome.

I agree that this is a concern, so making it less of an exception and more
of an overriding rule makes sense. This is really a matter of getting
the emphasis just right, of course.

				Ned

From: [email-masked]
Cc: [email-masked]
Date: Wed, 11 Apr 2007 14:29:45 -0700 (PDT)
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
To: David F Skoll <[email-masked]>
Sender: [email-masked]

> Tony Hansen wrote:

> > 	(iii) Prohibit different codes and, optionally, suggest
> > 	that it is ok for a client to select one of them and
> > 	assume that all of the others are the same.

> I vote for (iii)

I could live with (i), but I also vote for (iii) since I strongly suspect there
are clients out there that look at first, middle, last, and may even check and
see if all the coddes agree. (ii) is unacceptable IMO.

				Ned

Date: Wed, 11 Apr 2007 14:29:47 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Wed, Apr 11, 2007, David F. Skoll wrote:

> Hector Santos wrote:

> > What it probably interested here is that its uses a non-standard less
> > than 100 code for "informational", so something like this is valid under
> > SendMail.

It's "sendmail".

> >     099-Informational

> >     250 whatever

> Or even:
>    099 Info
>    099 Info
>    250 Whatever

> > Very interesting.

> Unused in practice, I believe.

I hope nobody plans to say "sendmail does this, so it must be right".
Please don't use an implementation that was done before the RFCs
to justify wierd/bogus behavior. It causes big problems for authors
of other MTAs (as they might be "forced" to implement that wierd
behavior too).  sendmail allows all kind of bogus things (mostly
to deal with broken software) which are not valid according to the
RFCs.  The RFCs should (clearly) define what a protocol should look
like, not some implementation.

Date: Wed, 11 Apr 2007 16:41:19 -0400
From: Jeff Macdonald <[email-masked]>
To: Tony Hansen <[email-masked]>
Cc: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, Apr 11, 2007 at 03:20:01PM -0400, Tony Hansen wrote:
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

+1

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Wed, 11 Apr 2007 22:32:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Cc: "David F. Skoll" <[email-masked]>, Hector Santos <[email-masked]>
Sender: [email-masked]

Hector Santos writes:
> The fact is, it hasn't shown to be a problem.  Now is the time to make 
> it OFFICIAL.

So write an internet-draft, get it published as proposed standard, and 
go on from there.

Arnt

Date: Wed, 11 Apr 2007 16:31:44 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

On Wed, Apr 11, 2007 at 09:25:04AM -0400, John C Klensin wrote:
> Proposed text:
> 
> 		The command has been accepted, but the requested action
> 		is being held in abeyance, pending confirmation of the
> 		information in this reply.  The SMTP client should send
> 		another command specifying whether to continue or abort
> 		the action.  

Does this allow the following?

s: 150
<time goes by>
s: 250

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Wed, 11 Apr 2007 16:11:21 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> I honestly don't grasp the resistance.
> 
> I am strongly resistant to different reply codes in a multiline reply
> for the following reasons:
> 
> (i)   Two different reasonable people could interpret the reply differently.

More like 1 in a million. :-)

> (ii)  It seems like a confusing hack.

eh, hence why we want to legitimate it.

> (iii) The only use-case proposed so far relies on client implementation
>       details upon which the server SHOULD NOT rely.

Then why bother with with 45x and 55x in tying to control clients?  In 
reality, you can't rely on that either.

If the client is going to hang up anyway because it is ignoring an 150- 
to reset a timer, the odds are good you probably don't that mail anyway.

The fact is, it hasn't shown to be a problem.  Now is the time to make 
it OFFICIAL.  Who want to wait until 5-10 years?

That said, I respect your opinion you are not wrong, but you not right 
either with the logistics here. :-)

--
HLS

Date: Wed, 11 Apr 2007 16:03:31 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

John C Klensin wrote:
> 
> --On Wednesday, 11 April, 2007 15:07 -0400 Hector Santos
> <[email-masked]> wrote:
> 
>> I honestly don't grasp the resistance. If this has all to to
>> do with your RFC time-line, thats one thing, but lets not try
>> to continue to look for reasons to not legitimize what is
>> really a wide practice. What is being suggested isn't breaking
>> ANYTHING and it has MORE value than for the future that what
>> is being suggested otherwise.
> 
> Let me try to explain at least one source of what you are
> characterizing as "the resistance".   Ignore your desired
> special use of 1yz codes for the moment.  We tell clients that
> they are supposed to be able to take action on the codes alone
> and the first digit of the codes in particular.
> 
> Suppose a particularly strange server responds to a RCPT command
> with 
> 
>    250- I think that address is ok...
>    550  Whoops, never mind.
> 
> or with its symmetric relative
>    550- Don't think I can reach that one
>    250  Ah, I found a path

No, that is illogical and that was discovered with FTP as well. You want 
to use a inconsequential code that is would not be interpretation as 
either negative or positive.

You need to use a code that is not typically used to interpret an 
initial negative or positive condition. It would be illogical do to so.

> Now, in spite of the "you can probably just look at the last
> one" recommendation in 821, there is no requirement that the
> client wait  for the last one before starting to set processing
> up... or even getting fairly far along with it.  

Agree. So if you don't wish to make it a normative text, then lets leave 
it alone between the test of time has solidify it anyway.

> What happens  here depends critically on whether the client picks the first
> one or the last one.   Worse, if one had a three-line sequences
 > of
 >    550-
 >    250-
 >    550
 > or
 >    250-
 >    550-
 >    250
 > It is even less clear what the poor client is to do.

Doesn't apply because it wouldn't be how it be used.  It needs to be an 
inconsequential code not a "heads up" code that may change status as you 
point out.

--
HLS

Date: Wed, 11 Apr 2007 20:59:59 +0100
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Tony Hansen wrote:

> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

http://www.imc.org/ietf-smtp/mail-archive/msg03247.html

My reasons are that (iia) and (iib) are unacceptable because of running
code, and (i) has lead to interoperability problems which I understand
this revision effort aims to minimise.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL: SOUTHWEST BACKING SOUTH 5 OR 6, INCREASING 7 OR GALE 8, PERHAPS
SEVERE GALE 9 LATER. MODERATE OR ROUGH BECOMING VERY ROUGH. MAINLY FAIR.
MODERATE OR GOOD.

Date: Wed, 11 Apr 2007 15:45:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Hansen wrote:
> John C Klensin wrote:
>> I've assigned Issue 17 to the question of whether, if reply
>> lines constitute a continuation set, all of the codes are
>> expected to be the same and whether this needs clarification.
>>
>> It appears that there are three options:
>>
>> 	(i) Do nothing, leaving the text as is
>> 	(ii) Make it clear that the codes may be different and
>> 	that clients are expected to examine all of them.
>> 	(iii) Prohibit different codes and, optionally, suggest
>> 	that it is ok for a client to select one of them and
>> 	assume that all of the others are the same.
> 
> <pseudo-chair hat on>
> I've been trying to see if there is any sort of consensus building.
> While there has been lots of discussion on this issue, it is unclear
> that I've seen clearcut stakes placed on particular choices. As I review
> the email, I see essentially *even preferences* for each of the 3
> options that John posted.
> 
> For (ii), we've heard of one implementation (Exim) that looks only at
> the 1st code, a claim of another (sendmail) that was later refuted by
> checking the code, and general reports of others that all look at the
> last code.
> 
> I think we've gotten to the point where the discussion is starting to
> repeat and we need to try this again.
> 
> Please state your preference for one of the following. (I've broken
> John's (ii) into two parts.)
> 
> 	(i) Do nothing, leaving the text as is
> 	(ii-a) Make it clear that the codes may be different and
> 	that clients are expected to respect the *last* value listed
> 	(ii-b) Make it clear that the codes may be different and
> 	that clients are expected to respect the *first* value listed
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.
> 
> If there is still no clear consensus, the default will be (i).
> </pseudo-chair hat off>

Against my inner feeling, I change my vote to (i).  DO NOTHING.

--
HLS

Date: Wed, 11 Apr 2007 15:46:10 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Wednesday, 11 April, 2007 15:07 -0400 Hector Santos
<[email-masked]> wrote:

> 
> I honestly don't grasp the resistance. If this has all to to
> do with your RFC time-line, thats one thing, but lets not try
> to continue to look for reasons to not legitimize what is
> really a wide practice. What is being suggested isn't breaking
> ANYTHING and it has MORE value than for the future that what
> is being suggested otherwise.

Let me try to explain at least one source of what you are
characterizing as "the resistance".   Ignore your desired
special use of 1yz codes for the moment.  We tell clients that
they are supposed to be able to take action on the codes alone
and the first digit of the codes in particular.

Suppose a particularly strange server responds to a RCPT command
with 

   250- I think that address is ok...
   550  Whoops, never mind.

or with its symmetric relative
   550- Don't think I can reach that one
   250  Ah, I found a path

Now, in spite of the "you can probably just look at the last
one" recommendation in 821, there is no requirement that the
client wait  for the last one before starting to set processing
up... or even getting fairly far along with it.  What happens
here depends critically on whether the client picks the first
one or the last one.   Worse, if one had a three-line sequences
of 
   550-
   250-
   550
or
   250-
   550-
   250
It is even less clear what the poor client is to do.

While I cannot imagine the conditions under which a real server
would do this sort of thing, if we explicitly permit mixed-code
responses, we have to either deal with _all_ of the cases or
explicitly exclude some of them.

For example, one could fix this by saying "the codes must all be
the same unless they are non-final values starting in 1 or 0",
but that would constitute adding significant new functionality
since we have never specified an interpretation or actions for
either 1yz or 0yz codes in the past.

And _that_ is where at least some of "the resistance" is coming
from.

    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Date:  Wed, 11 Apr 2007 21:44:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Tony Hansen wrote:

> (ii-a) Make it clear that the codes may be different and that
         clients are expected to respect the *last* value listed

I'd feel better with "SHOULD NOT be different".  But if they're
nevertheless different taking the last code for real as suggested
in RFC 2821 is the most obvious solution for this odd situation.

Frank

P.S. to Hector:  An SMTP extension specified in an RFC, offered
by the server, and explicitly accepted by the client, is a good
excuse to violate this general SHOULD NOT addressing backwards
compatibility.  IOW you can use different codes if both sides
agree - ideally you'd specify it in an RFC.  You can't claim that
2821bis allows this _without_ mutual agreement, it would violate
the proposed SHOULD NOT.

Date: Wed, 11 Apr 2007 15:40:26 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Hansen wrote:

> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

I vote for (iii)

Regards,

David.

Date: Wed, 11 Apr 2007 15:37:20 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions
Sender: [email-masked]

IMO, if a client want to issue IPv6 ready SMTP information, then maybe a 
EHLO extension should be made available that exposes the idea SERVER is 
ready and capable of handling such new IPV6 information.  Maybe a IPV6 
keyword as part of the EHLO response would be enough.

--
HLS

Frank Ellermann wrote:
> SM wrote:
> 
>>>> How about a more radical approach, just forbid "IPv6 only"
>>>> for the moment somewhere in section 5 ?  "IPv6 only" is IMO
>>>> completely against the spirit of "if it can't receive mail
>>>> it has no business to send mail".
> 
>> Frank, that's too radical.
> 
> I don't think so.  For "foreign" stuff there are already tons
> of MUSTard that addresses in the envelope and in the header
> are supposed to work in the environment where they are used.
> 
> Strictly speaking it's not allowed to use <[email-masked]>
> in Internet messages or SMTP envelopes in the public Internet
> as defined by STD 10, because we have no way to send a reply
> or error report to <[email-masked]> if we're not in the
> same local net.
> 
> There MUST be a gateway fixing this before it reaches us.  If
> not it's a standards violation.  Very likely it's also spam
> and/or a criminal offense like most mails today.
> 
> The EAI folks are just designing a complete new overlay net
> for port 25 traffic, so far they have an SMTP extension and
> a new superset of message/rfc822, and maybe they opt for a
> new MIME version implemented by redefining MIME Version 1.0.
> 
> IPv6 only addresses are not better than UTF8SMTP addresses,
> <[email-masked]>, or Fido addresses, they MUST go through
> a gateway replacing them by something users in IPv4 only nets
> can use.
> 
> For starters the MAIL FROM has to work.  That's essential for
> SMTP.  And it's enforced by receivers employing some kind of
> "call back verification".  Even if it's as little as checking
> that the domain in question has an associated IPv4 address.
> 
> And a plausible (receiver's POV) MAIL FROM isn't enough, the
> end user getting the mail would also like to click a "reply"
> button, expecting that the Reply-To (or From) address work.
> 
> There can be numerous cases where it won't work in practice,
> that's just the "S" in SMTP, but if it systematically doesn't
> work it's a violation of the spec.
> 
> Frank

Date: Wed, 11 Apr 2007 15:37:26 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> I honestly don't grasp the resistance.

I am strongly resistant to different reply codes in a multiline reply
for the following reasons:

(i)   Two different reasonable people could interpret the reply differently.
(ii)  It seems like a confusing hack.
(iii) The only use-case proposed so far relies on client implementation
      details upon which the server SHOULD NOT rely.

Regards,

David.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions
Date:  Wed, 11 Apr 2007 21:19:34 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

SM wrote:

> I didn't see the proposed change.  I suggest the following:

Sorry, I clicked "send" as soon as I had a version without 
too many "let's just return to the good old reverse paths"
rants... ;-)

Addition:  John's proposed changes match your suggestions.
That won't solve the "IPv6 only" problem, but it fixes all
missing AAAA cases.

Frank

Date: Wed, 11 Apr 2007 12:16:20 -0700
From: Chris Newman <[email-masked]>
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Tony Hansen <[email-masked]>, [email-masked]
Sender: [email-masked]

Tony Hansen wrote on 4/10/07 1:52 -0400:
> I also introduced X.7.14 and X.7.15 to replace the competing uses of
> X.7.8. Is this the best way to handle this conflict?

I recommend keeping x.7.8 for SMTP AUTH, and only changing the code for BURL.

There's actually no interop problem with the conflicting use as they apply to 
different SMTP commands.  So that means it's only necessary to change one of 
the codes as a matter of cleanliness.  Changing the BURL code will have the 
least impact to deployed software, IMHO.

                - Chris

Date: Wed, 11 Apr 2007 15:20:01 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: recap rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:
> I've assigned Issue 17 to the question of whether, if reply
> lines constitute a continuation set, all of the codes are
> expected to be the same and whether this needs clarification.
>
> It appears that there are three options:
>
> 	(i) Do nothing, leaving the text as is
> 	(ii) Make it clear that the codes may be different and
> 	that clients are expected to examine all of them.
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

<pseudo-chair hat on>
I've been trying to see if there is any sort of consensus building.
While there has been lots of discussion on this issue, it is unclear
that I've seen clearcut stakes placed on particular choices. As I review
the email, I see essentially *even preferences* for each of the 3
options that John posted.

For (ii), we've heard of one implementation (Exim) that looks only at
the 1st code, a claim of another (sendmail) that was later refuted by
checking the code, and general reports of others that all look at the
last code.

I think we've gotten to the point where the discussion is starting to
repeat and we need to try this again.

Please state your preference for one of the following. (I've broken
John's (ii) into two parts.)

	(i) Do nothing, leaving the text as is
	(ii-a) Make it clear that the codes may be different and
	that clients are expected to respect the *last* value listed
	(ii-b) Make it clear that the codes may be different and
	that clients are expected to respect the *first* value listed
	(iii) Prohibit different codes and, optionally, suggest
	that it is ok for a client to select one of them and
	assume that all of the others are the same.

If there is still no clear consensus, the default will be (i).
</pseudo-chair hat off>

	Tony Hansen
	[email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions
Date:  Wed, 11 Apr 2007 21:09:44 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 51
Sender: [email-masked]

SM wrote:

>>> How about a more radical approach, just forbid "IPv6 only"
>>> for the moment somewhere in section 5 ?  "IPv6 only" is IMO
>>> completely against the spirit of "if it can't receive mail
>>> it has no business to send mail".

> Frank, that's too radical.

I don't think so.  For "foreign" stuff there are already tons
of MUSTard that addresses in the envelope and in the header
are supposed to work in the environment where they are used.

Strictly speaking it's not allowed to use <[email-masked]>
in Internet messages or SMTP envelopes in the public Internet
as defined by STD 10, because we have no way to send a reply
or error report to <[email-masked]> if we're not in the
same local net.

There MUST be a gateway fixing this before it reaches us.  If
not it's a standards violation.  Very likely it's also spam
and/or a criminal offense like most mails today.

The EAI folks are just designing a complete new overlay net
for port 25 traffic, so far they have an SMTP extension and
a new superset of message/rfc822, and maybe they opt for a
new MIME version implemented by redefining MIME Version 1.0.

IPv6 only addresses are not better than UTF8SMTP addresses,
<[email-masked]>, or Fido addresses, they MUST go through
a gateway replacing them by something users in IPv4 only nets
can use.

For starters the MAIL FROM has to work.  That's essential for
SMTP.  And it's enforced by receivers employing some kind of
"call back verification".  Even if it's as little as checking
that the domain in question has an associated IPv4 address.

And a plausible (receiver's POV) MAIL FROM isn't enough, the
end user getting the mail would also like to click a "reply"
button, expecting that the Reply-To (or From) address work.

There can be numerous cases where it won't work in practice,
that's just the "S" in SMTP, but if it systematically doesn't
work it's a violation of the spec.

Frank
-- 
In any case, the SMTP adds its own identifier to the reverse-path.
[RFC 821]

Date: Wed, 11 Apr 2007 15:07:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

John C Klensin wrote:
 >
> But, as several people have pointed out, there is ample reason
> to infer an assumption that all codes in a multiline reply will
> be the same from the text.

You know thats fine, and its a natural assumption.  But its doesn't 
represent reality and that reality has evolved from the fact it has been 
possible all along to have inconsequential and different reply codes in 
continuation lines.

Remember the ONLY thing that a developer has to go by is a 25 year old 
suggestion and it is the ONLY thing to go by for implementing a 
extremely no brainer concept that the only ultimate reply code is the 
last one, hence why it is in place in wide practice.  It is not by 
coincidence.  Why would any system not USE the only logical suggestion 
for a reply code design is beyond me.

I honestly don't grasp the resistance. If this has all to to do with 
your RFC time-line, thats one thing, but lets not try to continue to 
look for reasons to not legitimize what is really a wide practice. What 
is being suggested isn't breaking ANYTHING and it has MORE value than 
for the future that what is being suggested otherwise.

--
HLS

Date: Wed, 11 Apr 2007 14:39:59 -0400
From: John C Klensin <[email-masked]>
To: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
cc: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and         draft-siemborski-rfc2554bis
Sender: [email-masked]

--On Wednesday, 11 April, 2007 14:31 +0100 Alexey Melnikov
<[email-masked]> wrote:

>> that an IANA registry for
>> enhanced status codes could be approved before 2554bis is
>> published,  in which
>> as I noted below a revision with an RFC Editor note would be 
>> appropriate if
>> not required.
> 
> It might be just as easy to add registration of 2554bis
> enhanced status codes to Tony's/John's draft.

Yes, or to take advantage of the weasel-words in mine (suggested
by Chris) that effectively include the codes from any document
in the approval or publication pipeline in the initial
registration list.

    john

Date: Wed, 11 Apr 2007 14:37:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:
> 
>> If I understand the question, you are saying that an I-D
>> proposal can update and even go as far as change an RFC and my
>> "assumption" that it can not is an incorrect assumption?
> 
> An I-D cannot.  But the current flavor of SMTP, from RFC1425
> forward, is rather carefully designed, and I hope clearly
> documented, so that a standardized extension that is offered by
> the  server and invoked by the client, may change just about any
> aspect of SMTP behavior.  
 > ..
 > ..
> So I think your implied assumption that text in an RFC saying
> "this is forbidden" prevents doing exactly the same thing via an
> extension is incorrect in two respects.  

Ok, this is all confusing to me which basically means, I like to keep 
things simple. :-)

>> At this point, maybe Hansen had the best (personal hat on)
>> advice:
>>
>>   "don't do anything now that would prevent such an
>>    extension from being written."
> 
> And no such thing has been proposed.  By anyone.

Man, I am not trying to fight you guys. I don't know why you wrote that 
or what you mean by it.

Hansen wrote:

<personal hat on>
Yes, I have to be careful about expressing both my personal opinions and
process opinions. Hopefully I've been succeeding at doing that.
Personally, I think it would be very useful and important to get an
extension I-D written up for this case. That's why I said previously "I
could easily see an extension I-D being written that defines and blesses
its usage. So I wouldn't want to deny its possibility within 2821bis."
This can be translated as: don't do anything now that would prevent such
an extension from being written.
</personal hat on>

Borrowing the old Chinese proverbial "Be careful for what you wish for", 
maybe at this point I agree with him personal version of this issue.

Don't do anything, leaving it alone might be the best thing here.

--
HLS

Date: Wed, 11 Apr 2007 14:37:09 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

--On Wednesday, 11 April, 2007 08:25 -0700
[email-masked] wrote:

>...
>> While I agree with you in principle, I think we need to be a
>> bit careful here.  If the goal is to prevent the same code
>> from being used for different purposes, I think it is an
>> unpleasant necessity to recognize and register codes acquired
>> by squatting based on non-conforming code that has become
>> deployed widely enough to be significant.
> 
> If so, that is NOT an argument for the "RFC Required" model -
> I see essentially no chance that a code-squatter is going to
> write an RFC of any sort, let alone write one but then send it
> to the RFC Editor instead of putting it through the IETF
> process.

Agreed.  See below.

>> Have a look at the registration-specification language in
>> draft-klensin-smtp-code-registry-00 (a parallel development to
>> Tony's draft that I hope will be quickly consolidated with it)
>> and let us know what you think.
> 
> You have gone with the "specification required" approach. This
> is in one sense more lenient than the "RFC Required" since it
> allows for non-RFC specificaitons and would avoid the problem
> that code-squatters rarely if ever seem to be willing to write
> RFCs to document their usage.

That was, more or less, the intent.
 
> However, in another sense it is less lenient since it requires
> approval by a designated expert. I think this is sufficient
> (barely) to address my concerns about overlapping or poorly
> defined codes getting into the registry.

One could, in principle, combine the two, allowing registration
by RFC if a document could make it through either the IESG or
the RFC Editor, but a specification for other cases.  I don't
know if that would be helpful or not.

> However, one of the requirements for expert review is that the
> review criteria be clearly spelled out in the registration
> document. I don't think what you have is sufficient in this
> regard. I think it needs to have a list of requirements, such
> as:
> 
> (1) The meaning of the code must be clearly defined.
> (2) The code most not duplicate the functionality of a
> previously defined     code.
> (3) Whenever possible the use-cases new codes should not
> overlap the use-cases     for existing codes, except when a
> new code is intended to allow     identification of a more
> specific subcase of an existing code.

This is definitely an improvement.   I'm going to leave the
merging and word-smithing to Tony unless he asked for help, but
would consider adding this worthwhile.
 
> Finally, the document should say that exceptions to the rules
> can be made when an unregistered code has achieved significant
> deployment since the main purpose of the registry is to
> document all code use in order to avoid confusion and prevent
> multiple different definitions for the same code.

My bottom line is that, as long as some provision such as the
above is included  -- I tried to treat it not as an exception
but as a nearly-sufficient justification for approval  -- I can
live with almost any plausible "normal case" language.   I am
concerned about getting into situations in which a code that is
in wide use cannot make it into the registry because a
capricious RFC Editor or IESG decide they don't like the idea
behind the code or the tone of the description and would prefer
to make it clear that would be an undesirable outcome.

       john

Date: Wed, 11 Apr 2007 14:29:00 -0400
From: John C Klensin <[email-masked]>
To: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Wednesday, 11 April, 2007 14:00 -0400 "David F. Skoll"
<[email-masked]> wrote:

>...
>> That would be an unnecessary restriction. Clients or servers
>> doen't require this tightness.
> 
> I believe it was intended by the original authors.  In fact, I
> think they didn't even imagine the possibility of different
> return codes on different lines.  But I don't know for sure;
> we'd have to ask.

Asking the original author would be difficult for various
practical and theological reasons; getting an answer would be
more so.

But, as several people have pointed out, there is ample reason
to infer an assumption that all codes in a multiline reply will
be the same from the text.

   john

Date: Wed, 11 Apr 2007 14:19:40 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

--On Wednesday, 11 April, 2007 09:25 -0400 Hector Santos
<[email-masked]> wrote:

>...
>> I hope everyone understands that nothing in 2821 prevents an
>> extension from being designed, proposed, and adopted that, if
>> appropriately agreed to between server and client, changes any
>> or all of those rules.   If 2821 is not clear on that topic,
>> then it should be and I would welcome appropriate text for
>> 2821bis.
>> 
>> To the extent to which the concern is that making these rules
>> more specific (or implied prohibitions more clear) in 2821bis
>> would prevent such extensions, I think we need to clarify that
>> the concern is groundless.
>> 
>> Does that bring our positions closer together?
> 
> If I understand the question, you are saying that an I-D
> proposal can update and even go as far as change an RFC and my
> "assumption" that it can not is an incorrect assumption?

An I-D cannot.  But the current flavor of SMTP, from RFC1425
forward, is rather carefully designed, and I hope clearly
documented, so that a standardized extension that is offered by
the  server and invoked by the client, may change just about any
aspect of SMTP behavior.  "Just about" because an extension
clearly cannot change either the behavior of the greeting
message nor the syntax and semantics of EHLO and its reply.
Other than that, it is pretty much open season.   The protection
against stupid things is the review and standardization process,
not words in 2821 or its successors that say "the extension
mechanism can never be used to do so-and-so".

So I think your implied assumption that text in an RFC saying
"this is forbidden" prevents doing exactly the same thing via an
extension is incorrect in two respects.  First, to all practical
intents and purposes, any statement in 2821 that permits or
forbids something must be read to include "unless a specific
extension that is offered by the server and invoked by the
client changes this".  And, second, in that you take those
statements as banning particular styles of extensions.

In particular, while I can think of several approaches I would
prefer to it were it proposed, there is no restriction in 2821,
and no restriction anyone has considered putting into 2821bis,
that would prevent
  Extension reply value: KEEPALIVE150
  New command: ENKP     (enable keepalive)

  Semantics: If KEEPALIVE150 is included in the EHLO reply from
the server, the client may issue the EKNP command before a mail
transaction is initiated (e.g., before a MAIL command or after a
RSET or completion of a DATA command and before the next MAIL
command).  Once issued, the EKNP command applies for the rest of
the connection.  With the EKNP command in effect, a 150 reply
code is to be taken as a "keepalive" by the client with any
associated text ignored.  The 150 may appear as part of a
multiline reply, in which case the client's timers for timeout
purposes are reset but the line is otherwise ignored.  If it
appears as the code on a single-line reply, or as the final code
in a multi-line reply, the client must respond with either (i) a
QUIT or RSET command, with the usual meaning or (ii) a NOOP
command, to which the server may respond with either a reply
containing a 250 "ok" code or one containing another 150 code.

That would, with a few more details, be a valid proposal
regardless about what 2821 said about unextended SMTP.  Again,
I'm not suggesting it would be either optimal or wise -- that is
a separate discussion.

> It is my understanding that an proposal can propose changes to
> current spec.
> 
> But I have a hard time answering that because whether you
> believe it or not, I am extremely pragmatic and conservative.
> I don't want to break operations for anyone nor for us. Thats
> the last thing I want.

And that is exactly why these things should be done, if at all,
by extensions to which the server and client must explicitly
concur rather than by tweaking the text to open up more
loopholes and/or make existing conforming behavior invalid.

> Here is my opinion above the above items:

Not what I asked.  I was only trying to check whether, even if
those restrictions were stated in the very restrictive way in
which I stated them, you understood that an extension could
override them.  It is still not clear to me that we agree on
that.

However, to try to clarify one thing...

> I don't (iii) is necessary because
> it is SUBJECTIVE plus you have no control over this usage.

(iii) was "multiline replies MUST NOT be used as an attempt to
affect client timing or timeouts".  I don't think that is
subjective at all.  The server implementer knows his or her
intent about the codes being sent.   And this authorizes the
client to treat any sort of "150- Please wait" line as lying
well outside the standard and then decide whether to simply
ignore or generate a command at its earliest convenience that
would probably be "RSET" or "QUIT".

>...
> At this point, maybe Hansen had the best (personal hat on)
> advice:
> 
>      "don't do anything now that would prevent such an
> extension
>       from being written."

And no such thing has been proposed.  By anyone.

    john

Date: Wed, 11 Apr 2007 14:00:03 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> What it probably interested here is that its uses a non-standard less
> than 100 code for "informational", so something like this is valid under
> SendMail.
>
>     099-Informational
>     099-Informational
>     099-Informational
>     099-Informational
>     250 whatever

Or even:
   099 Info
   099 Info
   250 Whatever

> Very interesting.

Unused in practice, I believe.

[...]

> I'll say it again this way: this is a common "method" concept in C/S
> (Client/Server) designs, especially in a TCP/IP environment. The point
> here is that C/S (or even P2P) communication timeouts is generally based
> on channel idleness.

I don't disagree.  I merely make the point that a server SHOULD NOT
rely on this design.  I also make the point that modern conditions
on the Internet MAY make a client choose an overall timeout rather
than an idleness timeout.

>> I believe the RFC should be
>> reworded to make it clear that all reply codes in a multiline reply
>> MUST be the same code.  

> That would be an unnecessary restriction. Clients or servers doen't
> require this tightness.

I believe it was intended by the original authors.  In fact, I think
they didn't even imagine the possibility of different return codes on
different lines.  But I don't know for sure; we'd have to ask.

> IMV, this is backwards. This is a server driven control concept.

No, it's *not*.  A server *cannot* expect a client to increase its
timeout just because the server wants it to.  A client and server must
*agree* on a mechanism for processing-in-progress messages.  Just
because your hack happens to work with many SMTP clients today doesn't
mean it's a good design or should be blessed by an RFC.

Regards,

David.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Date:  Wed, 11 Apr 2007 19:56:35 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

[email-masked] wrote:

> there's plenty of "running code" in this space - for example, there are
> heaven knows how many nonstandard header field in use, but aside from the
> attempts by Jacob Palme and other third parties leading up to the current
> header registration model, how many independent RFC submissions have
> there been by authors seeking to document their header usage?

BTW, RFC 3864 is an _excellent_ model for an "expert review" registry:

It allows to "reserve" header fields with provisional registrations
(example: Archived-At) while they're not yet standardized, it allows
to deprecate non-standard header fields (example: Errors-To), and if
a header field finally makes into an RFC (or similar, but we can ignore
"similar" for status codes), it's added to the "permanent" registry.

With that rules in place the old x.7.8 defined in an I-D could have
been registered provisionally, avoiding the current mess.

Models without a "provisional registry" don't avoid such collisions in
practice, they only avoid the collisions in standards published years
later.

Frank

Date: Wed, 11 Apr 2007 13:51:46 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

John C Klensin wrote:

> Proposed text:
> 
> 		The command has been accepted, but the requested action
> 		is being held in abeyance, pending confirmation of the
> 		information in this reply.  The SMTP client should send
> 		another command specifying whether to continue or abort
> 		the action.  Note: This specification does not define
> 		any commands that allow this type of reply, nor is there
> 		a continuation command for use with it.   Such commands,
> 		or modifications to the behavior of existing commands,
> 		and semantics for these codes could be specified using
> 		the extension mechanism.
> 

John, for the sake of the group progress, if we are talking about 
non-continuation lines only, then I'm going to vote "Fine", +1.

But IMV, it still has the implied semantic argument that it CANNOT be 
used without an extended SMTP consideration and if that is what you are 
seeking, then why not spell it out?  "It MUST NOT used by SERVERS."

The way I proposed to structured it left it "open ended" for new designs 
and just laying down the idea that the current scope is undefined.

Just consider the new discoveries with SENDMAIL:

   - It doesn't care,
   - it Ignore junk lines,
   - It has built-in support for non-standard 0xx- "Informational" lines,
   - It supports the LAST REPLY LINE concept as suggested by 4.2.1

So its really a non-issue as far as running code.

Anyway, I'll leave it with your final judgment on this.  I appreciated 
your consideration on this.

--
HLS

Date: Wed, 11 Apr 2007 13:30:09 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> This RFC 821 (clone in 2821) "recommendation" in question:
> 
>>     In many cases the sender-SMTP then simply needs to search for
>>     the reply code followed by <SP> at the beginning of a line, and
>>     ignore all preceding lines.  In a few cases, there is important
>>     data for the sender in the reply "text".  The sender will know
>>     these cases from the current context
> 
> It's not a recommendation.  It's merely an illustration of one
> possible implementation.

Like we have many to choose from :-)

> 
> Anyway, I read the source code to Sendmail 8.14.0.  It's not exactly
> easy to follow, but this is what it does:  Sendmail uses the reply
> code from the LAST line, but the text message from the FIRST line.
> So if it receives:
> 
>    150-One second please
>    250 Message accepted
> 
> it returns a reply code of 250 with a text part of "One second please".
> (Yes, this is weird behaviour and could lead to very confusing log messages.)

I just downloaded it (Chef Boraydee <g>).

What I see is:

   - checks for valid lines ("XYZ" "XYZ-" or "XYZ ")
     and ignores the rest (continues reading). Junk
     lines are ignored.

   - does an atoi() to get the reply code which has a natural
     delimiter for non-digits

   - if the code is less 100, continue reading

   - breaks on the non-continuation line, which coupled with the
     above, it returns the last reply code.

So from this, as I expected, SENDMAIL does not error out with 
continutation lines (i.e, 150-) when reading reply lines for ALL states.

What it probably interested here is that its uses a non-standard less 
than 100 code for "informational", so something like this is valid under 
SendMail.

     099-Informational
     099-Informational
     099-Informational
     099-Informational
     250 whatever

Very interesting.

>> Also, I stand my engineering that most, if not all, even SENDMAIL until
>> shown otherwise, will reset their timers when the channel is active again.
> 
> In the case of Sendmail, you are correct.  It does reset the timer
> when it receives a line.  But as I wrote, that's an implementation
> detail and there are perfectly valid reasons why you should
> have an overall timeout.

I'll say it again this way: this is a common "method" concept in C/S 
(Client/Server) designs, especially in a TCP/IP environment. The point 
here is that C/S (or even P2P) communication timeouts is generally based 
on channel idleness.  Hence, I wasn't surprise SendMail also falls in 
the same group.  We can call it an implementation detail if we like, we 
are not going to find too many systems that deviate from this 
fundamental design concept.

Yes, overall timeout is fine, That is an implementation detail that is 
not found across the board.  A great idea to have, but I would not be 
surprise if its not utilitzed across all systems.

In my SMTP design experience, the basic overall concern has been 
unexpected or ungraceful DROPS and not really trying to regulate how 
long a session is taken simply because there is no hard core rule on how 
long a total session should take.  But I agree, this are necessary 
considerations.

If we wanted to juggle this idea, we can say that a maximum transaction 
time for a CLIENT would be:

    GREETTING 5
    HELO/EHO  5
    MAIL FROM 5
    RCPT TO   5
    DATA      10 (5 for 352 response, 5 for upload response)
    QUIT      5

Or 25 or 35 minutes if how you want to include DATA/QUIT.

> Permitting different reply codes in a multiline reply is confusing and
> of doubtful utility.  

The only current useful I see is KEEP ALIVE.  I can't think of any other 
need.  But as you can see from SENDMAIL it obviously found a need for 
what it calls "informational" reply lines.

> I believe the RFC should be
> reworded to make it clear that all reply codes in a multiline reply
> MUST be the same code.  

That would be an unnecessary restriction. Clients or servers doen't 
require this tightness.

> Then leave it up to an extension to negotiate
> keepalive if the client wants it and the server is willing.

IMV, this is backwards. This is a server driven control concept. The 
client may have restrictions but it is the server that will drive this, 
not the client.  The server determines if its something it would want to 
do to process the transaction.

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Date:  Wed, 11 Apr 2007 19:19:11 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 6
Sender: [email-masked]

Tony Finch wrote:

> Fine.

+1

Date: Wed, 11 Apr 2007 17:27:44 +0100
From: Tony Finch <[email-masked]>
To: "David F. Skoll" <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

On Wed, 11 Apr 2007, David F. Skoll wrote:
>
> Anyway, I read the source code to Sendmail 8.14.0.  It's not exactly
> easy to follow, but this is what it does:  Sendmail uses the reply
> code from the LAST line, but the text message from the FIRST line.

Oops, looks like I misread the code yesterday. Sendmail also saves the
first line in another buffer if its first character is 4. The buffer will
be left containing garbage if the server uses inconsistent reply codes
ending in 4yz, which I guess would cause Sendmail to log a bogus reason
for deferring the message.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FAEROES SOUTHEAST ICELAND: SOUTHEAST VEERING SOUTHWEST 5 OR 6, BACKING SOUTH 7
OR GALE 8, PERHAPS SEVERE GALE 9 LATER. ROUGH OR VERY ROUGH . RAIN OR DRIZZLE
AT TIMES. MODERATE OCCASIONALLY POOR.

From: [email-masked]
Cc: [email-masked], Frank Ellermann <[email-masked]>, [email-masked]
Date: Wed, 11 Apr 2007 08:25:09 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Tuesday, 10 April, 2007 17:24 -0700
> [email-masked] wrote:

> >...
> > Enhanced status codes, OTOH, are only used by a single
> > protocol - SMTP - that's incredibly widely deployed.
> > Interoperability and security considerations are of paramount
> > importance here. It is therefore essential that any codes that
> > are defined be carefully and completely reviewed for conflicts
> > and clarity and there can easily be serious security
> > considerations attached to their definition and use. While I
> > think the chances are excellent that the IESG would, upon
> > reviewing any independent submission that defined such codes
> > improperly, tell the RFC Editor that the document is in
> > conflict with the IETF's ongoing work on SMTP, that therefore
> > it should not be published, and that the RFC Editor would
> > agree, I don't want to take even a small risk.

> While I agree with you in principle, I think we need to be a bit
> careful here.  If the goal is to prevent the same code from
> being used for different purposes, I think it is an unpleasant
> necessity to recognize and register codes acquired by squatting
> based on non-conforming code that has become deployed widely
> enough to be significant.

If so, that is NOT an argument for the "RFC Required" model - I see essentially
no chance that a code-squatter is going to write an RFC of any sort, let alone
write one but then send it to the RFC Editor instead of putting it through the
IETF process.

And there's plenty of "running code" in this space - for example, there are
heaven knows how many nonstandard header field in use, but aside from the
attempts by Jacob Palme and other third parties leading up to the current
header registration model, how many independent RFC submissions have there been
by authors seeking to document their header usage?

> Have a look at the registration-specification language in
> draft-klensin-smtp-code-registry-00 (a parallel development to
> Tony's draft that I hope will be quickly consolidated with it)
> and let us know what you think.

You have gone with the "specification required" approach. This is in one sense
more lenient than the "RFC Required" since it allows for non-RFC specificaitons
and would avoid the problem that code-squatters rarely if ever seem to be
willing to write RFCs to document their usage.

However, in another sense it is less lenient since it requires approval by a
designated expert. I think this is sufficient (barely) to address my concerns
about overlapping or poorly defined codes getting into the registry.

However, one of the requirements for expert review is that the review
criteria be clearly spelled out in the registration document. I don't
think what you have is sufficient in this regard. I think it needs to have
a list of requirements, such as:

(1) The meaning of the code must be clearly defined.
(2) The code most not duplicate the functionality of a previously defined
    code.
(3) Whenever possible the use-cases new codes should not overlap the use-cases
    for existing codes, except when a new code is intended to allow
    identification of a more specific subcase of an existing code.

Finally, the document should say that exceptions to the rules can be made
when an unregistered code has achieved significant deployment since
the main purpose of the registry is to document all code use in order to avoid
confusion and prevent multiple different definitions for the same code.

				Ned

Date: Wed, 11 Apr 2007 11:43:41 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> This RFC 821 (clone in 2821) "recommendation" in question:

>     In many cases the sender-SMTP then simply needs to search for
>     the reply code followed by <SP> at the beginning of a line, and
>     ignore all preceding lines.  In a few cases, there is important
>     data for the sender in the reply "text".  The sender will know
>     these cases from the current context

It's not a recommendation.  It's merely an illustration of one
possible implementation.

Anyway, I read the source code to Sendmail 8.14.0.  It's not exactly
easy to follow, but this is what it does:  Sendmail uses the reply
code from the LAST line, but the text message from the FIRST line.
So if it receives:

   150-One second please
   250 Message accepted

it returns a reply code of 250 with a text part of "One second please".
(Yes, this is weird behaviour and could lead to very confusing log messages.)

> Also, I stand my engineering that most, if not all, even SENDMAIL until
> shown otherwise, will reset their timers when the channel is active again.

In the case of Sendmail, you are correct.  It does reset the timer
when it receives a line.  But as I wrote, that's an implementation
detail and there are perfectly valid reasons why you should
have an overall timeout.

Permitting different reply codes in a multiline reply is confusing and
of doubtful utility.  I do not think it should be blessed in an RFC,
even if it's only in a MAY section.  I believe the RFC should be
reworded to make it clear that all reply codes in a multiline reply
MUST be the same code.  Then leave it up to an extension to negotiate
keepalive if the client wants it and the server is willing.

Regards,

David.

Date: Wed, 11 Apr 2007 16:19:06 +0100
From: Tony Finch <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Philip Guenther wrote:
>
> Personally, this seems more like a mental exercise in how 1yz reply codes
> could be utilized than a particularly good starting point for dealing with the
> timeout/keepalive problem.  Why settle for just keepalives, which don't help
> in the presence of real connection failure, when something more along the
> lines of the CHECKPOINT extension (RFC 1845) would let the problem be *solved*
> instead of just made less likely?

http://www.ietf.org/internet-drafts/draft-fanf-smtp-rfc1845bis-01.txt :-)

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FISHER GERMAN BIGHT: WEST 4 OR 5, BECOMING VARIABLE 3 OR LESS. MODERATE
OCCASIONALLY ROUGH IN GERMAN BIGHT. FOG PATCHES LATER. MODERATE OR GOOD,
OCCASIONALLY VERY POOR LATER.

Date: Wed, 11 Apr 2007 08:28:26 -0600
From: Philip Guenther <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Hector Santos wrote:
...
> What you are implying is that SENDMAIL/EXIM is not FOLLOWING the continuation 
> lines correctly, taken the first line results and ignoring all remaining all 
> other lines in the response buffer.

I don't know about exim, but sendmail doesn't precisely ignore them.  If 
you're wondering what processing it does on them, well, you should 
download the source and eyeball it: usersmtp.c, reply().

> That is obvious broken CODE behavior and would be for over 25 years.

Please point to any text in RFC 821 that states that multiple *different* 
reply codes may be returned for a single command.

> And for some reason, I beginning to doubt your claim that it will break.
>
> I find that completely unbelieve that SENDMAIL/EXIM will error out on this 
> type of usage:

Again, the code is out there.  Looks to me like it's behaved like this for 
over a dozen years.

Philip Guenther

Date: Wed, 11 Apr 2007 10:03:02 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

David F. Skoll wrote:

>> The specific "keep alive" functionality still needs to widely tested.
>> But the proof of concept is well established.  It does work and there is
>> no surprise from a programmability technical standpoint that it does.
> 
> It does not work, apparently, if the SMTP client is Sendmail, which
> I'm told treats the 150- as an error.  Sendmail is still a
> sufficiently-popular SMTP client that any server using 150-
> continuation lines will break on a large proportion of incoming
> e-mail.

This RFC 821 (clone in 2821) "recommendation" in question:

     In many cases the sender-SMTP then simply needs to search for
     the reply code followed by <SP> at the beginning of a line, and
     ignore all preceding lines.  In a few cases, there is important
     data for the sender in the reply "text".  The sender will know
     these cases from the current context

has been in place for 25 years.

For the exact reason we all know that SENDMAIL is popular I find it 
rather odd that in these 25 years this would the first time it has 
encountered such considerations, so by that I am beginning to doubt that 
claim.  I believe the devils in the details, especially since there are 
many versions of SENDMAIL.

Also, I stand my engineering that most, if not all, even SENDMAIL until 
shown otherwise, will reset their timers when the channel is active again.

--
HLS

Date: Wed, 11 Apr 2007 08:00:45 -0600
From: Philip Guenther <[email-masked]>
To: Tony Finch <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Tony Finch wrote:
> On Tue, 10 Apr 2007, william(at)elan.net wrote:
>>
>> How would you see it working with extension ID?
>
> Client sends KEEPALIVE <interval> at the start of the session (or again
> later to change the interval). Server sends 1yz reply after interval if
> normal reply is taking longer. Client responds to 1yz with CONTINUE or
> RSET. This screws up pipelining so the server probably should not offer
> pipelining and keepalive, or perhaps the keepalive command turns off
> pipelining.

Personally, this seems more like a mental exercise in how 1yz reply codes 
could be utilized than a particularly good starting point for dealing with 
the timeout/keepalive problem.  Why settle for just keepalives, which 
don't help in the presence of real connection failure, when something more 
along the lines of the CHECKPOINT extension (RFC 1845) would let the 
problem be *solved* instead of just made less likely?

Philip Guenther

Date: Wed, 11 Apr 2007 14:38:13 +0100
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

On Wed, 11 Apr 2007, John C Klensin wrote:
>
> 		The command has been accepted, but the requested action
> 		is being held in abeyance, pending confirmation of the
> 		information in this reply.  The SMTP client should send
> 		another command specifying whether to continue or abort
> 		the action.  Note: This specification does not define
> 		any commands that allow this type of reply, nor is there
> 		a continuation command for use with it.   Such commands,
> 		or modifications to the behavior of existing commands,
> 		and semantics for these codes could be specified using
> 		the extension mechanism.

Fine.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
VIKING NORTH UTSIRE: SOUTH, VEERING WEST FOR A TIME, 3 INCREASING 4 OR 5.
ROUGH OR VERY ROUGH. RAIN OR DRIZZLE, THEN FAIR. MODERATE OR POOR BECOMING
GOOD.

Date: Wed, 11 Apr 2007 09:35:33 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Finch wrote:
> On Wed, 11 Apr 2007, Hector Santos wrote:
>>      In many cases the sender-SMTP then simply needs to search for
>>      the reply code followed by <SP> at the beginning of a line, and
>>      ignore all preceding lines.  In a few cases, there is important
>>      data for the sender in the reply "text".  The sender will know
>>      these cases from the current context.
> 
> That is a suggestion, not normative text.

It is a technical requirement a xyz-DASH is continuation line.

Given two possible OUTCOMES:

1) single line

    250 xxxxxxxxxx

2) multiple line:

    250-xxxxxxxxxx
    250 xxxxxxxxxx

It is clearly understood that only outcome #1 is what matters most. 
Otherwise the system will not work because 99.99% of the time that is 
what clients see.

What you are implying is that SENDMAIL/EXIM is not FOLLOWING the 
continuation lines correctly, taken the first line results and ignoring 
all remaining all other lines in the response buffer.

That is obvious broken CODE behavior and would be for over 25 years.

And for some reason, I beginning to doubt your claim that it will break.

I find that completely unbelieve that SENDMAIL/EXIM will error out on 
this type of usage:

    S: 352
    S: 150-
    S: 250  or 45x or 55x

And it does, its broken.

--
HLS

Date: Wed, 11 Apr 2007 14:31:57 +0100
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

Ned Freed wrote:

>> [email-masked] wrote:
>
>> >>5.7.8, 5.7.9, 5.7.11, 4,7.12, 5.7.12, and x.5.6.
>> >>
>> >>
>> >All of these have are consistently defined in 2554bis and in Tony's 
>> draft. If
>> >Tony's draft moves forward and is approved before 2554bis
>> >
>> This wouldn't be possible, as 2554bis is already approved ;-).
>
> Read the rest of my sentence - I was referring to publication of 
> 2554bis, not
> approval. There have been plenty of cases where a draft has been 
> approved and
> then awaited publication for months or even years while numerous other 
> related
> drafts are written, approved, and even published. This is especially 
> true with
> security-related documents.

Fair enough.

> In the specific case of 2554bis I don't see anything that would delay
> publication - in particular, all normative references are to published 
> RFCs.
> But even so, it is possible, albeit unlikely,

Yes, I think it is unlikely.

> that an IANA registry for
> enhanced status codes could be approved before 2554bis is published, 
> in which
> as I noted below a revision with an RFC Editor note would be 
> appropriate if
> not required.

It might be just as easy to add registration of 2554bis enhanced status 
codes to Tony's/John's draft.

Date: Wed, 11 Apr 2007 07:28:10 -0600
From: Philip Guenther <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Hector Santos wrote:
> Tony Finch wrote:
...
>> Absolutely not. You CANNOT break running code like this.
>
> That should not break running code, but merely highlight that SENDMAIL and 
> EXIM didn't follow a 25 year old RFC 821 specification to begin with:
>
>     In many cases the sender-SMTP then simply needs to search for
>     the reply code followed by <SP> at the beginning of a line, and
>     ignore all preceding lines.  In a few cases, there is important
>     data for the sender in the reply "text".  The sender will know
>     these cases from the current context.

That's a description of a possible implementation strategy and not a 
requirement that implementations only use the reply code in the final 
line.  How can it be a requirement when it doesn't say when it applies?

Note again the use of "the reply code" in that text, implying that the 
same reply code must be used.  If more than one reply code could appear in 
a multiline reply, then the text would have used "a reply code" to reflect 
their multiplicity, or maybe "one of the reply codes".  Indeed, the very 
description of multiline replies seems like the strongest argument that 
the same code must be used:

          The format for multiline replies requires that every line,
          except the last, begin with the reply code, followed
          immediately by a hyphen, "-" (also known as minus), followed by
          text.  The last line will begin with the reply code, followed
          immediately by <SP>, optionally some text, and <CRLF>.

How can there be different reply codes when _the_ reply code is at the 
start of every line?  If there had been any conception of using different 
reply codes on some or all continuation lines, that text would have 
referred to "a reply code" or "some reply code".

How can sendmail and exim fail to comply when there's no suggestion that 
the last reply line may be different in any way but rather evidence to the 
contrary?

> It makes them non-compliant.

I obviously disagree.

> I think the NEW sentence for the NEW SPEC 
> should include a clarification such as:
>
>  The client SHOULD use the final line as the ultimate
>  server response.

Huh.  So, you think an implementation that doesn't use the final line 
doesn't comply with 821, but you want to add text so that they *would* be 
(conditionally) compliant to the new spec.  That means that using 
different reply codes in continuation lines would fail to interoperate 
with conditionally compliant client implementations, so they would be 
impossible to use reliably.  You seem to be sabotaging your goal.

Philip Guenther

Date: Wed, 11 Apr 2007 09:25:26 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:

> Hector,
> 
> One small clarification, just to be sure we understand where we
> disagree, if at all.  I am not proposing this, just trying to
> understand...
> 
> Suppose 2821bis were clarified in just the way I think you have
> been arguing against.   In other words, it clearly said:
> 
> 	(i) In a multiline reply, all lines MUST use the same
> 	reply code.
> 	
> 	(ii) 1yz reply codes MUST NOT be sent by a server unless
> 	they are specified by a specific extension that
> 	overrides this provision of the specification for a
> 	specific purpose.
> 	
> 	(iii) multiline replies MUST NOT be used as an attempt
> 	to affect client timing or timeouts.
> 
> I hope everyone understands that nothing in 2821 prevents an
> extension from being designed, proposed, and adopted that, if
> appropriately agreed to between server and client, changes any
> or all of those rules.   If 2821 is not clear on that topic,
> then it should be and I would welcome appropriate text for
> 2821bis.
> 
> To the extent to which the concern is that making these rules
> more specific (or implied prohibitions more clear) in 2821bis
> would prevent such extensions, I think we need to clarify that
> the concern is groundless.
> 
> Does that bring our positions closer together?

If I understand the question, you are saying that an I-D proposal can 
update and even go as far as change an RFC and my "assumption" that it 
can not is an incorrect assumption?

It is my understanding that an proposal can propose changes to current spec.

But I have a hard time answering that because whether you believe it or 
not, I am extremely pragmatic and conservative. I don't want to break 
operations for anyone nor for us. Thats the last thing I want.

Here is my opinion above the above items:

I think it would be mistake to have above, but I can see the benefits of 
locking it down. I am for 100% technical clarity and 100% SMTP 
compliant.  I don't (iii) is necessary because it is SUBJECTIVE plus you 
have no control over this usage.

If you write into stone the above three clarifications,  I extremely 
doubt anyone will have the incentive to do any more beyond that because 
if they do, they will begin to break systems. I know I wouldn't want to 
go there. Paraphrase: It would have to be an extremely groundbreaking 
new concept viewed as having a high acceptance and high adoptability 
aspect to it before I would want to waste time on such a new proposal or 
even consider it for implementation.

Throwing aside all the IETF regulations for "ratifying" this new RFC,
I frankly believe this is a natural element of the protocol and ideally 
should be part of this document for one reason - it is compatible for 
most systems and those are are not compatible are technically 
non-compliant with not just 2821bis but also 821.

But I can understand the IETF restrictions or framework you guys are in 
for this 2821bis work.  I do understand.  So all I can do is hopefully 
structure my input as best as I can to make it a consideration.  If not, 
so be it.

At this point, maybe Hansen had the best (personal hat on) advice:

     "don't do anything now that would prevent such an extension
      from being written."

--
HLS

Date: Wed, 11 Apr 2007 09:25:55 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> IMO, I don't think it will be good engineering on the client part if it
> does not reset a SMTP timeout when reading each line buffered or not. As
> long as each line begins with a XYZ- number, its all good

I disagree.  Now that some people have the (misguided) view that
tarpitting is a good thing, and they sometimes incorrectly trigger
tarpitting on valid e-mail, I think a defensive client *should*
enforce an overall timeout on a command-and-complete-reply.  If you're
a very large e-mail sender, you really don't want clients on your main
machine tied up for too long -- if the SMTP server is being slow, you
want to time out and shunt the mail to a machine dedicated to dealing
with slow recipients.

Regards,

David.

Date: Wed, 11 Apr 2007 09:25:04 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies -- proposed text
Sender: [email-masked]

Hi.

In the hope of settling this and moving on, I propose to modify
the text for "1yz" to read as follows.  This text is based on
trying to combine Hector's proposal with a need to avoid getting
entangled with interpretations of "unextended SMTP".  

The multiline issues are not addressed: they don't belong in
this paragraph.  In addition, even if those were resolved in the
direction of "only the last one is significant", that still
wouldn't justify treating the other lines as noise to the extent
that the use of 
    150- ...
    250 ...
would be any more justified than
    000- ...
    250  ...
or
    666- ...
    250 ...

Proposed text:

		The command has been accepted, but the requested action
		is being held in abeyance, pending confirmation of the
		information in this reply.  The SMTP client should send
		another command specifying whether to continue or abort
		the action.  Note: This specification does not define
		any commands that allow this type of reply, nor is there
		a continuation command for use with it.   Such commands,
		or modifications to the behavior of existing commands,
		and semantics for these codes could be specified using
		the extension mechanism.

Comments welcome, but please read this in the context of an
attempted compromise that may not completely satisfy everyone
rather than as a proposal that is in complete agreement with the
many suggestions that have been made about this and related
topics.

     john

Date: Wed, 11 Apr 2007 09:18:35 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

> But I always thought it wasn't necessary, and simply having a
> clarification on three key questions would be sufficient:

>   - Must all the reply codes be the same?

Yes, IMO.

>   - If not, the final return code (last line) is valid code?

Not applicable.

>   - 150 SHOULD/MUST NOT be used without extended clients?

Not applicable, because "all reply codes MUST be the same" takes
care of this.

> The specific "keep alive" functionality still needs to widely tested.
> But the proof of concept is well established.  It does work and there is
> no surprise from a programmability technical standpoint that it does.

It does not work, apparently, if the SMTP client is Sendmail, which
I'm told treats the 150- as an error.  Sendmail is still a
sufficiently-popular SMTP client that any server using 150-
continuation lines will break on a large proportion of incoming
e-mail.

Regards,

David.

Date: Wed, 11 Apr 2007 14:04:40 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Wed, 11 Apr 2007, Hector Santos wrote:
>
>      In many cases the sender-SMTP then simply needs to search for
>      the reply code followed by <SP> at the beginning of a line, and
>      ignore all preceding lines.  In a few cases, there is important
>      data for the sender in the reply "text".  The sender will know
>      these cases from the current context.

That is a suggestion, not normative text.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL: SOUTHWEST 5 OR 6, BACKING SOUTH 6 TO GALE 8. MODERATE OR ROUGH
BECOMING VERY ROUGH. FAIR. MODERATE OR GOOD.

Date: Wed, 11 Apr 2007 08:39:28 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, "David F. Skoll" <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

--On Tuesday, 10 April, 2007 14:50 -0400 Hector Santos
<[email-masked]> wrote:

> 
> David F. Skoll wrote:
>> Hector Santos wrote:
>> 
>>>     Under most scenarios, servers SHOULD use the same reply
>>>     code for each multiline response line when it is going to
>>>     be displayed at the same time. However there are
>>>     scenarios (Keep Alive Client Timeout issues) where
>>>     servers MAY use "150-" as an alternative reply code in a
>>>     continuation line when the server needs to issue a
>>>     heartbeat to keep the client alive during unsual delayed
>>>     backend processing:
>> 
>> I don't like that.  There's no reason to assume that parts of
>> a multiline response will have any effect on the client
>> timeout.  It might not reset its timer just because it gets
>> part of a reply back.  Maybe many current implementations do,
>> but that's just good luck.
>> 
>> I believe that a proper negotiated ESMTP extension for
>> "keepalive" intermediate replies should be developed.  We
>> shouldn't try to hack something in.
> 
> Hi David,
> 
> I agree 100% and this issue began a few years back including
> during the last previous call out for 2821bis discussions
> where I brought the "Keep Alive" consideration.  So John is
> very aware this isn't a first time discussion.  I might just
> go ahead and write a I-D finally.  Back then I had never
> written an I-D. But with a few under my belt now, I am more
> familiar with the process and more comfortable writing one if
> need be.
> 
> However, for 2821bis, I just would like make sure that it
> doesn't pre-empt such new efforts.   Thats really all I would
> like to seek here and thus I go back to my original post to
> Frank with a suggested text, which was really just a starting
> point and would be very happy for a documentation person to
> reword it appropriately:
>...
 
Hector,

One small clarification, just to be sure we understand where we
disagree, if at all.  I am not proposing this, just trying to
understand...

Suppose 2821bis were clarified in just the way I think you have
been arguing against.   In other words, it clearly said:

	(i) In a multiline reply, all lines MUST use the same
	reply code.
	
	(ii) 1yz reply codes MUST NOT be sent by a server unless
	they are specified by a specific extension that
	overrides this provision of the specification for a
	specific purpose.
	
	(iii) multiline replies MUST NOT be used as an attempt
	to affect client timing or timeouts.

I hope everyone understands that nothing in 2821 prevents an
extension from being designed, proposed, and adopted that, if
appropriately agreed to between server and client, changes any
or all of those rules.   If 2821 is not clear on that topic,
then it should be and I would welcome appropriate text for
2821bis.

To the extent to which the concern is that making these rules
more specific (or implied prohibitions more clear) in 2821bis
would prevent such extensions, I think we need to clarify that
the concern is groundless.

Does that bring our positions closer together?

     john

Date: Wed, 11 Apr 2007 08:24:52 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 10 Apr 2007, Hector Santos wrote:
>>    Only the final or last reply-code is important in a multiline
>>    response. The client MUST|SHOULD use the final line as the ultimate
>>    server response.
> 
> Absolutely not. You CANNOT break running code like this.

That should not break running code, but merely highlight that SENDMAIL 
and EXIM didn't follow a 25 year old RFC 821 specification to begin with:

      In many cases the sender-SMTP then simply needs to search for
      the reply code followed by <SP> at the beginning of a line, and
      ignore all preceding lines.  In a few cases, there is important
      data for the sender in the reply "text".  The sender will know
      these cases from the current context.

It makes them non-compliant.  I think the NEW sentence for the NEW SPEC 
should include a clarification such as:

   The client SHOULD use the final line as the ultimate
   server response.

and it would be still be inline with a 25 year old SPEC.

The next question is whether SENDMAIL/EXIM is going to fix their code. 
:-)  Which BTW, are you referring to current versions or old versions? 
and you referring to just "150-" or "150 " when you say it breaks them?

--
HLS

Date: Wed, 11 Apr 2007 08:23:33 -0400
From: John C Klensin <[email-masked]>
To: Tony Finch <[email-masked]>, "william(at)elan.net" <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

--On Wednesday, 11 April, 2007 12:07 +0100 Tony Finch
<[email-masked]> wrote:

> On Tue, 10 Apr 2007, william(at)elan.net wrote:
>> 
>> How would you see it working with extension ID?
> 
> Client sends KEEPALIVE <interval> at the start of the session
> (or again later to change the interval). Server sends 1yz
> reply after interval if normal reply is taking longer. Client
> responds to 1yz with CONTINUE or RSET. This screws up
> pipelining so the server probably should not offer pipelining
> and keepalive, or perhaps the keepalive command turns off
> pipelining.

Yep.   I think it is also possible to model this as slightly
server-controlled with client confirmation, as outlined in one
of my earlier notes.   But all of them have, I think, the
properties that:

	(1) There is some specific negotiation/ agreement that
	overrides the timeouts that would be exhibited by a 2821
	implementation with no extensions advertised or
	requested.
	
	(2) The model of "send a command, get a unitary and
	homogeneous (even if multi-line) reply" is not altered
	through some back door that some 2821-conforming
	implementations may not observe (or even notice).

john

Date: Wed, 11 Apr 2007 08:16:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Douglas Otis <[email-masked]>, Tony Finch <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

John C Klensin wrote:

>>      Note: unextended SMTP does not have any commands that
>>      allow this type of reply, and so does not have continue
>>      or abort commands.
> 
> That is clearly wrong.   By the time 2821 came around, it
> probably should have been modified to say "...a continue command
> to correspond to it" might have been thinking in terms of actual
> CONTinue  and ABORt commands, the latter presumably with
> different semantics than RSET (although, right now, I can't
> imagine what they might be).   I will make that fix unless the
> group agrees on some broader change (or concludes that the
> "Note" should be dropped entirely).  Sorry for not understanding
> what you were referring to earlier.

No problem john. I vote towards dropping the note, or clarify it with 
something to akin to this:

    Note: This specification does not have any current commands
    that defines usage for 150 non-continuation
    COMPLETION reply-code.  As such, there is no guidance as
    to what command a client can issue next or what state
    the client should move into in response to a server 150
    reply-code.  Such definitions are out of scope and primarily
    intended for Extended SMTP definitions.

> Note that FTP (see RFC765), on which SMTP was modeled, does have
> a 1yz reply, but its semantics are exactly what you are trying
> to accomplish with the 

That's correct.  All this had stem from FTP work where a keep alive 
concept was necessary to resolve FTP CLIENT 30 second socket timeouts 
issues. That engineering was extended to SMTP when it was becoming 
apparent with new dynamic backend operations was a high consideration, 
AVS, OPES, DKIM, etc.

>>      C: DATA
>>      S: 352 Begin sending your data
>>         [client uploads data]
>>      S: 150-DKIM signature found!
>>      S: 150-Please wait while we process your DKIM message!
>>      S: 150-Waiting for trust server to respond...
>>      S: 150-Still waiting for trust server to respond...
>>      C: 550-Sorry, The DKIM POLICY has failed this transaction
>>      C: 550 Please see http://trust-r-us.com/msgid=123213123
>>
>> Who knows?  Is this not possible?  It is out of the whelm of
>> real possibility?

Side note: Someone pointed out that typo above. Thought it should be 
mentioned: The last two lines should be server S: lines:

> No, it is is clearly not impossible.  However, it raises two
> issues in addition to the one about the codes all being the same
> which, IMO (and in the text of 2821), arises from one of them.  
> 
> One is that it is a perfectly conforming and rational
> implementation of SMTP as now defined for the client to do
> nothing but buffer a continued set of replies until the last one
> arrives.  That would result in the "please wait" part of those
> 150 reply lines having no client effect and, in particular, no
> effect on any timeouts the client imposes on a continued set of
> reply codes.

I am not sure I agree because buffered or not, the client must look for 
each EOL. But more important, the timeouts are typically programmed into 
the socket timeout as well.  It depends on how the socket I/O is done. 
It can be a telnet like session with interactive character I/O or its 
buffered until a timeout or a EOL is sensed or both.

I think the more robust design is to MAKE sure each line begins with a 
numeric xyz code. If there is garbage (non-numeric), how a client reacts 
is undefined, but most likely abort.  In C, it is easy since atoi() is 
delimiter with a non-numeric character or null.

IMO, I don't think it will be good engineering on the client part if it 
does not reset a SMTP timeout when reading each line buffered or not. 
As long as each line begins with a XYZ- number, its all good

Now, is there a limit?

There might be. We have a 100 line limit at the welcome because we found 
a few zombies that spew out non-stop "220-" lines.

(Thats another note that might be considered - is there a limit in the 
total lines for a multi-line response?)

> Second, if the client does time out, one runs directly afoul of
> the message-duplication principles of RFC 1047 (affirmed by 1123
> and 2821).  

Right, thus why the server needs the ability to keep the client live in 
such scenarios.

> For other reasons, I essentially substituted "some random
> protocol that might need to process or validate the content" for
> "DKIM" in your examples and discussion above.  Certainly one can
> imagine several families of such protocols.  The real question
> is whether, with SMTP as now defined (including the application
> of the mandate that formally originated with RFC 1047 to
> minimize time between EOD (CRLF.CRLF) and a reply indicating
> message receipt, one can define something in which extended and
> potentially time-consuming processing at that point makes sense.

Agree. I agree that the extension would be useful, but I also think 
2821bis clarification would have a current transparent benefit as well 
and also prepare such new extensions.

In in mind that DATA is the best example here, but in general, it also 
applies to all states with their own 5 minute recommended minimal timeout.

    C: MAIL FROM:<foobar @ whereever.com>
    S: 250 Ok, Return Path Verification Pending...
    C: RCPT TO: <someuser @ host.com>

       [for some reason, backend is taking long time, 4 mins uses up,
        so it sends 150- with the idea of keeping the client alive]

    S: 150-Please wait, still trying to verify...
    S: 250 Its all good, continue.

Is that desirable?   Of course not.

Is it possible?  Yes

Should the backend put a 5 minute cap on the backend thread residence 
time?   That would be a design goal for maximum reliability.

In practice, we have a maximum time local policy 5 minutes default 
maximum time and I can't recall a situation off hand where its taken 
longer than 1 minute if that.

--
HLS

Date: Wed, 11 Apr 2007 07:59:17 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

--On Tuesday, 10 April, 2007 17:24 -0700
[email-masked] wrote:

>...
> Enhanced status codes, OTOH, are only used by a single
> protocol - SMTP - that's incredibly widely deployed.
> Interoperability and security considerations are of paramount
> importance here. It is therefore essential that any codes that
> are defined be carefully and completely reviewed for conflicts
> and clarity and there can easily be serious security
> considerations attached to their definition and use. While I
> think the chances are excellent that the IESG would, upon
> reviewing any independent submission that defined such codes
> improperly, tell the RFC Editor that the document is in
> conflict with the IETF's ongoing work on SMTP, that therefore
> it should not be published, and that the RFC Editor would
> agree, I don't want to take even a small risk.

Ned,

While I agree with you in principle, I think we need to be a bit
careful here.  If the goal is to prevent the same code from
being used for different purposes, I think it is an unpleasant
necessity to recognize and register codes acquired by squatting
based on non-conforming code that has become deployed widely
enough to be significant.

Have a look at the registration-specification language in
draft-klensin-smtp-code-registry-00 (a parallel development to
Tony's draft that I hope will be quickly consolidated with it)
and let us know what you think.

best,
     john

p.s. still hoping to see some text for RFC2821bis from you.

Date: Wed, 11 Apr 2007 07:11:35 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
To: [email-masked]
Sender: [email-masked]

John C Klensin wrote:

 > All that I have been objecting to is trying to use 1yz replies
 > in a continuation context to fake up that "keep alive" behavior.
 > And I object to it largely because of a sense that too many
 > clients will impose logic that will ignore those intermediate
 > responses and time out anyway.  Certainly there is nothing in
 > 821 or any of its successors that suggests restarting a timer
 > that is waiting on a reply code with each line of the reply that
 > is received.  Of course, some systems might do that, but it is
 > nothing on which a server should depend.... again, absent an
 > extension that is used to get specific agreement from the client
 > about how the intended "keep alive" is to be expressed and
 > interpreted.

 From the onset, I tried to approach this from a technical clarification 
standpoint with no mentioning of "keep alive":

    Only the final or last reply-code is important in a multiline
    response. The client MUST|SHOULD use the final line as the ultimate
    server response.  It is possible for a server to use a
    preliminary reply code 150- as part of multiline response
    with a final non-continuation completion code.

But in practice, I couldn't find any other reason for the last sentence 
to exist other than to use an example that illustrated a "keep alive" or 
"please wait" concept, and to tie it with other official IETF WG 
efforts, OPES.  So I showed this too:

    For example:

    354 Please Start sending....
        [ client uploads message ]
    150-Please Wait...
    150-Please Wait...
    451 Sorry, Rejected, try again later

But I always thought it wasn't necessary, and simply having a 
clarification on three key questions would be sufficient:

   - Must all the reply codes be the same?
   - If not, the final return code (last line) is valid code?
   - 150 SHOULD/MUST NOT be used without extended clients?

I think once this is clear, then you lay the ground work for such ideas 
as the official IETF synergistic efforts such as OPES, DKIM to have a 
enhanced chance of implementation where is it very realistically 
conceivable there might be "time out" issues.

I think this more about clarifying what already exist and not 
introducing new functionality. However, the benefits from the 
clarification is new functionality.

The specific "keep alive" functionality still needs to widely tested. 
But the proof of concept is well established.  It does work and there is 
no surprise from a programmability technical standpoint that it does. 
Most, if not all, timeout logics are based on "silence" or "idleness" so 
it was not difficult the extension of the FTP usage to SMTP as well.

So can a server depend on it? I say it is damn good engineering guess 
that will work. :-) and it did.

But am I suggesting that it should be WRITTEN as such?

No, not until you have the 3 questions above first clarified and go from 
there.

Like I said, you do have 3 type of clients:

  - One that uses the first reply-code and assumes it will be the
    same for all subsequent lines:

    example:

    250-winserver.com, Pleased to meet you.
    250-SIZE 5120000
    250-SUBMITTER
    250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
    250-AUTH=LOGIN
    250 HELP

  - One that uses the compares each code as a integrity checker:

    250-winserver.com, Pleased to meet you.
    garbage
    250 HELP

  - and one that simply looks for the last line and ignores
    all preceding lines.

By far, the last logic is what I find in the field, but I've seen the 
others in smaller smtp client scripts, logic, etc.  But we shouldn't be 
surprise because 4.2.1 does recommend that only the last line is important.

Is the statement current in 4.2.1 enough?  Maybe it is. Maybe that 
explains the proof of concept does work because most SMTP clients do 
follow the 4.2.1 last line suggestion.

So in that vain, maybe Hansen is right with his personal hat suggestion:

    "This can be translated as: don't do anything now that would
     prevent such an extension from being written."

But I do think some simple clarification may prevent such heartaches in 
the future and may also begin to prepare clients to naturally consider 
such things in their 2821bis improved software.

Maybe the only question for consideration is this sentence:

    The client SHOULD use the final line as the ultimate
    server response.

in combination or replacement of the last paragraph in 4.2.1:

    In many cases the SMTP client then simply needs to search for a line
    beginning with the reply code followed by <SP> or <CRLF> and ignore
    all preceding lines.  In a few cases, there is important data for the
    client in the reply "text".  The client will be able to identify
    these cases from the current context.

-- 
HLS

Date: Wed, 11 Apr 2007 12:08:38 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: "David F. Skoll" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, 10 Apr 2007, Hector Santos wrote:
>
>    Only the final or last reply-code is important in a multiline
>    response. The client MUST|SHOULD use the final line as the ultimate
>    server response.

Absolutely not. You CANNOT break running code like this.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
MALIN HEBRIDES: SOUTHWEST 4 OR 5, OCCASIONALLY 6, INCREASING 7 OR PERHAPS GALE
8 LATER IN HEBRIDES. MODERATE OR ROUGH. MAINLY FAIR. MODERATE OR GOOD.

Date: Wed, 11 Apr 2007 12:07:15 +0100
From: Tony Finch <[email-masked]>
To: "william(at)elan.net" <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, 10 Apr 2007, william(at)elan.net wrote:
>
> How would you see it working with extension ID?

Client sends KEEPALIVE <interval> at the start of the session (or again
later to change the interval). Server sends 1yz reply after interval if
normal reply is taking longer. Client responds to 1yz with CONTINUE or
RSET. This screws up pipelining so the server probably should not offer
pipelining and keepalive, or perhaps the keepalive command turns off
pipelining.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTH UTSIRE FORTIES CROMARTY: WEST BACKING SOUTH 3 OR 4, OCCASIONALLY 5.
MODERATE OR ROUGH, LOCALLY SLIGHT IN CROMARTY. MAINLY FAIR. MODERATE OR GOOD.

Date: Wed, 11 Apr 2007 11:53:54 +0100
From: Tony Finch <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue  15 syntax for multiline replies
Sender: [email-masked]

On Tue, 10 Apr 2007, Robert A. Rosenberg wrote:
>
> This is a question of semantics and definitions. Does "Unextended SMTP" mean a
> command that is capable of being issued due to occurring in the 220 reply to a
> EHLO connection or does it mean a session started with EHLO as opposed to
> HELO?

It means SMTP as described in RFC 2821, not including any service
extensions described in other RFCs.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
DOVER WIGHT PORTLAND PLYMOUTH: EAST OR NORTHEAST 3 OR 4, OCCASIONALLY 5.
SMOOTH OR SLIGHT, OCCASIONALLY MODERATE LATER. FAIR. MODERATE OR GOOD.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date:  Wed, 11 Apr 2007 09:00:27 +0200
Lines: 14
Sender: [email-masked]

Tony Finch wrote:

>> (i) Do nothing, leaving the text as is
> I really cannot work out how to make it clearer.

Apparently "unextended" in the 1yz paragraph isn't=20
clear enough for some readers.  It actually means
"an extension specified elsewhere, offered by the
 server and accepted by the client."

The "unextended" doesn't mean "using 1yz is fine as
soon as the client said EHLO instead of HELO".

Frank

Date: Tue, 10 Apr 2007 22:57:00 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions (was: Re: IPv6)
Sender: [email-masked]

At 17:24 31-03-2007, John C Klensin wrote:
> > How about a more radical approach, just forbid "IPv6 only"
> > for the moment somewhere in section 5 ?  "IPv6 only" is IMO
> > completely against the spirit of "if it can't receive mail
> > it has no business to send mail".

Frank, that's too radical.

>On the one hand, the "no substantive changes" rule, combined
>with my impression that we don't have a lot of operational
>experience with this, argues for just that solution.  On the
>other, having to prohibit IPv6 (or IPv6-without-IPv4) in 2007
>seems strange and very unfortunate.  RFC3974, which does derive
>from operational experience, doesn't shed a lot of light on
>this: a quick summary is that IPv4-only is ok, IPv6-only is ok,
>but, for mixtures or unknown destinations, any IPv6 host had
>better be running dual-stack with some IPv4 MX records.  That
>document contains a few statements that I believe to be serious
>errors (perhaps the reason it was published as an independent
>submission with strong disclaimers), but it is at least a
>thoughtful attempt to explore the problem.

Adding an IPv6 section to cover the above cases would mean a 
substantial change.  I suggest against getting into that.

>* Issue 11 was assigned to getting rid of the "A RR" references
>in favor of something more general.  Those changes have already
>been made in the working version of -02.

I didn't see the proposed change.  I suggest the following:

1. Replace "A RRs" in Section 2.3.5 and Section 5 with "address records".

2. Add the following note -

Note that where this document refers to "address records", it means A 
RR's or AAAA RR's.

Regards,
-sm 

Date: Tue, 10 Apr 2007 23:37:57 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

Parallel evolution. We both came up with the same idea, based on work in
different mailing lists. We're currently discussing merging.

	Tony Hansen
	[email-masked]

Frank Ellermann wrote:
> Tony Hansen wrote:
>  
>> Getting back to this draft.
> 
> I'm a bit confused by I-D.klensin-smtp-code-registry, this isn't an
> intentionally competing draft, or is it ?  

From: [email-masked]
Cc: [email-masked]
Date: Tue, 10 Apr 2007 17:24:42 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> > RFC 2434 is very clear about this. It says:
> >    IETF Consensus - New values are assigned through the IETF
> >    consensus process. Specifically, new assignments are made via
> >    RFCs approved by the IESG. Typically, the IESG will seek
> >    input on prospective assignments from appropriate persons
> >    (e.g., a relevant Working Group if one exists).
 
> >    Examples: SMTP extensions [SMTP-EXT], BGP Subsequent Address
> >    Family Identifiers [BGP4-EXT].

> "RFCs approved by the IESG".  My proposal "published RFC" doesn't
> exclude *_other_* RFCs.
 
OK, so your concern is that non-IETF RFCs be allowed to use the registry, Not
only do I not share your ocncern, I think allowing such usage is a really
bad idea.

Mind you, I'm not saying no protocol parameters will ever fit this set of
criteria. A parameter space that's plentiful, used by multiple protocols
including experimental ones, has no intrinsic security issues, where overlaps
or conflicts aren't an overwhelming concern, but for which it is important to
have each addigned value publicly documented, would be one where "published
RFC" criteria make sense.

Enhanced status codes, OTOH, are only used by a single protocol - SMTP - that's
incredibly widely deployed. Interoperability and security considerations are of
paramount importance here. It is therefore essential that any codes that are
defined be carefully and completely reviewed for conflicts and clarity and
there can easily be serious security considerations attached to their
definition and use. While I think the chances are excellent that the IESG
would, upon reviewing any independent submission that defined such codes
improperly, tell the RFC Editor that the document is in conflict with the
IETF's ongoing work on SMTP, that therefore it should not be published, and
that the RFC Editor would agree, I don't want to take even a small risk.

> > as Tony has already pointed out, "IETF Concensus" is the name
> > for the process where registrations are made by publishing an
> > RFC. The term "published RFC" is not one that RFC 2434 defines.

> ACK, it's defined in 2234bis.
 
2234 is the old ABNF spec so I assume you meant 2434bis. Even so, the term
"published RFC" is not one of the "well known IANA policy definitions" in
2434bis.

> >>> If 2434bis gets published first, this statement should be
> >>> revised accordingly.
 
> >> Yes, the 2234bis term is "IETF review", and it clearly requires
> >> an IETF RFC.
 
> > And which RFC 2434bis also states is the replacement name for
> > "IETF consensus":
 
> >     IETF Review - (Formerly called "IETF Consensus" in [IANA-
> >              CONSIDERATIONS])
> [...]

> And one paragraph above that you see the different definition for
> "RFC required" also allowing non-IETF-RFCs:

> | RFC Required - RFC publication (either as IETF Submission or as an
> |     RFC Editor submission [RFC3932]) suffices. Unless otherwise
> |     specified, any type of RFC is sufficient (e.g.,
> |     Informational, Experimental, Standards Track, BCP, etc.)

> The section with these (and more) definitions claims to enumerate
> "Well-Known IANA Policy Definitions", it's no new invention.  Maybe
> you intentionally want to exclude _other_ RFCs, and then we disagree
> about what we want, not about the terminology, and also not about
> the 2234 / 2234bis source of the terminology.

> Is there a compelling reason to exclude independent RFCs ?

Yes, I think there are fairly compelling reasons to exclude them. I also think
that erring on the side of caution is preferable to making a hash of the
enhanced status code space.

> > If Tony's draft moves forward and is approved before 2554bis is
> > published 2554bis will need to be revised to use the proper
> > registration templates. But this can easily be done with an RFC
> > Editor note.

> An RFC editor note in the approval of Tony's draft intended to edit
> another approved RFC already waiting for its number in the editor
> queue ?  That would beat the "RFC editor note" stunts I've seen so
> far (okay, only two).

Actually this sort of thing is quite common, especially when there are
publication delays. In fact there's example of an almost identical set of
changes taking place in the Sieve WG right now - various Sieve drafts that have
normative references to 3028bis have been approved and are waiting for 3028bis
to make it through the process. However, in the meantime 3028bis has been
revised to create an IANA registry for Sieve extensions, along with a specific
template to use for these registrations. All of the approved Sieve drafts -
five at last count - are expected to be revised to include the new template.

> >> The IESG has approved RFCs without Last Call and without IETF
> >> consensus, for examples see RFCs 4405, 4406, 4407, and 4408.
> >> (4408 doesn't invent a status code, but it has a 3464 reference)
 
> > None of which has anything to do with what's being proposed here.

> It has, in theory one of these RFCs could have tried to register a
> new reply code.  Under Tony's rules that might be not allowed.  The
> EAI drafts will be experimental, they might introduce new reply
> codes, and there might be no Last Call / IETF consensus about it.

EAI is a WG, with all that implies - it's documents are going to be IETF
documents, not independent submissions to the RFC Editor.

> > You're getting hung up on what the term "IETF Consensus" seems to
> > you to mean rather than what it is defined to mean.

> Apparently 2343bis agrees with my interpretation, it offers an "RFC
> required" different from "IETF review (formerly IETF consensus)".
 
> >> Please add 2554bis here.
> > A citation seems fine to me.

> Stating the source of a citation can't hurt.  I think Tony's I-D can
> just update 2554bis like it updates 4468 (depending on the outcome
> of the race condition with an "RFC editor note" strategy).

As I said, the correct course of action depends critically on whether or not
2554bis is published before this document makes it through the process.

				Ned

Date: Tue, 10 Apr 2007 21:17:12 -0400
From: John C Klensin <[email-masked]>
To: Douglas Otis <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Tuesday, 10 April, 2007 15:22 -0700 Douglas Otis
<[email-masked]> wrote:

> 
> On Apr 10, 2007, at 1:46 PM, John C Klensin wrote:
> 
>> And, for whatever it is worth, the text that starts "The
>> sender-  SMTP should send..." and ends "...so does not have
>> the continue or   abort commands." comes verbatim out of RFC
>> 821.  It has been with   us for a very long time and this is,
>> at least in my memory, the   first time someone has
>> identified it as pathologically confusing.    That doesn't
>> mean you are wrong of course, but it is interesting.
> 
> AV filtering and downstream acknowledgment are often essential
> to ensure against spoofed bounce exploits that have become all
> too common.   As a result, it is normal to enter a situation
> where the receiver wishes to issue a "keep-alive" or
> "please-wait" to the client.  DKIM does not changing this
> situation much if any.  Some ploys may be encumbered.  An
> extension is likely the safest alternative.

As soon as one adopts an extension model, we are in agreement.
Whatever I may feel about "keep alive" notions, rather than
quick replies as soon as the data transmission ends, I think an
extension that changes that model and is agreed to by both
server and client is a reasonable approach.

All that I have been objecting to is trying to use 1yz replies
in a continuation context to fake up that "keep alive" behavior.
And I object to it largely because of a sense that too many
clients will impose logic that will ignore those intermediate
responses and time out anyway.  Certainly there is nothing in
821 or any of its successors that suggests restarting a timer
that is waiting on a reply code with each line of the reply that
is received.  Of course, some systems might do that, but it is
nothing on which a server should depend.... again, absent an
extension that is used to get specific agreement from the client
about how the intended "keep alive" is to be expressed and
interpreted.

    john

Date: Tue, 10 Apr 2007 20:28:44 -0400
From: John C Klensin <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue           15 syntax for multiline replies
Sender: [email-masked]

--On Tuesday, 10 April, 2007 17:46 -0400 "Robert A. Rosenberg"
<[email-masked]> wrote:

> 
> At 13:23 +0100 on 04/10/2007, Tony Finch wrote about Re:
> rfc2821bis-01 Issue 14 Continuation of 222 greeting and:
> 
>>  >       Note: Unextended SMTP does not have any
>>>        commands that allow this type of reply, and so does
>>>        not have continue or abort commands.
>>> 
>>>  I "believe" the last note had the intention to NOTE that
>>>  150 would  be used as
>>  > a response that the client will REACT too.  The "Unextended
>>>  SMTP" implies they will be ignorant on how to react. i.e,
>>>  an ESMTP  spec might
>>>  define this.
>>> 
>>>  However, it is still in play for normal 2821 operations.
>> 
>> Absolutely not. The sentence you quoted means that 1xy
>> replies can only occur if an extension is specified to use
>> them. Without the server offering an explicit extension and
>> without the client explicitly activating that extension, they
>> cannot be used.
> 
> This is a question of semantics and definitions. Does
> "Unextended SMTP" mean a command that is capable of being
> issued due to occurring in the 220 reply to a EHLO connection
> or does it mean a session started with EHLO as opposed to HELO?
> 
> I think that we can agree that using HELO is the original SMTP
> Protocol. Using a EHLO EXTENDS this protocol so that you get a
> 220 list of EXTRA supported capabilities (most of which list
> commands that can be issued to extend the protocol but some of
> which by-themself provide the information needed for the
> extension [I think SIZE is such a 220 reply that tells the MUA
> the maximum acceptable message size]).
> 
> Thus the ability to accept EHLO can in-and-of-itself make an
> SMTP Server capable of being defined as using the/an
> "Unextended SMTP" Protocol (even if it does not return ANY
> command extensions in its 220 reply).

Except that, other than imposing a few new (really clarified),
requirements, an extended server (one that accepts an EHLO
command sent by the client) is required to behave _exactly_ like
an unextended (RFC 821-compliant) server except for (i) the
return of a list of supported extensions as the multiline reply
(with a 250 code, not 220) to the EHLO command and (ii) altering
that base (again, 821-compliant) behavior precisely as specified
by the specifications of the particular extensions that are
offered by the server and, where required, requested by the
client.  

I'm not positive that I understand what you are suggesting here,
but:

(1) The prohibition on the use of 1yz codes appears in RFC 821,
which is the definition for SMTP with no extensions at all
(including no EHLO command).

(2) There is nothing in RFC 1425 or its successors RFC 1651 or
RFC 
1869 that authorizes the use of 1yz codes in the presence of
EHLO without specific extensions that might authorize it.

(3) There is nothing in any of those documents, or in 2821, that
standardizes or permits the use of extensions or options listed
in the 220 Greeting to which clients are expected to pay
attention.

(4) The prohibition on the use of 1yz codes is copied exactly
into 2821 from 821.

(5) To the best of my knowledge, there is no standards-track
extension defined and registered that authorizes specifies, or
requires the use of 1yz codes.  Of course there may be private
extensions of which I'm not aware. 

I think that means they are prohibited in SMTP unless an
extension is defined that specifies their use, context, and
semantics.  And what I, at least, mean when I say "not available
in unextended SMTP" is that these codes should not be sent by
the server unless it has an agreement with the client --
expressed in a specific extension -- about how they are used,
what they mean, and how they are going to be interpreted.  The
only standard way to arrange and validate that agreement
involves a specific extension being offered by the server and, I
believe in this case, being requested by the client in some
specific way (extra command or parameter on an existing mail
command).

I don't know how to say it any more clearly than that and, if
the text in 2821bis needs clarification, I would welcome
suggested text.

Or did I misunderstand what you were suggesting?

    john

Date: Tue, 10 Apr 2007 19:21:55 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Robert A. Rosenberg wrote:

> Not relevant since the scenario we are talking about is not the MUA
> sending a 150 to the MTA but the MTA sending the 150 to the MUA (so as
> to restart the Time-Out clock and keep the session Open/Alive).

In general, an SMTP server cannot know whether the SMTP client is MS
Outlook or Sendmail.

The whole business of sending back part of a multiline reply to trick
the client into extending its timeout seems smelly to me.  In today's
environment when you might have hostile servers trying to tarpit
clients, I can see a client not wanting to extend its timeout just
because a network packets are trickling in.  The proper way to do
in-progress reply codes is to make an in-progress ESMTP extension that
the server advertises and the client asks for.

Regards,

David.

Date: Tue, 10 Apr 2007 17:14:18 -0400
To: Tony Hansen <[email-masked]>
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Cc: [email-masked]
Sender: [email-masked]

At 01:52 -0400 on 04/10/2007, Tony Hansen wrote about Re: Conflicting 
Enhanced Status Codes between RFC 4468 and :

>  X.7.12 A password transition is needed  This response to the AUTH
>          command indicates that the user needs to transition to the
>          selected authentication mechanism.  This is typically done by
>          authenticating once using the [PLAIN] authentication mechanism.
>          The selected mechanism SHOULD then work for authentications in
>          subsequent sessions.

You reference [PLAIN] but do not list it in Section 5.1 or 5.2. I'd 
assume that it should be there for completeness purposes.

Date: Tue, 10 Apr 2007 18:16:03 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

At 17:39 +0100 on 04/10/2007, Tony Finch wrote about Re: 
rfc2821bis-01 Issue 17: all contination lines must use :

>  > The OPES team looked long and hard for a way to break out of the timeout
>>  situation and the "150-" situation really was the best solution to get
>>  proposed, and it actually seemed to work with current existing servers.
>
>Sendmail and Exim will treat 1yz-... as an error.

Not relevant since the scenario we are talking about is not the MUA 
sending a 150 to the MTA but the MTA sending the 150 to the MUA (so 
as to restart the Time-Out clock and keep the session Open/Alive).

The only situation where the fact that Sendmail or Exim do not like 
150s is valid is when they are RELAYING a message to another SMTP 
Server that needs to do some extended housekeeping before accepting 
the message and this is sending interim 150s to say it is working on 
the request and the session is to be maintained.

Date: Tue, 10 Apr 2007 17:46:21 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue           15 syntax for multiline replies
Sender: [email-masked]

At 13:23 +0100 on 04/10/2007, Tony Finch wrote about Re: 
rfc2821bis-01 Issue 14 Continuation of 222 greeting and:

>  >       Note: Unextended SMTP does not have any
>>        commands that allow this type of reply, and so does not have
>>        continue or abort commands.
>>
>>  I "believe" the last note had the intention to NOTE that 150 would 
>>be used as
>  > a response that the client will REACT too.  The "Unextended
>>  SMTP" implies they will be ignorant on how to react. i.e, an ESMTP 
>>spec might
>>  define this.
>>
>>  However, it is still in play for normal 2821 operations.
>
>Absolutely not. The sentence you quoted means that 1xy replies can only
>occur if an extension is specified to use them. Without the server
>offering an explicit extension and without the client explicitly
>activating that extension, they cannot be used.

This is a question of semantics and definitions. Does "Unextended 
SMTP" mean a command that is capable of being issued due to occurring 
in the 220 reply to a EHLO connection or does it mean a session 
started with EHLO as opposed to HELO?

I think that we can agree that using HELO is the original SMTP 
Protocol. Using a EHLO EXTENDS this protocol so that you get a 220 
list of EXTRA supported capabilities (most of which list commands 
that can be issued to extend the protocol but some of which 
by-themself provide the information needed for the extension [I think 
SIZE is such a 220 reply that tells the MUA the maximum acceptable 
message size]).

Thus the ability to accept EHLO can in-and-of-itself make an SMTP 
Server capable of being defined as using the/an "Unextended SMTP" 
Protocol (even if it does not return ANY command extensions in its 
220 reply).

From: [email-masked]
Cc: [email-masked], [email-masked]
Date: Tue, 10 Apr 2007 15:14:34 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> >>5.7.8, 5.7.9, 5.7.11, 4,7.12, 5.7.12, and x.5.6.
> >>
> >>
> >All of these have are consistently defined in 2554bis and in Tony's draft. If
> >Tony's draft moves forward and is approved before 2554bis
> >
> This wouldn't be possible, as 2554bis is already approved ;-).

Read the rest of my sentence - I was referring to publication of 2554bis, not
approval. There have been plenty of cases where a draft has been approved and
then awaited publication for months or even years while numerous other related
drafts are written, approved, and even published. This is especially true with
security-related documents.

In the specific case of 2554bis I don't see anything that would delay
publication - in particular, all normative references are to published RFCs.
But even so, it is possible, albeit unlikely, that an IANA registry for
enhanced status codes could be approved before 2554bis is published, in which
as I noted below a revision with an RFC Editor note would be appropriate if
not required.

> >is published 2554bis
> >will need to be revised to use the proper registration templates. But this can
> >easily be done with an RFC Editor note.
> >

				Ned

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date: Tue, 10 Apr 2007 15:22:56 -0700
To: John C Klensin <[email-masked]>
Sender: [email-masked]

On Apr 10, 2007, at 1:46 PM, John C Klensin wrote:

> And, for whatever it is worth, the text that starts "The sender- 
> SMTP should send..." and ends "...so does not have the continue or  
> abort commands." comes verbatim out of RFC 821.  It has been with  
> us for a very long time and this is, at least in my memory, the  
> first time someone has identified it as pathologically confusing.   
> That doesn't mean you are wrong of course, but it is interesting.

AV filtering and downstream acknowledgment are often essential to  
ensure against spoofed bounce exploits that have become all too  
common.   As a result, it is normal to enter a situation where the  
receiver wishes to issue a "keep-alive" or "please-wait" to the  
client.  DKIM does not changing this situation much if any.  Some  
ploys may be encumbered.  An extension is likely the safest alternative.

-Doug

Date: Tue, 10 Apr 2007 15:46:03 -0700 (PDT)
From: "william(at)elan.net" <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, 10 Apr 2007, Tony Hansen wrote:

> I'm not suggesting that this use be blessed within 2821bis. However, I
> could easily see an extension I-D being written that defines and blesses
> its usage. So I wouldn't want to deny its possibility within 2821bis. I
> guess I'm falling into the (ii) camp here.

How would you see it working with extension ID? I mean the server
really does not know if client would accept this or not so the only
way to do is to have client issue some kind of extension keyword
during "RCPT" or "MAIL" or "DATA" but then question becomes which
one as none really fit all that well (or rather you need one issued
for each and everyone to really be correct). Is that really worth it?
I think its better that if it works with existing servers that you
specify it as part of bis.

-- 
William Leibzon
Elan Networks
[email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Date:  Tue, 10 Apr 2007 23:14:30 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 99
Sender: [email-masked]

[email-masked] wrote:
 
>>>> Published RFC is IMO simpler, they can link to a RFC in the
>>>> registry.
 
>>> That *IS* the IETF Consensus method; see the definition in RFC
>>> 2434.
 
>> IMO it's very different:  "Published RFC" allows informative and
>> experimental RFCs submitted independently (direct to RFC editor).
 
> That may be your opinion but the facts are otherwise.

Don't try to confuse me, it could be a waste oif time... :-)

> RFC 2434 is very clear about this. It says:
>    IETF Consensus - New values are assigned through the IETF
>    consensus process. Specifically, new assignments are made via
>    RFCs approved by the IESG. Typically, the IESG will seek
>    input on prospective assignments from appropriate persons
>    (e.g., a relevant Working Group if one exists).
 
>    Examples: SMTP extensions [SMTP-EXT], BGP Subsequent Address
>    Family Identifiers [BGP4-EXT].

"RFCs approved by the IESG".  My proposal "published RFC" doesn't
exclude *_other_* RFCs.  
 
> as Tony has already pointed out, "IETF Concensus" is the name
> for the process where registrations are made by publishing an
> RFC. The term "published RFC" is not one that RFC 2434 defines.

ACK, it's defined in 2234bis.
 
>>> If 2434bis gets published first, this statement should be
>>> revised accordingly.
 
>> Yes, the 2234bis term is "IETF review", and it clearly requires
>> an IETF RFC.
 
> And which RFC 2434bis also states is the replacement name for 
> "IETF consensus":
 
>     IETF Review - (Formerly called "IETF Consensus" in [IANA-
>              CONSIDERATIONS])
[...]

And one paragraph above that you see the different definition for
"RFC required" also allowing non-IETF-RFCs:

| RFC Required - RFC publication (either as IETF Submission or as an
|     RFC Editor submission [RFC3932]) suffices. Unless otherwise
|     specified, any type of RFC is sufficient (e.g.,
|     Informational, Experimental, Standards Track, BCP, etc.)

The section with these (and more) definitions claims to enumerate
"Well-Known IANA Policy Definitions", it's no new invention.  Maybe
you intentionally want to exclude _other_ RFCs, and then we disagree
about what we want, not about the terminology, and also not about
the 2234 / 2234bis source of the terminology.

Is there a compelling reason to exclude independent RFCs ?

> If Tony's draft moves forward and is approved before 2554bis is
> published 2554bis will need to be revised to use the proper
> registration templates. But this can easily be done with an RFC
> Editor note.

An RFC editor note in the approval of Tony's draft intended to edit
another approved RFC already waiting for its number in the editor
queue ?  That would beat the "RFC editor note" stunts I've seen so
far (okay, only two).

>> The IESG has approved RFCs without Last Call and without IETF
>> consensus, for examples see RFCs 4405, 4406, 4407, and 4408.
>> (4408 doesn't invent a status code, but it has a 3464 reference)
 
> None of which has anything to do with what's being proposed here.

It has, in theory one of these RFCs could have tried to register a
new reply code.  Under Tony's rules that might be not allowed.  The
EAI drafts will be experimental, they might introduce new reply
codes, and there might be no Last Call / IETF consensus about it.

> You're getting hung up on what the term "IETF Consensus" seems to
> you to mean rather than what it is defined to mean.

Apparently 2343bis agrees with my interpretation, it offers an "RFC
required" different from "IETF review (formerly IETF consensus)".
 
>> Please add 2554bis here.
> A citation seems fine to me.

Stating the source of a citation can't hurt.  I think Tony's I-D can
just update 2554bis like it updates 4468 (depending on the outcome
of the race condition with an "RFC editor note" strategy).

Frank

Date: Tue, 10 Apr 2007 17:00:56 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

<personal hat on>
Yes, I have to be careful about expressing both my personal opinions and
process opinions. Hopefully I've been succeeding at doing that.
Personally, I think it would be very useful and important to get an
extension I-D written up for this case. That's why I said previously "I
could easily see an extension I-D being written that defines and blesses
its usage. So I wouldn't want to deny its possibility within 2821bis."
This can be translated as: don't do anything now that would prevent such
an extension from being written.
</personal hat on>

<pseudo chair hat on>
However, doing it as part of 2821bis is improper. It really is
specifying a new response code and could cause a potential restart at
Proposed Standard. And *that* would be bad.
</pseudo chair hat on>

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> Oh sorry. I thought I was approaching it as a clarification and not
> really introducing any new functionality per se, i.e. simply fitting
> what already exist hence why in the field it seems to work.
> 
> You said it yourself with your former opes chair hat on:
> 
>    The OPES team looked long and hard for a way to break out
>    of the timeout situation and the "150-" situation really was
>    the best solution to get proposed, and it actually seemed
>    to work with current existing servers.
> 
> and added:
> 
>    I'm not suggesting that this use be blessed within 2821bis.
>    However, I could easily see an extension I-D being written
>    that defines and blesses its usage. So I wouldn't want to deny
>    its possibility within 2821bis. I guess I'm falling into the
>    (ii) camp here.
> 
> But if with your 2821bis pseudo-chair hat on now feels it still make it
> out of scope, then I throw up my hands.

Date: Tue, 10 Apr 2007 16:46:55 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Douglas Otis <[email-masked]>
cc: Tony Finch <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

--On Tuesday, 10 April, 2007 15:23 -0400 Hector Santos
<[email-masked]> wrote:

> First, all it says in so many words is that there no
> expectation for it because unextended clients do not now how
> use it. It specifically says:
> 
>      Note: unextended SMTP does not have any commands that
>      allow this type of reply, and so does not have continue
>      or abort commands.

That is clearly wrong.   By the time 2821 came around, it
probably should have been modified to say "...a continue command
to correspond to it" might have been thinking in terms of actual
CONTinue  and ABORt commands, the latter presumably with
different semantics than RSET (although, right now, I can't
imagine what they might be).   I will make that fix unless the
group agrees on some broader change (or concludes that the
"Note" should be dropped entirely).  Sorry for not understanding
what you were referring to earlier.
 
> It is clearly vague and incorrect because "unextended" clients
> do have the abort commands - RSET and/or QUIT.   If it is
> clear that this will break unextended clients, then it should
> be noted.

The text, at least as I read it, tells clients that they should
be prepared to see a 1yz reply and to then either send CONTinue
(which they can't send because, absent extensions, it doesn't
exist) or RSET.  If, in practice, it can only be followed by
RSET (or QUIT or anything else that implies RSET), a server that
sends it to a client without benefit of an extension is
generating a silly state that would better be avoided by sending
a 4yz or 5yz code.

One might well extend SMTP to use 1yz codes, but in a way
different from what you are trying to accomplish by sending
periodic "150-wait" lines.  That would involve the following
sort of dialogue:

  S:  150 I'm going to have to screw around for a while, are you
willing to stand by?
  Client then responds with one of..
  C:  RSET (i.e., give up completely)   or
  C:  ABOR (i.e., don't do whatever you intended to do, but 
continue processing, sending me an NDN later if needed)  or
  C:  CONT nnnnn   (where nnnn is a number of seconds that the
server may go on with processing before either sending out
another 150 code for confirmation or sending a final (for the
mail session)reply)

Of course, this would require an SMTP extension.  See below for
another version of this theme.

Note that FTP (see RFC765), on which SMTP was modeled, does have
a 1yz reply, but its semantics are exactly what you are trying
to accomplish with the 
   150-please wait
   150-please wait
   550 forget it
sequence.  In FTP-land, that would be expressed as 
   150 content received, please stand by
   550 nope
i.e., without a continuation arrangement and with the
expectation that the client should wait until another, final,
code is sent.  Probably, if this were carefully engineered, the
risks of duplicates that motivated RFC 1047 could be avoided:
the client would know that, if it got the 150 code, the server
had successfully received the content and that, if it didn't
stay connected long enough to get a real reply and then send a
QUIT, it might well trigger some DSN behavior.  But that has
never been done with SMTP and, of course, FTP has an
asynchronous data channel.  FTP also supports ABORt, RESTart,
and STATus commands that can take advantage of the asynchrony.

And, for whatever it is worth, the text that starts "The
sender-SMTP should send..." and ends "...so does not have the
continue or
abort commands." comes verbatim out of RFC 821.  It has been
with us for a very long time and this is, at least in my memory,
the first time someone has identified it as pathologically
confusing.  That doesn't mean you are wrong of course, but it is
interesting.

> Second, there is no restriction in using it as part of a
> continuation line.  Example:
> 
>      C: DATA
>      S: 352 Begin sending your data
>         [client uploads data]
>      S: 150-DKIM signature found!
>      S: 150-Please wait while we process your DKIM message!
>      S: 150-Waiting for trust server to respond...
>      S: 150-Still waiting for trust server to respond...
>      C: 550-Sorry, The DKIM POLICY has failed this transaction
>      C: 550 Please see http://trust-r-us.com/msgid=123213123
> 
> Who knows?  Is this not possible?  It is out of the whelm of
> real possibility?

No, it is is clearly not impossible.  However, it raises two
issues in addition to the one about the codes all being the same
which, IMO (and in the text of 2821), arises from one of them.  

One is that it is a perfectly conforming and rational
implementation of SMTP as now defined for the client to do
nothing but buffer a continued set of replies until the last one
arrives.  That would result in the "please wait" part of those
150 reply lines having no client effect and, in particular, no
effect on any timeouts the client imposes on a continued set of
reply codes.

Second, if the client does time out, one runs directly afoul of
the message-duplication principles of RFC 1047 (affirmed by 1123
and 2821).  

If one were doing an extension to support this, it would be,
IMO, sensible to move more toward the FTP model or the somewhat
related model discussed above, i.e., one in which the server
sent (a non-continuing) 1yz reply with the semantics of "please
stand by while I do some other processing" followed by a real
reply (which might be 2yz, 4yz, or 5yz).   I might even use the
1yz reply to give the client a hint about how long I expected my
server-side processing to take, i.e., how long it should be
expected to wait.

> Maybe someone will invent a OPES-DKIM shim that needs to call
> up some trust repository that is taking its sweet time
> returning with a response or whatever the circumstances, I
> think 2821bis should be clear either way. Either it is going
> to stop such possibilities or it will allow for become a
> helper in future technology.

And I think that what people are saying to you, from several
different perspectives, is that such an approach might be
perfectly reasonable, but should be provided through an explicit
SMTP extension that provides a specific 1yz code, specifies its
semantics and intended client and server behavior, and
explicitly modifies the normal SMTP timeout behavior.
Sequences such as the one you propose above are simply not going
to work (in terms of client behavior) consistently enough in
practice to be appropriate solutions.

> Please, lets not get into a discussion of where a DKIM or any
> other DATA processing should be postpone to a POST-ACCEPT
> design.  Thats a not starter for our system and hopefully
> others in this new era of minimizing accept/bounce potential
> issues.

For other reasons, I essentially substituted "some random
protocol that might need to process or validate the content" for
"DKIM" in your examples and discussion above.  Certainly one can
imagine several families of such protocols.  The real question
is whether, with SMTP as now defined (including the application
of the mandate that formally originated with RFC 1047 to
minimize time between EOD (CRLF.CRLF) and a reply indicating
message receipt, one can define something in which extended and
potentially time-consuming processing at that point makes sense.

     john

Date: Tue, 10 Apr 2007 14:45:50 -0700 (PDT)
From: "william(at)elan.net" <[email-masked]>
To: "David F. Skoll" <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

On Tue, 10 Apr 2007, David F. Skoll wrote:

> Hector Santos wrote:
>
>>     C: DATA
>>     S: 352 Begin sending your data
>>        [client uploads data]
>>     S: 150-DKIM signature found!
>>     S: 150-Please wait while we process your DKIM message!
>>     S: 150-Waiting for trust server to respond...
>>     S: 150-Still waiting for trust server to respond...
>>     C: 550-Sorry, The DKIM POLICY has failed this transaction
>>     C: 550 Please see http://trust-r-us.com/msgid=123213123
>
> This is what the RFC says about timeouts:
>
>   An SMTP client MUST provide a timeout mechanism.  It MUST use per-
>   command timeouts rather than somehow trying to time the entire mail
>   transaction.  Timeouts SHOULD be easily reconfigurable, preferably
>   without recompiling the SMTP code.  To implement this, a timer is set
>   for each SMTP command and for each buffer of the data transfer.  The
>   latter means that the overall timeout is inherently proportional to
>   the size of the message.
>
> It says the client MUST user per-command timeouts, so I don't see how
> a server can expect to increase a client's timeout by providing parts
> of a multiline response.  My understanding is that the timeout starts
> when the command has been transmitted, and stops when the timer expires
> or the complete reply code has been received.  The only exception (as
> noted in the RFC) is the Data Block timeout.
>
> I really don't think we want to bless this (mis-)use of 150 in a revision
> to RFC 2821, even if it is only in an example in a MAY section.

As with any may one of the issues would be inability to tell if client
would or would not support this. SMTP protocol is not very good at
negotiation of capabilities ("good" is in the eye of ... what I mean
is that some advanced functionalities that other protocols have are
not there) and it also does not properly support bi-directional 
communication, i.e. it would have been nice if in above case client
could tell "I'm tired of waiting for your full reply, in 2 seconds
I'll close connection and will retry this message later".

One other thing to note about above, the SMTP text states that
  DATA Termination: 10 minutes.
       This is while awaiting the "250 OK" reply.

So personally I think if server needs to wait more then 10 minutes
to give reply to DATA, it should just go ahead and give 45x reply
(but feel free to put 100 codes as in your example to say what
server is doing).

BTW, I always thought it would be neat (for greylisting for example) to
have capability for SMTP server to tell client to retry and specify how
long after current time it should/can retry. For above case it
would mean after 10 minutes it could give shorter reply and cache
the data about previous message so next time server attempts to deliver
this message it would know how to answer.

-- 
William Leibzon
Elan Networks
[email-masked]

Date: Tue, 10 Apr 2007 21:40:30 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

[email-masked] wrote:

>>5.7.8, 5.7.9, 5.7.11, 4,7.12, 5.7.12, and x.5.6.
>>    
>>
>All of these have are consistently defined in 2554bis and in Tony's draft. If
>Tony's draft moves forward and is approved before 2554bis
>
This wouldn't be possible, as 2554bis is already approved ;-).

>is published 2554bis
>will need to be revised to use the proper registration templates. But this can
>easily be done with an RFC Editor note.  
>

Date: Tue, 10 Apr 2007 16:13:51 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Oh sorry. I thought I was approaching it as a clarification and not 
really introducing any new functionality per se, i.e. simply fitting 
what already exist hence why in the field it seems to work.

You said it yourself with your former opes chair hat on:

    The OPES team looked long and hard for a way to break out
    of the timeout situation and the "150-" situation really was
    the best solution to get proposed, and it actually seemed
    to work with current existing servers.

and added:

    I'm not suggesting that this use be blessed within 2821bis.
    However, I could easily see an extension I-D being written
    that defines and blesses its usage. So I wouldn't want to deny
    its possibility within 2821bis. I guess I'm falling into the
    (ii) camp here.

But if with your 2821bis pseudo-chair hat on now feels it still make it 
out of scope, then I throw up my hands.

--
HLS

Tony Hansen wrote:
> <pseudo-chair hat on>
> 
> Adding language about a new 150 reply code in a multiline response would
> be introducing new functionality to the spec and would be out of scope.
> 
> </pseudo-chair hat on>
> 
> 	Tony Hansen
> 	[email-masked]
> 
> Hector Santos wrote:
>> I would like to suggest one paragraph wording after the final paragraph
>> in 4.2.1:
>>
>>    Only the final or last reply-code is important in a multiline
>>    response. The client MUST use the final line as the ultimate
>>    server response.  It is possible for a server to use a
>>    preliminary reply code 150- as part of multiline response
>>    with a final non-continuation completion code.
> 
> 
> 

Date: Tue, 10 Apr 2007 15:59:05 -0400
From: "David F. Skoll" <[email-masked]>
To: Hector Santos <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Hector Santos wrote:

>     C: DATA
>     S: 352 Begin sending your data
>        [client uploads data]
>     S: 150-DKIM signature found!
>     S: 150-Please wait while we process your DKIM message!
>     S: 150-Waiting for trust server to respond...
>     S: 150-Still waiting for trust server to respond...
>     C: 550-Sorry, The DKIM POLICY has failed this transaction
>     C: 550 Please see http://trust-r-us.com/msgid=123213123

This is what the RFC says about timeouts:

   An SMTP client MUST provide a timeout mechanism.  It MUST use per-
   command timeouts rather than somehow trying to time the entire mail
   transaction.  Timeouts SHOULD be easily reconfigurable, preferably
   without recompiling the SMTP code.  To implement this, a timer is set
   for each SMTP command and for each buffer of the data transfer.  The
   latter means that the overall timeout is inherently proportional to
   the size of the message.

It says the client MUST user per-command timeouts, so I don't see how
a server can expect to increase a client's timeout by providing parts
of a multiline response.  My understanding is that the timeout starts
when the command has been transmitted, and stops when the timer expires
or the complete reply code has been received.  The only exception (as
noted in the RFC) is the Data Block timeout.

I really don't think we want to bless this (mis-)use of 150 in a revision
to RFC 2821, even if it is only in an example in a MAY section.

Regards,

David.

Date: Tue, 10 Apr 2007 15:35:46 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

<pseudo-chair hat on>

Adding language about a new 150 reply code in a multiline response would
be introducing new functionality to the spec and would be out of scope.

</pseudo-chair hat on>

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> 
> I would like to suggest one paragraph wording after the final paragraph
> in 4.2.1:
> 
>    Only the final or last reply-code is important in a multiline
>    response. The client MUST use the final line as the ultimate
>    server response.  It is possible for a server to use a
>    preliminary reply code 150- as part of multiline response
>    with a final non-continuation completion code.

Date: Tue, 10 Apr 2007 15:23:29 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Douglas Otis <[email-masked]>
CC: Tony Finch <[email-masked]>, John C Klensin <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

Douglas Otis wrote:
> 
> 
> On Apr 10, 2007, at 8:41 AM, Tony Finch wrote:
> 
>> On Tue, 10 Apr 2007, John C Klensin wrote:
>>>
>>>     (i) Do nothing, leaving the text as is
>>
>> I really cannot work out how to make it clearer.
> 
> Agreed.

There is no restriction that it can't be used or how it can be used.

First, all it says in so many words is that there no expectation for it 
because unextended clients do not now how use it. It specifically says:

     Note: unextended SMTP does not have any commands that
     allow this type of reply, and so does not have continue
     or abort commands.

It is clearly vague and incorrect because "unextended" clients do have 
the abort commands - RSET and/or QUIT.   If it is clear that this will 
break unextended clients, then it should be noted.

Second, there is no restriction in using it as part of a continuation 
line.  Example:

     C: DATA
     S: 352 Begin sending your data
        [client uploads data]
     S: 150-DKIM signature found!
     S: 150-Please wait while we process your DKIM message!
     S: 150-Waiting for trust server to respond...
     S: 150-Still waiting for trust server to respond...
     C: 550-Sorry, The DKIM POLICY has failed this transaction
     C: 550 Please see http://trust-r-us.com/msgid=123213123

Who knows?  Is this not possible?  It is out of the whelm of real 
possibility?

Maybe someone will invent a OPES-DKIM shim that needs to call up some 
trust repository that is taking its sweet time returning with a response 
or whatever the circumstances, I think 2821bis should be clear either 
way. Either it is going to stop such possibilities or it will allow for 
become a helper in future technology.

Please, lets not get into a discussion of where a DKIM or any other DATA 
processing should be postpone to a POST-ACCEPT design.  Thats a not 
starter for our system and hopefully others in this new era of 
minimizing accept/bounce potential issues.

--
HLS

Date: Tue, 10 Apr 2007 15:16:42 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Hector Santos wrote:

>    Only the final or last reply-code is important in a multiline
>    response. The client MUST|SHOULD use the final line as the ultimate
>    server response.  It is possible for a server to use a
>    preliminary reply code 150- as part of multiline response
>    with a final non-continuation completion code.

I could live with that.  I can't live with someone doing something like:

    2xx-Go Ahead
    5xx Go Away

So maybe we need to say:

     Each line of a multiline response MUST start with the same reply
     code, except that initial lines MAY start with a 150- reply code
     to indicate that processing is in progress.  The client MUST
     use the code in the final line as the ultimate server response.

I really think it's important to limit the scope of different reply codes
in a multiline reply to avoid abuse.

>    For example:

>    354 Please Start sending....
>        [ client uploads message ]
>    150-Please Wait...
>    150-Please Wait...
>    451 Sorry, Rejected, try again later

I would add:

     Preliminary lines of a multiline response code MAY have an effect on
     a client's timeout, but a server SHOULD NOT assume they do unless it
     has a-priori knowledge of how the client handles its timeouts.

Regards,

David.

Date: Tue, 10 Apr 2007 14:50:57 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>>     Under most scenarios, servers SHOULD use the same reply
>>     code for each multiline response line when it is going to
>>     be displayed at the same time. However there are  scenarios
>>     (Keep Alive Client Timeout issues) where servers MAY use
>>     "150-" as an alternative reply code in a continuation line
>>     when the server needs to issue a heartbeat to keep the client
>>     alive during unsual delayed backend processing:
> 
> I don't like that.  There's no reason to assume that parts of a multiline
> response will have any effect on the client timeout.  It might not reset
> its timer just because it gets part of a reply back.  Maybe many current
> implementations do, but that's just good luck.
> 
> I believe that a proper negotiated ESMTP extension for "keepalive"
> intermediate replies should be developed.  We shouldn't try to hack
> something in.

Hi David,

I agree 100% and this issue began a few years back including during the 
last previous call out for 2821bis discussions where I brought the "Keep 
Alive" consideration.  So John is very aware this isn't a first time 
discussion.  I might just go ahead and write a I-D finally.  Back then I 
had never written an I-D. But with a few under my belt now, I am more 
familiar with the process and more comfortable writing one if need be.

However, for 2821bis, I just would like make sure that it doesn't 
pre-empt such new efforts.   Thats really all I would like to seek here 
and thus I go back to my original post to Frank with a suggested text, 
which was really just a starting point and would be very happy for a 
documentation person to reword it appropriately:

    Only the final or last reply-code is important in a multiline
    response. The client MUST|SHOULD use the final line as the ultimate
    server response.  It is possible for a server to use a
    preliminary reply code 150- as part of multiline response
    with a final non-continuation completion code.

    For example:

    354 Please Start sending....
        [ client uploads message ]
    150-Please Wait...
    150-Please Wait...
    451 Sorry, Rejected, try again later

I think the 1st and 2nd sentences are just a clarification of what is 
already implied in 4.2.1 (as you pointed out) in:

    In many cases the SMTP client then simply needs to search for a line
    beginning with the reply code followed by <SP> or <CRLF> and ignore
    all preceding lines.

If this is sufficient, I can agree with that.  But I do think it should 
be highlighted.

--
HLS

From: [email-masked]
Cc: [email-masked]
Date: Tue, 10 Apr 2007 11:11:10 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> Tony Hansen wrote:
 
> > Getting back to this draft.

> I'm a bit confused by I-D.klensin-smtp-code-registry, this isn't an
> intentionally competing draft, or is it ?

> >>> IANA is directed to create the registry Mail Enhanced Status
> >>> Codes.  In the terms of [5], values of Enhanced Status Codes
> >>> must be registered with IANA under the IETF Consensus method.

> >> Published RFC is IMO simpler, they can link to a RFC in the
> >> registry.
 
> > That *IS* the IETF Consensus method; see the definition in RFC
> > 2434.

> IMO it's very different:  "Published RFC" allows informative and
> experimental RFCs submitted independently (direct to RFC editor).

That may be your opinion but the facts are otherwise. RFC 2434 is very clear
about this. It says:

   IETF Consensus - New values are assigned through the IETF
   consensus process. Specifically, new assignments are made via
   RFCs approved by the IESG. Typically, the IESG will seek
   input on prospective assignments from appropriate persons
   (e.g., a relevant Working Group if one exists).

   Examples: SMTP extensions [SMTP-EXT], BGP Subsequent Address
   Family Identifiers [BGP4-EXT].

So, as Tony has already pointed out, "IETF Concensus" is the name for the
process where registrations are made by publishing an RFC. The term "published
RFC" is not one that RFC 2434 defines.

The name may not be the best and may in fact be counterintuitive to some, but
it is what it is.

> IETF consensus is IMO something based on an IETF Last Call.  It's
> not necessarily published as RFC.  Generally it's an "IETF last
> called RFC".  If the IESG could decree "IETF consensus" without
> Last Call it would be another reason why 2234bis should replace
> 2234 a.s.a.p.

Actually, the IESG can declare consensus through means other than a last call,
but that's really beside the point.

> > If 2434bis gets published first, this statement should be
> > revised accordingly.

> Yes, the 2234bis term is "IETF review", and it clearly requires
> an IETF RFC.
 
And which RFC 2434bis also states is the replacement name for "IETF consensus":

    IETF Review - (Formerly called "IETF Consensus" in [IANA-
             CONSIDERATIONS]) New values are assigned only through RFCs
             that have been shepherded through the IESG as AD-Sponsored
             IETF (or WG) Documents [RFC3932,RFC3978]. The intention is
             that the document and proposed assignment will be reviewed
             by the IESG and appropriate IETF WGs (or experts, if
             suitable working groups no longer exist) to ensure that the
             proposed assignment will not negatively impact
             interoperability or otherwise extend IETF protocols in an
             inappropriate or damaging manner.

             To ensure adequate community review, such documents are
             shepherded through the IESG as AD-sponsored documents with
             an IETF Last Call.

             Examples: SMTP extensions [SMTP-EXT], BGP Subsequent
             Address Family Identifiers [BGP4-EXT].

The significant change is that this new text makes it clear the document has to
a sherparded draft that goes through last call. 

> > There is an issue of what to do with the values X.7.8 through
> > X.7.13.  These were all defined in a draft that has since
> > expired (draft-newman-auth-resp-00.txt), but have been used in
> > other documents such as draft-siemborski-rfc2554bis.

> 2554bis-09 was approved some days ago, I see 2.7.0, 4.7.0, 5.7.0,

These are all obviously previously defined.

> 5.7.8, 5.7.9, 5.7.11, 4,7.12, 5.7.12, and x.5.6.

All of these have are consistently defined in 2554bis and in Tony's draft. If
Tony's draft moves forward and is approved before 2554bis is published 2554bis
will need to be revised to use the proper registration templates. But this can
easily be done with an RFC Editor note.

> >| must be registered with IANA under the IETF Consensus method.
> >| (Specifically, new assignments are made via RFCs approved by
> >| the IESG.)

> The IESG has approved RFCs without Last Call and without IETF
> consensus, for examples see RFCs 4405, 4406, 4407, and 4408.
> (4408 doesn't invent a status code, but it has a 3464 reference)

None of which has anything to do with what's being proposed here. You're
getting hung up on what the term "IETF Consensus" seems to you to mean rather
than what it is defined to mean.

> Please add the sources to your enumeration (e.g. RFC 2554bis)...

> >| X.7.14 Trust relationship required  The submission server requires
> >|    a configured trust relationship with a third-party server in
> >|    order to access the message content.  This value replaces the
> >|    prior use of X.7.8 for this error condition.
> >| X.7.15 Authentication credentials invalid  Authentication failed
> >|    due to invalid or insufficient authentication credentials.
> >|    This value replaces the prioruse of X.7.8 for this error
> >|    condition.

> ...or in these cases the source [RFC XXXX], i.e. your draft.

> >| 5.1.  Normative References
> [...]

> Please add 2554bis here.

A citation seems fine to me.

				Ned

Cc: John C Klensin <[email-masked]>, [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Date: Tue, 10 Apr 2007 10:52:45 -0700
To: Tony Finch <[email-masked]>
Sender: [email-masked]

On Apr 10, 2007, at 8:41 AM, Tony Finch wrote:

> On Tue, 10 Apr 2007, John C Klensin wrote:
>>
>> 	(i) Do nothing, leaving the text as is
>
> I really cannot work out how to make it clearer.

Agreed.

-Doug 

Date: Tue, 10 Apr 2007 13:52:41 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Hector Santos wrote:

>     Under most scenarios, servers SHOULD use the same reply
>     code for each multiline response line when it is going to
>     be displayed at the same time. However there are  scenarios
>     (Keep Alive Client Timeout issues) where servers MAY use
>     "150-" as an alternative reply code in a continuation line
>     when the server needs to issue a heartbeat to keep the client
>     alive during unsual delayed backend processing:

I don't like that.  There's no reason to assume that parts of a multiline
response will have any effect on the client timeout.  It might not reset
its timer just because it gets part of a reply back.  Maybe many current
implementations do, but that's just good luck.

I believe that a proper negotiated ESMTP extension for "keepalive"
intermediate replies should be developed.  We shouldn't try to hack
something in.

Regards,

David.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Date:  Tue, 10 Apr 2007 19:43:04 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 68
Sender: [email-masked]

Tony Hansen wrote:
 
> Getting back to this draft.

I'm a bit confused by I-D.klensin-smtp-code-registry, this isn't an
intentionally competing draft, or is it ?  

>>> IANA is directed to create the registry Mail Enhanced Status
>>> Codes.  In the terms of [5], values of Enhanced Status Codes
>>> must be registered with IANA under the IETF Consensus method.

>> Published RFC is IMO simpler, they can link to a RFC in the 
>> registry.
 
> That *IS* the IETF Consensus method; see the definition in RFC
> 2434.

IMO it's very different:  "Published RFC" allows informative and
experimental RFCs submitted independently (direct to RFC editor).

IETF consensus is IMO something based on an IETF Last Call.  It's
not necessarily published as RFC.  Generally it's an "IETF last
called RFC".  If the IESG could decree "IETF consensus" without
Last Call it would be another reason why 2234bis should replace
2234 a.s.a.p.

> If 2434bis gets published first, this statement should be 
> revised accordingly.

Yes, the 2234bis term is "IETF review", and it clearly requires
an IETF RFC.
 
> There is an issue of what to do with the values X.7.8 through
> X.7.13.  These were all defined in a draft that has since 
> expired (draft-newman-auth-resp-00.txt), but have been used in
> other documents such as draft-siemborski-rfc2554bis.

2554bis-09 was approved some days ago, I see 2.7.0, 4.7.0, 5.7.0,
5.7.8, 5.7.9, 5.7.11, 4,7.12, 5.7.12, and x.5.6.

>| must be registered with IANA under the IETF Consensus method.
>| (Specifically, new assignments are made via RFCs approved by
>| the IESG.)

The IESG has approved RFCs without Last Call and without IETF
consensus, for examples see RFCs 4405, 4406, 4407, and 4408.
(4408 doesn't invent a status code, but it has a 3464 reference)

Please add the sources to your enumeration (e.g. RFC 2554bis)...

>| X.7.14 Trust relationship required  The submission server requires
>|    a configured trust relationship with a third-party server in
>|    order to access the message content.  This value replaces the
>|    prior use of X.7.8 for this error condition.
>| X.7.15 Authentication credentials invalid  Authentication failed
>|    due to invalid or insufficient authentication credentials.
>|    This value replaces the prioruse of X.7.8 for this error
>|    condition.

...or in these cases the source [RFC XXXX], i.e. your draft.

>| 5.1.  Normative References
[...]

Please add 2554bis here.

Frank

Date: Tue, 10 Apr 2007 13:30:28 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 10 Apr 2007, Tony Hansen wrote:
>> I think that the real reason it worked with most clients is that it was
>> *not* a 2yz/3yz/4yz/5yz code, putting them into the "punt" category, and
>> then they looked at the last line to determine the actual code to
>> believe.
> 
> Actually I think the clients that were tested ignore all codes except the
> last of a multiline reply so were oblivious to the use of 1yz. No
> deliberate decision to punt.
> 
>> The OPES team looked long and hard for a way to break out of the timeout
>> situation and the "150-" situation really was the best solution to get
>> proposed, and it actually seemed to work with current existing servers.
> 
> Sendmail and Exim will treat 1yz-... as an error.

I'm curious as to why.  Is it because it assumes each line is the same? 
  Or it specifically looks for 2xx, 3xx, 4xx, 5xx?

The specs does suggest to ignore all preceding lines:

    In many cases the SMTP client then simply needs to search for a line
    beginning with the reply code followed by <SP> or <CRLF> and ignore
    all preceding lines.

--
HLS

Date: Tue, 10 Apr 2007 13:24:28 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

John C Klensin wrote:
> I've assigned Issue 18 to the question of whether it is
> permitted to use 1yz codes without extensions and whether
> additional text is needed to clarify the answer (whatever it is).
> 
> It appears that there are four options:
> 
> 	(i) Do nothing, leaving the text as is
> 	
> 	(ii) Make it clear that 1yz codes are prohibited without
> 	extensions.
> 	
> 	(iii) Specify conditions under which a
> 	2821bis-conforming server, without extensions, may
> 	generate a 1yz code and what clients are expected to do
> 	when such codes are received. 
> 	
> 	(iv) Say nothing more about servers generating these
> 	codes, but offer some guidance to clients if they
> 	receive them (whether the server is otherwise conformant
> 	or not).
> 	
>          john

I vote (iii) but also (iv) is ok.

      Clients SHOULD rely on using the final reply-code in
      a multiline response as the servers intended reply-code
      to the current command.  Other other codes SHOULD be
      ignored as it may not match the final code.

I think that is all it should be added, really.

--
HLS

Date: Tue, 10 Apr 2007 13:17:45 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John,

Our field experience has shown is there are 3 types of clients in how it 
deals with multi-line mixed response codes:

   1) Uses the first line code and ASSUMES it will be the same for
      subsequent lines and the final line.

   2) Use the first and the last and compares the two as a quick
      "Integrity Checker" - again, with the assumption that all
      are the same

   3) Use the final line only.

By far, the majority have a logic #3.  However as Tony Hansen pointed 
out, it could be that they are ignorant of 150- hence why it works to
keep clients alive.

My vote is (ii).

This is my rough suggestion text to only allow 150- to be the alternate 
reply-code in continuation lines.

     Under most scenarios, servers SHOULD use the same reply
     code for each multiline response line when it is going to
     be displayed at the same time. However there are  scenarios
     (Keep Alive Client Timeout issues) where servers MAY use
     "150-" as an alternative reply code in a continuation line
     when the server needs to issue a heartbeat to keep the client
     alive during unsual delayed backend processing:

     Example:

        352 Start your data ...
            [data uploaded]
        150-Processing data... please wait..
        150-Processing data... please wait..
        250 Message accepted

--

John C Klensin wrote:
> I've assigned Issue 17 to the question of whether, if reply
> lines constitute a continuation set, all of the codes are
> expected to be the same and whether this needs clarification.
> 
> It appears that there are three options:
> 
> 	(i) Do nothing, leaving the text as is
> 	
> 	(ii) Make it clear that the codes may be different and
> 	that clients are expected to examine all of them.
> 	
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.
> 
> If we select (ii), we may need to make a statement about which
> of the codes is the "final" one with regard to client actions,
> especially if the first digits are different.
> 
> Note that this may (or may not) have an impact on various
> proposals for delayed responses to RCPT, so any new text that
> goes in may need explicit provisions for extensions changing the
> rules.
> 
>      john
> 
> 
> 

Date: Tue, 10 Apr 2007 17:39:48 +0100
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, 10 Apr 2007, Tony Hansen wrote:
>
> I think that the real reason it worked with most clients is that it was
> *not* a 2yz/3yz/4yz/5yz code, putting them into the "punt" category, and
> then they looked at the last line to determine the actual code to
> believe.

Actually I think the clients that were tested ignore all codes except the
last of a multiline reply so were oblivious to the use of 1yz. No
deliberate decision to punt.

> The OPES team looked long and hard for a way to break out of the timeout
> situation and the "150-" situation really was the best solution to get
> proposed, and it actually seemed to work with current existing servers.

Sendmail and Exim will treat 1yz-... as an error.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTIES CROMARTY FORTH: WEST OR SOUTHWEST 5 OR 6. SLIGHT OR MODERATE,
OCCASIONALLY ROUGH. MAINLY FAIR. MODERATE OR GOOD.

Date: Tue, 10 Apr 2007 12:12:41 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:
> I've assigned Issue 17 to the question of whether, if reply
> lines constitute a continuation set, all of the codes are
> expected to be the same and whether this needs clarification.
> 
> It appears that there are three options:
> 
> 	(i) Do nothing, leaving the text as is
> 	
> 	(ii) Make it clear that the codes may be different and
> 	that clients are expected to examine all of them.
> 	
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.
> 
> If we select (ii), we may need to make a statement about which
> of the codes is the "final" one with regard to client actions,
> especially if the first digits are different.
> 
> Note that this may (or may not) have an impact on various
> proposals for delayed responses to RCPT, so any new text that
> goes in may need explicit provisions for extensions changing the
> rules.

<former opes chair hat on>
The proposed use of 1xy in the example seen the other day

 150-Please Wait...
 150-Please Wait...
 451 Sorry, Rejected, try again later

is in the spirit of the use of 1xy, but also technically incorrect in
that the 1yz description says that the SMTP *client* must respond by
sending another command. But since it also had the continuation
character, the client was prevented from doing so until the command was
concluded. I think that the real reason it worked with most clients is
that it was *not* a 2yz/3yz/4yz/5yz code, putting them into the "punt"
category, and then they looked at the last line to determine the actual
code to believe.

The OPES team looked long and hard for a way to break out of the timeout
situation and the "150-" situation really was the best solution to get
proposed, and it actually seemed to work with current existing servers.

I'm not suggesting that this use be blessed within 2821bis. However, I
could easily see an extension I-D being written that defines and blesses
its usage. So I wouldn't want to deny its possibility within 2821bis. I
guess I'm falling into the (ii) camp here.

This actually brings up the question in my mind: how do different
servers today actually behave when faced with multiline responses that
have different values?

</former opes chair hat on>

	Tony Hansen
	[email-masked]

Date: Tue, 10 Apr 2007 16:55:24 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

Tony Finch wrote:

>On Tue, 10 Apr 2007, Alexey Melnikov wrote:
>  
>
>>Tony Finch wrote:
>>    
>>
>>>Exim doesn't do ESCs.
>>>      
>>>
>>Maybe I am mistaken, but I've googled for 4.7.8 and found some log files.
>>    
>>
>My Google-fu is weak, but I guess you found Exim acting as a client and
>reporting a response from a server.
>  
>
You are right, the SMTP server is NTMail:
<http://www.exim.org/mail-archives/exim-users/Week-of-Mon-20060814/msg00009.html>

Date: Tue, 10 Apr 2007 16:46:17 +0100
From: Tony Finch <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and         draft-siemborski-rfc2554bis
Sender: [email-masked]

On Tue, 10 Apr 2007, Alexey Melnikov wrote:
> Tony Finch wrote:
> >
> > Exim doesn't do ESCs.
>
> Maybe I am mistaken, but I've googled for 4.7.8 and found some log files.

My Google-fu is weak, but I guess you found Exim acting as a client and
reporting a response from a server.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOLE LUNDY FASTNET: NORTHEAST BECOMING VARIABLE 3 OR 4. SLIGHT. FOG PATCHES.
MODERATE OR GOOD, VERY POOR IN FOG.

Date: Tue, 10 Apr 2007 16:41:01 +0100
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

On Tue, 10 Apr 2007, John C Klensin wrote:
>
> 	(i) Do nothing, leaving the text as is

I really cannot work out how to make it clearer.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HEBRIDES BAILEY: MAINLY SOUTHWESTERLY 5 TO 7, OCCASIONALLY GALE 8 IN BAILEY.
ROUGH OR VERY ROUGH. RAIN OR DRIZZLE. MODERATE OR GOOD.

Date: Tue, 10 Apr 2007 16:37:38 +0100
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, 10 Apr 2007, John C Klensin wrote:
>
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

I suggest replacing:

   The format for multiline replies requires that every line, except the
   last, begin with the reply code, followed immediately by a hyphen,
   "-" (also known as minus), followed by text.  The last line will
   begin with the reply code, followed immediately by <SP>, optionally
   some text, and <CRLF>.

with (inserting "same" in appropriate places):

   The format for multiline replies requires that every line, except the
   last, begin with the same reply code, followed immediately by a hyphen,
   "-" (also known as minus), followed by text.  The last line will
   begin with the same reply code, followed immediately by <SP>, optionally
   some text, and <CRLF>.

The following text is really bad advice and causes us a lot of support
problems because clients discard parts of error messages and lie to
their users.

   In many cases the SMTP client then simply needs to search for a line
   beginning with the reply code followed by <SP> or <CRLF> and ignore
   all preceding lines.  In a few cases, there is important data for the
   client in the reply "text".  The client will be able to identify
   these cases from the current context.

I suggest replacing it with:

   In many cases the SMTP client then simply needs to search for a line
   beginning with the reply code followed by <SP> or <CRLF>, accumulating
   the text appearing on the preceding lines. The client MAY assume that
   the server has used the same reply code on each line, and therefore MAY
   act on whichever line's code is convenient.

I believe the second sentence describes what running code actually does.
Good implementations do what the first sentence describes.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: WESTERLY 3 OR 4, OCCASIONALLY 5. SLIGHT OR MODERATE.
FAIR. MODERATE OR GOOD.

Date: Tue, 10 Apr 2007 11:35:40 -0400
From: Jeff Macdonald <[email-masked]>
To: [email-masked]
Subject: Section 4.2.1 3yz - should send another command
Sender: [email-masked]

Hi,

This is probably a minor nit, but this section:

   3yz  Positive Intermediate reply 

The command has been accepted, but the requested action is being held
in abeyance, pending receipt of further information. The SMTP client
should send another command specifying this information. This reply is
used in command sequence groups (i.e., in DATA).

Says "the SMTP client should send another command". But for DATA, another
command is not sent, but actual data. I suppose it depends on how one
defines command, and if command has been defined to be anything a
client sends, then I suppose this is ok. If it isn't, a possible fix
would be:

The SMTP client should send another command specifying this information
or proceed by sending data that is relevant to the current SMTP state.

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Tue, 10 Apr 2007 16:32:35 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: [email-masked]
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

Tony Finch wrote:

>On Tue, 10 Apr 2007, Alexey Melnikov wrote:
>  
>
>>After talking to Chris, both Chris and I think that X.7.8 should be used by
>>draft-siemborski-rfc2554bis, as it was originally defined in
>>draft-newman-auth-resp-00.txt (in 1998) and at least Exim uses it.
>>    
>>
>Exim doesn't do ESCs.
>
Maybe I am mistaken, but I've googled for 4.7.8 and found some log files.

>Postfix and Sendmail seem to use only X.7.0 and
>X.7.1, based on a cursory grep.
>  
>

Date: Tue, 10 Apr 2007 16:25:35 +0100
From: Tony Finch <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked], Chris Newman <[email-masked]>
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and         draft-siemborski-rfc2554bis
Sender: [email-masked]

On Tue, 10 Apr 2007, Alexey Melnikov wrote:

> After talking to Chris, both Chris and I think that X.7.8 should be used by
> draft-siemborski-rfc2554bis, as it was originally defined in
> draft-newman-auth-resp-00.txt (in 1998) and at least Exim uses it.

Exim doesn't do ESCs. Postfix and Sendmail seem to use only X.7.0 and
X.7.1, based on a cursory grep.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL: SOUTHWEST 5 OR 6, OCCASIONALLY 7 LATER. MODERATE OR ROUGH, INCREASING
VERY ROUGH IN NORTH. DRIZZLE IN NORTH. MODERATE OR GOOD.

Date: Tue, 10 Apr 2007 11:24:18 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

On Tue, Apr 10, 2007 at 10:40:52AM -0400, John C Klensin wrote:
> 	(iii) Prohibit different codes and, optionally, suggest
> 	that it is ok for a client to select one of them and
> 	assume that all of the others are the same.

+1

While some may believe the current spec is clear enough, I've seen
individuals outside of this group that have been confused as to what is
valid and I've seen multi-line responses were different codes have been
used. So, the current spec isn't clear enough IMO. I'll try to dig up
some actual examples.

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Tue, 10 Apr 2007 11:07:54 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

John C Klensin wrote:

> I've assigned Issue 17 to the question of whether, if reply
> lines constitute a continuation set, all of the codes are
> expected to be the same and whether this needs clarification.

RFC 2821 currently says:

   In many cases the SMTP client then simply needs to search for a line
   beginning with the reply code followed by <SP> or <CRLF> and ignore
   all preceding lines.  In a few cases, there is important data for the
   client in the reply "text".  The client will be able to identify
   these cases from the current context.

(bottom of Sec. 4.2.1)

It's clear to me that clients are permitted only to use the last reply
code and ignore all others.  It's also clear to me that it doesn't
make sense to allow different reply codes in different parts of a
multiline reply, because a client is permitted to ignore all but the
last reply code.

Regards,

David.

From: [email-masked]
Cc: [email-masked], Chris Newman <[email-masked]>
Date: Tue, 10 Apr 2007 08:05:35 -0700 (PDT)
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
To: Tony Hansen <[email-masked]>
Sender: [email-masked]

> I expanded out the lists of what values are being pulled out of where.

The draft looks basically OK to me although there appear to be some
formatting issues with the short and long descriptions of the various
codes running into each other.

> There is an issue of what to do with the values X.7.8 through X.7.13.
> These were all defined in a draft that has since expired
> (draft-newman-auth-resp-00.txt), but have been used in other documents
> such as draft-siemborski-rfc2554bis.

> As a strawman, I pulled in versions of the definitions from that draft.

> I also introduced X.7.14 and X.7.15 to replace the competing uses of
> X.7.8. Is this the best way to handle this conflict?

It's not perfect, but I know of no better way to deal with it.

				Ned

Date: Tue, 10 Apr 2007 15:56:32 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked], Chris Newman <[email-masked]>
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

Tony Hansen wrote:

>I expanded out the lists of what values are being pulled out of where.
>
>There is an issue of what to do with the values X.7.8 through X.7.13.
>These were all defined in a draft that has since expired
>(draft-newman-auth-resp-00.txt), but have been used in other documents
>such as draft-siemborski-rfc2554bis.
>
>As a strawman, I pulled in versions of the definitions from that draft.
>
>I also introduced X.7.14 and X.7.15 to replace the competing uses of
>X.7.8. Is this the best way to handle this conflict?
>  
>
After talking to Chris, both Chris and I think that X.7.8 should be used 
by draft-siemborski-rfc2554bis, as it was originally defined in 
draft-newman-auth-resp-00.txt (in 1998) and at least Exim uses it. I 
don't think that people who've implemented SMTP AUTH are likely to 
switch to X.7.15.
rfc4468bis should be updated to use X.7.14.

Date: Tue, 10 Apr 2007 10:45:06 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-01 Issue 18: Usability of 1yz replies
Sender: [email-masked]

I've assigned Issue 18 to the question of whether it is
permitted to use 1yz codes without extensions and whether
additional text is needed to clarify the answer (whatever it is).

It appears that there are four options:

	(i) Do nothing, leaving the text as is
	
	(ii) Make it clear that 1yz codes are prohibited without
	extensions.
	
	(iii) Specify conditions under which a
	2821bis-conforming server, without extensions, may
	generate a 1yz code and what clients are expected to do
	when such codes are received. 
	
	(iv) Say nothing more about servers generating these
	codes, but offer some guidance to clients if they
	receive them (whether the server is otherwise conformant
	or not).
	
         john

Date: Tue, 10 Apr 2007 10:40:52 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-01 Issue 17: all contination lines must use same code
Sender: [email-masked]

I've assigned Issue 17 to the question of whether, if reply
lines constitute a continuation set, all of the codes are
expected to be the same and whether this needs clarification.

It appears that there are three options:

	(i) Do nothing, leaving the text as is
	
	(ii) Make it clear that the codes may be different and
	that clients are expected to examine all of them.
	
	(iii) Prohibit different codes and, optionally, suggest
	that it is ok for a client to select one of them and
	assume that all of the others are the same.

If we select (ii), we may need to make a statement about which
of the codes is the "final" one with regard to client actions,
especially if the first digits are different.

Note that this may (or may not) have an impact on various
proposals for delayed responses to RCPT, so any new text that
goes in may need explicit provisions for extensions changing the
rules.

     john

Date: Tue, 10 Apr 2007 10:18:50 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue          15 syntax for multiline replies
Sender: [email-masked]

--On Tuesday, 10 April, 2007 10:04 -0400 Hector Santos
<[email-masked]> wrote:

>...
>> You might be able to overcome some of these difficulties in an
>> extension, but it certainly does not belong in the base spec.
> 
> I disagree but I defer to your rules. Thats too bad because I
> can't help but feel if someone else would had brought this up,
> you would take a more open minded consideration. I am sorry to
> say that only because I find it extremely odd that technical
> clarity in this case is not taken serious when in reality the
> less serious nits are being debated else where. The odds are
> very good systems you will be confronted with this ISSUE more
> so than some of the other issues that essentially have little
> to no impact in improving the system.

I believe that, if you go back through the recent transactions
on this list, you will find that I have been quite resistant to
every proposed change to specify new capabilities and
restrictions that are not specified before.  If you have seen
more of that resistance than others, I believe you will discover
that is it only because you have proposed more changes that seem
to fall into that category than others.    While there may be
differences of opinion about whether the current language is
clear enough (and I remain willing to clarify text if there is
consensus that it is not), 2821 was intended to be clear that
(i) 1yz codes are outside the standard in the absence of
extensions and (ii) if continuation lines are used, all of the
codes are expected to be the same.  The latter is particularly
important because we know that there are implementations that
will act only on the first code to appear in a multiple-line
sequence and ones that will act only on the last such code, so a
change to say "they can be different and you should pay
attention only to the last one" would be, I believe, a
significant and substantive change to the specification.

  regards,
    john

Date: Tue, 10 Apr 2007 10:13:15 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 10 Apr 2007, Hector Santos wrote:
>> Sorry if I missed this, but where is this stated? Is it implied?
> 
> "The format for multiline replies requires that every line, except the
> last, begin with the reply code" - i.e. only one reply code.

That does not say that it must be the same reply code, especially when 
used with a PRELIMINARY response, not a COMPLETION response.

>> Even then, wouldn't be inconsistent with the 150 description?  and it
>> doesn't match the current MUA support?
> 
> None of the commands so far specified in SMTP or its extensions can
> produce a 150 reply.

And why not?  I see no restriction.

>>>>   - 150 SHOULD/MUST NOT be used without extended status codes?
>>> It must not be used at all.
>> Explain?
> 
> None of the commands so far specified in SMTP or its extensions can
> produce a 150 reply.

And why not?  I see no restriction.

>>       Note: Unextended SMTP does not have any
>>       commands that allow this type of reply, and so does not have
>>       continue or abort commands.
>>
>> However, it is still in play for normal 2821 operations.
> 
> Absolutely not. The sentence you quoted means that 1xy replies can only
> occur if an extension is specified to use them. 

That is not what it says and its very vague in regards to "does not have 
continue or abort" because in section "D.2.  Aborted SMTP Transaction 
Scenario" it specifically described ABORTED condition using the RSET 
command.

 > Without the server
> offering an explicit extension and without the client explicitly
> activating that extension, they cannot be used.

That is not what it says. If you want to SAY that then tell your buddy 
John to write it.  But it doesn't say that.

>> Nonetheless, there is no indication whatsoever whether the server
>> SHOULD/MUST/NOT use it.
> 
> None of the commands so far specified in SMTP or its extensions can
> produce a 150 reply.

And why not?  I see no restriction.

Look, yuns need to be consistent.  You are going to bring up other 
less-serious nits, this one is just as serious and I would venture to 
say, a tad more serious and it should be clarified.  If John is going to 
be so horrified if a server does behave using 150, then he should TELL 
us why in the 2821bis specs to avoid such horrible experiences.

--
HLS

Date: Tue, 10 Apr 2007 10:04:42 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

John C Klensin wrote:
> 
> Hector, in addition to the comments made by others 
> need to be the same, any use of 1yz requires extensions), I note
> that this sort of thing would (i) violate well-established
> recommendations about getting (final) responses back quickly to
> avoid duplicate messages and (ii) would probably result in
> clients disconnecting if the sequence were really

but that isn't the case.

> As a sometime SMTP implementer, I'd be pretty horrified if I was
> told wait... wait.... wait... try later (wait... wait... "5yz Go
> away and don't come back" would be a little more plausible).

As a FULL time SMTP developer, vendor and implementor, horrified or not, 
it is possible. The issue apply for FTP servers as well, where it might 
have a lengthy (time wise) response where SOCKETS may time out in 30 
seconds because of the lack of insight by the author that it MIGHT be 
even CONCEIVABLE that the backend may want to pre-process an upload 
response is confirmed.

In short, like it or not, HOW will your client react if a SERVER did use 
it?  You have nothing in the specs that prohibits it and fortunately, 
most MUA client I have tested and the few MTA clients I had tested all 
honor it. Thats not the say that it warrants or justified it, but if you 
  are suggesting that clients will break, then that is more the reason 
why you should make it very clear how it ought to be used.

> You might be able to overcome some of these difficulties in an
> extension, but it certainly does not belong in the base spec.

I disagree but I defer to your rules. Thats too bad because I can't help 
but feel if someone else would had brought this up, you would take a 
more open minded consideration. I am sorry to say that only because I 
find it extremely odd that technical clarity in this case is not taken 
serious when in reality the less serious nits are being debated else 
where. The odds are very good systems you will be confronted with this 
ISSUE more so than some of the other issues that essentially have little 
to no impact in improving the system.

--
HLS

Date: Tue, 10 Apr 2007 09:01:29 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue          15 syntax for multiline replies
Sender: [email-masked]

--On Tuesday, 10 April, 2007 05:24 -0400 Hector Santos
<[email-masked]> wrote:

> I would like to suggest one paragraph wording after the final
> paragraph in 4.2.1:
> 
>     Only the final or last reply-code is important in a
> multiline
>     response. The client MUST use the final line as the
> ultimate
>     server response.  It is possible for a server to use a
>     preliminary reply code 150- as part of multiline response
>     with a final non-continuation completion code.
> 
>     For example:
> 
>     354 Please Start sending....
>         [ client uploads message ]
>     150-Please Wait...
>     150-Please Wait...
>     451 Sorry, Rejected, try again later

Hector, in addition to the comments made by others (the codes do
need to be the same, any use of 1yz requires extensions), I note
that this sort of thing would (i) violate well-established
recommendations about getting (final) responses back quickly to
avoid duplicate messages and (ii) would probably result in
clients disconnecting if the sequence were really

     150-Please Wait...
(significant pause)
     150-Please Wait...
(significant pause)
     451 Sorry, Rejected, try again later

As a sometime SMTP implementer, I'd be pretty horrified if I was
told wait... wait.... wait... try later (wait... wait... "5yz Go
away and don't come back" would be a little more plausible).

You might be able to overcome some of these difficulties in an
extension, but it certainly does not belong in the base spec.

     john

Date: Tue, 10 Apr 2007 13:23:02 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue  15 syntax for multiline replies
Sender: [email-masked]

On Tue, 10 Apr 2007, Hector Santos wrote:
>
> Sorry if I missed this, but where is this stated? Is it implied?

"The format for multiline replies requires that every line, except the
last, begin with the reply code" - i.e. only one reply code.

> Even then, wouldn't be inconsistent with the 150 description?  and it
> doesn't match the current MUA support?

None of the commands so far specified in SMTP or its extensions can
produce a 150 reply.

Three MUAs is not a good sample of SMTP client implementations. Sendmail
and Exim only look at the first line of a reply.

> Well, same as above, shouldn't [differing reply codes] be defined?

Should the meaning of syntactically invalid garbage be defined?

> > >   - 150 SHOULD/MUST NOT be used without extended status codes?
> >
> > It must not be used at all.
>
> Explain?

None of the commands so far specified in SMTP or its extensions can
produce a 150 reply.

> I don't see how EXTENDED status codes are even related.

Why did you keep bringing them up?

>       Note: Unextended SMTP does not have any
>       commands that allow this type of reply, and so does not have
>       continue or abort commands.
>
> I "believe" the last note had the intention to NOTE that 150 would be used as
> a response that the client will REACT too.  The "Unextended
> SMTP" implies they will be ignorant on how to react. i.e, an ESMTP spec might
> define this.
>
> However, it is still in play for normal 2821 operations.

Absolutely not. The sentence you quoted means that 1xy replies can only
occur if an extension is specified to use them. Without the server
offering an explicit extension and without the client explicitly
activating that extension, they cannot be used.

> Nonetheless, there is no indication whatsoever whether the server
> SHOULD/MUST/NOT use it.

None of the commands so far specified in SMTP or its extensions can
produce a 150 reply.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTIES CROMARTY FORTH: WEST OR SOUTHWEST 5 OR 6. SLIGHT OR MODERATE,
OCCASIONALLY ROUGH. MAINLY FAIR. MODERATE OR GOOD.

Date: Tue, 10 Apr 2007 07:39:03 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 10 Apr 2007, Hector Santos wrote:
>>   - Must all the reply codes be the same?
> 
> Yes.

Sorry if I missed this, but where is this stated? Is it implied?

Even then, wouldn't be inconsistent with the 150 description?  and it 
doesn't match the current MUA support?

>>   - If not, the final return code (last line) is valid code?
> 
> Undefined.

Well, same as above, shouldn't it be defined?  Doesn't this fall into 
the documentation "nit" department yuns are trying to clear up?

>>   - 150 SHOULD/MUST NOT be used without extended status codes?
> 
> It must not be used at all. 

Explain?

> It's completely outside the spec,
> whether or not you support ehnanced status codes.

I don't see how EXTENDED status codes are even related. The current 
description is:

    1yx  Positive Preliminary reply The command has been accepted, but
       the requested action is being held in abeyance, pending
       confirmation of the information in this reply.  The SMTP client
       should send another command specifying whether to continue or
       abort the action.  Note: Unextended SMTP does not have any
       commands that allow this type of reply, and so does not have
       continue or abort commands.

I "believe" the last note had the intention to NOTE that 150 would be 
used as a response that the client will REACT too.  The "Unextended
SMTP" implies they will be ignorant on how to react. i.e, an ESMTP spec 
might define this.

However, it is still in play for normal 2821 operations.  Imagine. If 
the client begins the transaction with EHLO, it might be said that the 
server and client may be SAFE in using 150.   If the client used HELO, 
then the server might not use it.

Nonetheless, there is no indication whatsoever whether the server 
SHOULD/MUST/NOT use it.

I don't understand the resistance in clearing this up.

--
HLS

Date: Tue, 10 Apr 2007 07:25:34 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: 15 syntax for multiline replies
Sender: [email-masked]

Hector Santos wrote:
> Tony Finch wrote:

>> Absolutely not. This is way too new and experimental for this revision,
>> and utterly unsupported by running code.

Just a small tidbit:  The top three MUAs in the market (OUTLOOK, TBIRD 
and EUDORA) support it.

--

Date: Tue, 10 Apr 2007 12:06:00 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue  15 syntax for multiline replies
Sender: [email-masked]

On Tue, 10 Apr 2007, Hector Santos wrote:
>
>   - Must all the reply codes be the same?

Yes.

>   - If not, the final return code (last line) is valid code?

Undefined.

>   - 150 SHOULD/MUST NOT be used without extended status codes?

It must not be used at all. It's completely outside the spec,
whether or not you support ehnanced status codes.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
TYNE DOGGER: WEST OR SOUTHWEST 4 OR 5. SLIGHT OR MODERATE. FAIR. MAINLY GOOD.

Date: Tue, 10 Apr 2007 06:54:23 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 10 Apr 2007, Hector Santos wrote:
>> I would like to suggest one paragraph wording after the final paragraph in
>> 4.2.1:
>>
>>    Only the final or last reply-code is important in a multiline
>>    response. The client MUST use the final line as the ultimate
>>    server response.  It is possible for a server to use a
>>    preliminary reply code 150- as part of multiline response
>>    with a final non-continuation completion code.
> 
> Absolutely not. This is way too new and experimental for this revision,
> and utterly unsupported by running code.
> 
> If you want to define a 150 response, write a proper ESMTP service
> extension.

It really has nothing to do with experiments or too new in the field or 
whether there are a few running codes exploring this.

The questions are:

   - Must all the reply codes be the same?
   - If not, the final return code (last line) is valid code?
   - 150 SHOULD/MUST NOT be used without extended status codes?

MUST or SHOULD or MUST/SHOULD NOT (pick one) this revision answer these 
fundamental specification questions about the multi-line server responses?

Obviously, since you reject, I shouldn't be surprise if the cogs will too.

--
HLS

Date: Tue, 10 Apr 2007 10:57:18 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue  15 syntax for multiline replies
Sender: [email-masked]

On Tue, 10 Apr 2007, Hector Santos wrote:
>
> I would like to suggest one paragraph wording after the final paragraph in
> 4.2.1:
>
>    Only the final or last reply-code is important in a multiline
>    response. The client MUST use the final line as the ultimate
>    server response.  It is possible for a server to use a
>    preliminary reply code 150- as part of multiline response
>    with a final non-continuation completion code.

Absolutely not. This is way too new and experimental for this revision,
and utterly unsupported by running code.

If you want to define a 150 response, write a proper ESMTP service
extension.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
DOVER WIGHT PORTLAND PLYMOUTH: VARIABLE 2 OR 3, OCCASIONALLY 4. MAINLY SLIGHT.
FOG PATCHES. MODERATE OR GOOD, OCCASIONALLY VERY POOR.

Date: Tue, 10 Apr 2007 05:24:45 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Frank Ellermann wrote:
> Hector Santos wrote:
> 
>> 1) Must all the codes in the multiline response be the same?
> 
> We discussed this in summer 2005 in conjunction with the first
> draft, but I don't recall what the outcome at this time was.
> 
>> 2) Which is the final return code, the first or the last?
> 
> I don't think that different codes are permitted.  If they're
> nevertheles different the last paragraph in section 4.2.1 of
> the I-D suggests that the last code is the final code, compare
> http://tools.ietf.org/html/draft-klensin-rfc2821bis-01#section-4.2.1

I'm not quite seeing this suggestion.

> Your "150-wait... 150-wait... 4yz sorry" example is clear, but
> you can't use 1yz codes without an extension (confirmed by the
> client, I think), see the note at the end of the "1yz" part in
> section 4.2.1.

hmmmmm, that is vague.  What is the client abort command?  RSET?
QUIT?  required commands?

So we have three outstanding questions? Or are these implied and no need 
to address?

   - Must all the reply codes be the same?
   - If not, the final return code (last line) is valid code?
   - 150 SHOULD/MUST NOT be used without extended status codes?

I would like to suggest one paragraph wording after the final paragraph 
in 4.2.1:

    Only the final or last reply-code is important in a multiline
    response. The client MUST use the final line as the ultimate
    server response.  It is possible for a server to use a
    preliminary reply code 150- as part of multiline response
    with a final non-continuation completion code.

    For example:

    354 Please Start sending....
        [ client uploads message ]
    150-Please Wait...
    150-Please Wait...
    451 Sorry, Rejected, try again later

--
HLS

Date: Tue, 10 Apr 2007 01:52:05 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
CC: Chris Newman <[email-masked]>
Subject: Re: Conflicting Enhanced Status Codes between RFC 4468 and draft-siemborski-rfc2554bis
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------030001090900080100010909
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Getting back to this draft.

Frank Ellermann wrote:
> Tony Hansen wrote:
>  
>> I had a few minutes. See attached. Comments?
> 
>>    o  IANA is directed to create the registry Mail Enhanced Status
>>       Codes.  In the terms of [5], values of Enhanced Status Codes
>>       must be registered with IANA under the IETF Consensus method.
> 
> Published RFC is IMO simpler, they can link to a RFC in the registry.

That *IS* the IETF Consensus method; see the definition in RFC 2434.

If 2434bis gets published first, this statement should be revised
accordingly.

===

I expanded out the lists of what values are being pulled out of where.

There is an issue of what to do with the values X.7.8 through X.7.13.
These were all defined in a draft that has since expired
(draft-newman-auth-resp-00.txt), but have been used in other documents
such as draft-siemborski-rfc2554bis.

As a strawman, I pulled in versions of the definitions from that draft.

I also introduced X.7.14 and X.7.15 to replace the competing uses of
X.7.8. Is this the best way to handle this conflict?

	Tony Hansen
	[email-masked]

--------------030001090900080100010909
Content-Type: text/plain;
 name="draft-hansen-4468upd-mailesc-registry-01.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline;
 filename="draft-hansen-4468upd-mailesc-registry-01.txt"

Network Working Group                                          T. Hansen
Internet-Draft                                         AT&T Laboratories
Updates: 1893,4468                                         April 9, 2007
(if approved)
Intended status: Standards Track
Expires: October 11, 2007

               A Registry for Mail Enhanced Status Codes
                draft-hansen-4468upd-mailesc-registry-01

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on October 11, 2007.

Copyright Notice

   Copyright (C) The IETF Trust (2007).

Abstract

   This document establishes an IANA registry for SMTP Enhanced Status
   Codes.

Hansen                  Expires October 11, 2007                [Page 1]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
   2.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 3
   3.  Security Considerations . . . . . . . . . . . . . . . . . . . . 5
   4.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 5
   5.  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 5
     5.1.  Normative References  . . . . . . . . . . . . . . . . . . . 5
     5.2.  Informative References  . . . . . . . . . . . . . . . . . . 5
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 6
   Intellectual Property and Copyright Statements  . . . . . . . . . . 7

Hansen                  Expires October 11, 2007                [Page 2]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

1.  Introduction

   Enhanced Status Codes for SMTP were first defined in [RFC1893], which
   was replaced by [RFC3463].  Since that time, various RFCs have been
   published and internet drafts proposed that define further status
   codes.  However, no IANA registry was defined for the status codes
   and conflicts in definitions have begun to appear.  This RFC defines
   such an IANA registry and was written to help prevent further
   conflicts from appearing in the future.

   This document is being discussed on the SMTP mailing list,
   [email-masked] [1].

2.  IANA Considerations

   o  IANA is directed to create the registry Mail Enhanced Status
      Codes.  In the terms of [RFC2434], values of Enhanced Status Codes
      must be registered with IANA under the IETF Consensus method.
      (Specifically, new assignments are made via RFCs approved by the
      IESG.)
   o  The Mail Enhanced Status Codes registry will have three tables:
      *  class sub-code,
      *  subject sub-code, and
      *  enumerated status codes, which include both a subject sub-code
         and a detail sub-code.
   o  Each entry in the tables will include: the sub-code or enumerated
      status code, a definition title for the code, and a description of
      the code's use, similar to the entries shown in [RFC3463].  An
      example of an entry in the enumerated status code table would be:
      X.0.0 Other undefined Status  Other undefined status is the only
         undefined error code.  It should be used for all errors for
         which only the class of the error is known.
   o  The initial values for the class and subject sub-code tables is to
      be populated from section 2 of [RFC3463].  Specifically, these are
      the values for 2.XXX.XXX, 4.XXX.XXX and 5.XXX.XXX for the class
      sub-code table, and the values X.0.XXX, X.1.XXX, X.2.XXX, X.3.XXX,
      X.4.XXX, X.5.XXX, X.6.XXX and X.7.XXX for the subject sub-code
      table.
   o  The initial values for the enumerated status code table is to be
      populated from sections 3.1 through 3.8 of [RFC3463], (X.0.0,
      X.1.0 through X.1.8, X.2.0 through X.2.4, X.3.0 through X.3.5,
      X.4.0 through X.4.7, X.5.0 through X.5.5, X.6.0 through X.6.5, and
      X.7.0 through X.7.7) section 3.3.4 of [RFC3886] (X.1.9), and the
      definition of X.6.6 found in section 5 of [RFC4468].
   o  The following definitions are to be registered in the enumerated
      status code table.

Hansen                  Expires October 11, 2007                [Page 3]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

      X.5.6 Authentication Exchange line is too long  This enhanced
         status code SHOULD be returned when the server fails the AUTH
         command due to the client sending a response which is longer
         than the maximum buffer size available for the currently
         selected SASL mechanism.
      X.7.8 Trust relationship required or Authentication credentials
      invalid  Because of conflicting definitions in different
         documents, this value should no longer be used.
      X.7.9 Authentication mechanism is too weak  This response to the
         AUTH command indicates that the selected authentication
         mechanism is weaker than server policy permits for that user.
         The client SHOULD retry with a new authentication mechanism.
      X.7.10 Encryption Needed  This indicates that external strong
         privacy layer is needed in order to use the requested
         authentication mechanism.  This is primarily intended for use
         with clear text authentication mechanisms.  A client which
         receives this may activate a security layer such as TLS prior
         to authenticating, or attempt to use a stronger mechanism.
      X.7.11 Encryption required for requested authentication mechanism
         This indicates the user's passphrase or passphrase has expired
         and needs to be changed.  Many sites have a policy which
         forbids a passphrase or passphrase from being used too long.
         These sites will set a time period after which passphrases must
         be changed.  Some sites also pre-expire passphrases set by a
         system administrator, such that a user must change their
         passphrase prior to using their account.  A client which
         receives this error code can treat it as a user request to
         change her passphrase.
      X.7.12 A password transition is needed  This response to the AUTH
         command indicates that the user needs to transition to the
         selected authentication mechanism.  This is typically done by
         authenticating once using the [PLAIN] authentication mechanism.
         The selected mechanism SHOULD then work for authentications in
         subsequent sessions.
      X.7.13 User Account Disabled  Sometimes a system administrator
         will have to disable a user's account (e.g., due to lack of
         payment, abuse, evidence of a break-in attempt, etc).  This
         error code occurs after a successful authentication to a
         disabled account.  This informs the client that the failure is
         permanent until the user contacts their system administrator to
         get the account re- enabled.  It differs from a generic
         authentication failure where the client's best option is to
         present the passphrase entry dialog in case the user simply
         mistyped their passphrase.

Hansen                  Expires October 11, 2007                [Page 4]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

      X.7.14 Trust relationship required  The submission server requires
         a configured trust relationship with a third-party server in
         order to access the message content.  This value replaces the
         prior use of X.7.8 for this error condition.
      X.7.15 Authentication credentials invalid  Authentication failed
         due to invalid or insufficient authentication credentials.
         This value replaces the prioruse of X.7.8 for this error
         condition.

3.  Security Considerations

   As stated in [RFC1893], use of enhanced status codes may disclose
   additional information about how an internal mail system is
   implemented beyond that available through the SMTP status codes.

4.  Acknowledgements

   Thanks go to the members of the [email-masked] [1] mailing list.

5.  References

5.1.  Normative References

   [RFC3463]  Vaudreuil, G., "Enhanced Mail System Status Codes",
              RFC 3463, January 2003.

   [RFC3886]  Allman, E., "An Extensible Message Format for Message
              Tracking Responses", RFC 3886, September 2004.

   [RFC4468]  Newman, C., "Message Submission BURL Extension", RFC 4468,
              May 2006.

5.2.  Informative References

   [RFC1893]  Vaudreuil, G., "Enhanced Mail System Status Codes",
              RFC 1893, January 1996.

   [RFC2434]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", BCP 26, RFC 2434,
              October 1998.

URIs

   [1]  <mailto:[email-masked]>

Hansen                  Expires October 11, 2007                [Page 5]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

Author's Address

   Tony Hansen
   AT&T Laboratories
   200 Laurel Ave.
   Middletown, NJ  07748
   USA

   Email: [email-masked]

Hansen                  Expires October 11, 2007                [Page 6]
=0C
Internet-Draft     Mail Enhanced Status Code Registry         April 2007

Full Copyright Statement

   Copyright (C) The IETF Trust (2007).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   [email-masked].

Acknowledgment

   Funding for the RFC Editor function is provided by the IETF
   Administrative Support Activity (IASA).

Hansen                  Expires October 11, 2007                [Page 7]
=0C

--------------030001090900080100010909--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Date:  Mon, 09 Apr 2007 20:53:18 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Hector Santos wrote:

> 1) Must all the codes in the multiline response be the same?

We discussed this in summer 2005 in conjunction with the first
draft, but I don't recall what the outcome at this time was.

> 2) Which is the final return code, the first or the last?

I don't think that different codes are permitted.  If they're
nevertheles different the last paragraph in section 4.2.1 of
the I-D suggests that the last code is the final code, compare
http://tools.ietf.org/html/draft-klensin-rfc2821bis-01#section-4.2.1

Your "150-wait... 150-wait... 4yz sorry" example is clear, but
you can't use 1yz codes without an extension (confirmed by the
client, I think), see the note at the end of the "1yz" part in
section 4.2.1.

Frank

Date: Mon, 09 Apr 2007 08:58:56 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-01 Issue 16: General "is permitted"/ "is not permitted" language
Sender: [email-masked]

John C Klensin wrote:
> 	(4) Have me go through the document, identify the
> 	protocol-specific ones, and change them to
> 	MAY/SHOULD/MUST as appropriate.  This would leave
> 	readers needing to check those choices using a diff or
> 	equivalent and objecting if needed.

+1, with the suggestion that non-normative uses of language that implies 
permission be changed to language that describes choice.  For example, A is 
permitted... could become A might choose to...

The issue, here, is word-smithing to remove potential misunderstanding.  That 
takes diligent technical editing, not group discussion.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 9 Apr 2007 10:35:21 -0400
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: RFC2821bis-01 Issue 16: General "is permitted"/ "is not permitted" language
Sender: [email-masked]

On Sat, Apr 07, 2007 at 10:51:11AM -0400, John C Klensin wrote:
> 	(4) Have me go through the document, identify the
> 	protocol-specific ones, and change them to
> 	MAY/SHOULD/MUST as appropriate.  This would leave
> 	readers needing to check those choices using a diff or
> 	equivalent and objecting if needed.

4 please

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Mon, 09 Apr 2007 07:58:13 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Thanks Frank,

I just took a quick look at the current 2821bis and didn't see anything 
in section 4.2. "SMTP Replies" related to the following. Please tell me 
if you think is it important.

1) Must all the codes in the multiline response be the same?

2) Which is the final return code, the first or the last?

Examples:

  221-Blah blah blah
  221-Blah blah blah
  221-Blah blah blah
  220 Blah blah blah

The next example was a test for a proof of concept regarding potential 
client time outs issues in new environments where the server is 
performing extended backend processing, i.e., OPES for SMTP.

  354 Please Start sending....
  150-Please Wait...
  150-Please Wait...
  451 Sorry, Rejected, try again later

The clients I tested will use the last response-code as the server 
response code but more important it resolve the timeout issue.

Here is an old thread about this:

http://www.imc.org/ietf-openproxy/mail-archive/msg03282.html

--
HLS

Frank Ellermann wrote:
> Hector Santos wrote:
>  
>> Are you saying that you have included a statement in the new
>> text that suggest implementations intentionally use MULTI-LINE
>> welcomes in order to trap bad guys?
> 
> Nope, that was only a minor point in the discussion.  The new
> ABNF will be obvious, something in the direction of Tony's ABNF,
> maybe using "=/" instead of only "/" for the alternative -- the
> latter didn't have the "strongly advised" grouping -- and using
> a defined <Reply-text> instead of an undefined <text> unrelated
> to the 2822 <text>.
> 
> The old prose could be interpreted as if it limits multi-line to
> command replies, so probably John would now just enumerate known
> cases where multi-line is okay explictitly adding the "greeting".
> 
> In theory John could add a note that clients are considered as
> broken if they don't accept multi-line greeting, but IMO that's
> unnecessary, the new ABNF is obvious.  It's no convoluted case
> of "MUST/SHOULD accept" plus "MUST/SHOULD not generate", where
> an explicit explanation in the prose would be required.

Subject: Re: RFC2821bis-01 Issue 16: General "is permitted"/ "is not permitted" language
From: Douglas Otis <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Date: Sun, 08 Apr 2007 19:47:10 -0700
Sender: [email-masked]

On Sat, 2007-04-07 at 10:51 -0400, John C Klensin wrote:
>         (4) Have me go through the document, identify the
>         protocol-specific ones, and change them to
>         MAY/SHOULD/MUST as appropriate.  This would leave
>         readers needing to check those choices using a diff or
>         equivalent and objecting if needed.

4 please. :)

-Doug

Date: Sun, 08 Apr 2007 15:37:51 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Thanks John for your patience.

At this point, I should probably excuse myself from this documentation 
detail process.

One closing point regarding multi-line responses and consequential 
synergistic events:

-  With the advent of faster processing machines, growing,
    advanced/extended state processing, i.e,  AVS filtering,
    OPES, SHIMS, at the DATA state,

-  5 mins timeout recommendation,

-  Method for preventing client time-outs.

Not sure if it warrants a comment, just wanted to instill this neuron 
for future consideration.

Thanks again for your patience.

--
HLS

John C Klensin wrote:
> 
> 
> 
> --On Saturday, April 07, 2007 6:54 PM -0400 Hector Santos 
> <[email-masked]> wrote:
> 
>> John C Klensin wrote:
>>>
>>>
>>> --On Friday, 06 April, 2007 23:39 -0400 Tony Hansen
>>> <[email-masked]> wrote:
>>>
>>>> I've also seen multi-line greetings from various servers for
>>>> several years now.
>>>>
>>>> I think the trend to use multi-line greeting was begun as an
>>>> anti-zombie tactic, and the valid clients were quickly fixed.
>>>
>>> Unless someone wants to argue this the other way, I consider
>>> this item to be closed.  The text has been changed in the
>>> working draft.
>>
>> I'm confused. Please correct me.
>>
>> Are you saying that you have included a statement in the new
>> text that suggest implementations intentionally use MULTI-LINE
>> welcomes in order to trap bad guys?
> 
> No.  I am saying only that I have changed the _syntax_ for the 220 
> greeting to permit a multiline greeting, something that 2821 formally 
> did not permit.
> 
> The draft makes no comment at all about why would would want to generate 
> such a greeting (or not).
> 
>> ...
> 
>    john
> 
> 
> 

Date: Sun, 08 Apr 2007 14:51:32 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

--On Saturday, April 07, 2007 6:54 PM -0400 Hector Santos 
<[email-masked]> wrote:

> John C Klensin wrote:
>>
>>
>> --On Friday, 06 April, 2007 23:39 -0400 Tony Hansen
>> <[email-masked]> wrote:
>>
>>> I've also seen multi-line greetings from various servers for
>>> several years now.
>>>
>>> I think the trend to use multi-line greeting was begun as an
>>> anti-zombie tactic, and the valid clients were quickly fixed.
>>
>> Unless someone wants to argue this the other way, I consider
>> this item to be closed.  The text has been changed in the
>> working draft.
>
> I'm confused. Please correct me.
>
> Are you saying that you have included a statement in the new
> text that suggest implementations intentionally use MULTI-LINE
> welcomes in order to trap bad guys?

No.  I am saying only that I have changed the _syntax_ for the 
220 greeting to permit a multiline greeting, something that 2821 
formally did not permit.

The draft makes no comment at all about why would would want to 
generate such a greeting (or not).

>...

    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Date:  Sun, 08 Apr 2007 16:22:26 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Hector Santos wrote:
 
> Are you saying that you have included a statement in the new
> text that suggest implementations intentionally use MULTI-LINE
> welcomes in order to trap bad guys?

Nope, that was only a minor point in the discussion.  The new
ABNF will be obvious, something in the direction of Tony's ABNF,
maybe using "=/" instead of only "/" for the alternative -- the
latter didn't have the "strongly advised" grouping -- and using
a defined <Reply-text> instead of an undefined <text> unrelated
to the 2822 <text>.

The old prose could be interpreted as if it limits multi-line to
command replies, so probably John would now just enumerate known
cases where multi-line is okay explictitly adding the "greeting".

In theory John could add a note that clients are considered as
broken if they don't accept multi-line greeting, but IMO that's
unnecessary, the new ABNF is obvious.  It's no convoluted case
of "MUST/SHOULD accept" plus "MUST/SHOULD not generate", where
an explicit explanation in the prose would be required.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-01 Issue 16: General "is permitted"/ "is not permitted" language
Date:  Sun, 08 Apr 2007 16:06:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

John C Klensin wrote:
 
> (2) Go through these one at a time, generating issues on
> a case by case basis.  I would recommend against this if
> we really want to get the document out.

Combined with (4) that sounds good, limiting (2) to whatever
other folks (4 = you, 2 = others) identify as important.

> (4) Have me go through the document, identify the
> protocol-specific ones, and change them to
> MAY/SHOULD/MUST as appropriate.  This would leave
> readers needing to check those choices using a diff or
> equivalent and objecting if needed.

If you don't modify too much at once that should work... 
 
> unless at least a few people are willing to carefully
> review the choices made under (4)

...looking at a diff is simple when you didn't shuffle
complete sections.  Actually I think "not permitted" is
"not permitted", adding MUSTard would only emphasize it
for critical cases.  With 2119 keywords I tend to think
that less is better.  Some folks have really odd ideas
what SHOULD means, "a mere recommendation" is the worst
I've heard, I'm in the "cut your throat before violating
a SHOULD" camp (but ready to do this if necessary... :-)

Frank

Date: Sat, 07 Apr 2007 20:00:06 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Hector, Issue 14 is about fixing the ABNF to include the case of
continuation lines in 222 greetings. There was nothing proposed about
changing any wording anywhere.

Do you have a problem with changing the ABNF?

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> John C Klensin wrote:
>>
>>
>> --On Friday, 06 April, 2007 23:39 -0400 Tony Hansen
>> <[email-masked]> wrote:
>>
>>> I've also seen multi-line greetings from various servers for
>>> several years now.
>>>
>>> I think the trend to use multi-line greeting was begun as an
>>> anti-zombie tactic, and the valid clients were quickly fixed.
>>
>> Unless someone wants to argue this the other way, I consider
>> this item to be closed.  The text has been changed in the
>> working draft.
> 
> I'm confused. Please correct me.
> 
> Are you saying that you have included a statement in the new text that
> suggest implementations intentionally use MULTI-LINE welcomes in order
> to trap bad guys?
> 
> I hope not because its not the reason we offer a multi-line system
> policy display.  Its not the reason AOL and others do so as well.  It is
> what is, a logged system policy display and done so for legal reasons.
> 
> If there is a indirect benefit trapping bad guys who do not properly
> honoring it, it is just that, a side effect benefit.  But its not the
> reason for using a multi-line welcome display.
> 
> -- 
> 

Date: Sat, 07 Apr 2007 18:54:26 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

John C Klensin wrote:
> 
> 
> --On Friday, 06 April, 2007 23:39 -0400 Tony Hansen
> <[email-masked]> wrote:
> 
>> I've also seen multi-line greetings from various servers for
>> several years now.
>>
>> I think the trend to use multi-line greeting was begun as an
>> anti-zombie tactic, and the valid clients were quickly fixed.
> 
> Unless someone wants to argue this the other way, I consider
> this item to be closed.  The text has been changed in the
> working draft.

I'm confused. Please correct me.

Are you saying that you have included a statement in the new text that 
suggest implementations intentionally use MULTI-LINE welcomes in order 
to trap bad guys?

I hope not because its not the reason we offer a multi-line system 
policy display.  Its not the reason AOL and others do so as well.  It is 
what is, a logged system policy display and done so for legal reasons.

If there is a indirect benefit trapping bad guys who do not properly 
honoring it, it is just that, a side effect benefit.  But its not the 
reason for using a multi-line welcome display.

--

Date: Sat, 07 Apr 2007 11:55:47 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

--On Friday, 06 April, 2007 23:39 -0400 Tony Hansen
<[email-masked]> wrote:

> 
> I've also seen multi-line greetings from various servers for
> several years now.
> 
> I think the trend to use multi-line greeting was begun as an
> anti-zombie tactic, and the valid clients were quickly fixed.

Unless someone wants to argue this the other way, I consider
this item to be closed.  The text has been changed in the
working draft.

    john

> 
> 	Tony Hansen
> 	[email-masked]

Date: Sat, 07 Apr 2007 11:08:52 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Tony Finch wrote:
> On Fri, 6 Apr 2007, John C Klensin wrote:
>> I don't know how many of them still exist in the wild, but there
>> was a time that a multiline opening greeting would cause some
>> clients to self-destruct.
> 
> There are a number of servers that have a multi-line greeting by default.
> I've been running my server for years now with a multi-line greeting and
> observed no interop problems (3M messages per week).
> 
> Tony.

Concur.

My customer support observations:

Since the time (circa 2003) we added the multi-line 220 system policy
template in our package as part of our overall SMTP AVS/security
enhancements efforts, I recall there some early but few interop reports
from customers.

No doubt there are clients that continue to have problems. However, its
become a matter of "bad vs good".  Good guys complain, report and
resolve issues.  Bad guys don't complain, so they either evolve and
resolve client legacy issues or just ignore it.

Case in point:

The old PHP 3.0 internal "send mail" function was very simplistic and 
was ignorant of multi-line SMTP server responses.

When used by GOOD intention operators, they reported the problems.
Solution:

     a) Upgrade to PHP 4.x/5.x, or
     b) Disable their 220 Multi-line System Policy display.

When used by BAD intention mail blasting bulk spammers, they didn't 
report problems. Solution:

     a) Evolved to SMTP compliant operations, or
     b) DIE!

--
HLS

Date: Sat, 07 Apr 2007 10:51:11 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC2821bis-01 Issue 16: General "is permitted"/ "is not permitted" language
Sender: [email-masked]

Hi.

In going through 2821bis looking for some other things, and in
conjunction with Issue 4, I've discovered that language about
what implementations are permitted (or not permitted) to do is
fairly widespread in 2821.  The two cases identified in Issue 4
(about reply codes) are a tiny minority of the total.   Based on
a quick scan, most of these are intended to be informal,
describing operational alternatives, but some are actually
protocol constraints.  I think we can:

	(1) Do nothing, on the theory that trying to patch these
	as a group would be likely to make things worse.  That
	would probably extend to Issue 4 as well.
	
	(2) Go through these one at a time, generating issues on
	a case by case basis.  I would recommend against this if
	we really want to get the document out.
	
	(3) Extend Section 2.3 to make it clear that, when used
	in a protocol context, these terms are fully as
	normative as the traditional MAY/SHOULD/MUST.
	
	(4) Have me go through the document, identify the
	protocol-specific ones, and change them to
	MAY/SHOULD/MUST as appropriate.  This would leave
	readers needing to check those choices using a diff or
	equivalent and objecting if needed.

My personal preference is for (4) or (1), with the preference
shifting to (1) unless at least a few people are willing to
carefully review the choices made under (4) and we are willing
to accept that we might miss one or two cases.

        john

Date: Sat, 07 Apr 2007 01:09:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

Tony Hansen wrote:

> I've also seen multi-line greetings from various servers for several
> years now.
> 
> I think the trend to use multi-line greeting was begun as an anti-zombie
> tactic, and the valid clients were quickly fixed.

Actually, its done for legal reasons, to display the system policy as 
required by certain US ECPA provisions since 1986. You always had the 
ability to "sue" spammers (or anyone for that matter) for abusing of 
your system. You just had to prove it with damages to make it worth the 
effort. In fact, criminal attacks will get the FBI involved if can you 
show $5000 in damages. (I think it has been raised to $25,000, since we 
used this tax-paid "federal service" in 1999)

For example, a few years back when AOL.COM was suing spammers big time, 
they were using current US laws in the books. If memory serves me right, 
they began to display the following after the major case ended (I don't 
specifically if it was settled or judged).

220-rly-mc05.mail.aol.com ESMTP mail_relay_in-mc5.1;
     Sat, 07 Apr 2007 00:38:31 -0400
220-America Online (AOL) and its affiliated companies do not
220-     authorize the use of its proprietary computers and computer
220-     networks to accept, transmit, or distribute unsolicited bulk
220-     e-mail sent from the internet.  Effective immediately:  AOL
220-     may no longer accept connections from IP addresses which
220      have no reverse-DNS (PTR record) assigned.

After the case, we (Santronics) also followed suit in providing a 
similar default template for our own SMTP customers.

Note: You can see AOL at a later point, expanded it to cover their rDNS 
policy.

--
HLS

Date: Fri, 06 Apr 2007 23:39:02 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies
Sender: [email-masked]

I've also seen multi-line greetings from various servers for several
years now.

I think the trend to use multi-line greeting was begun as an anti-zombie
tactic, and the valid clients were quickly fixed.

	Tony Hansen
	[email-masked]

Tony Finch wrote:
> On Fri, 6 Apr 2007, John C Klensin wrote:
>> I don't know how many of them still exist in the wild, but there
>> was a time that a multiline opening greeting would cause some
>> clients to self-destruct.
> 
> There are a number of servers that have a multi-line greeting by default.
> I've been running my server for years now with a multi-line greeting and
> observed no interop problems (3M messages per week).
> 
> Tony.

Date: Fri, 06 Apr 2007 19:10:27 -0400
From: John C Klensin <[email-masked]>
To: Tony Finch <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting
Sender: [email-masked]

--On Friday, 06 April, 2007 23:57 +0100 Tony Finch
<[email-masked]> wrote:

> 
> On Fri, 6 Apr 2007, John C Klensin wrote:
>> 
>> I don't know how many of them still exist in the wild, but
>> there was a time that a multiline opening greeting would
>> cause some clients to self-destruct.
> 
> There are a number of servers that have a multi-line greeting
> by default. I've been running my server for years now with a
> multi-line greeting and observed no interop problems (3M
> messages per week).

Ok.  Unless someone else objects, that is persuasive.

     john

Date: Fri, 6 Apr 2007 23:57:07 +0100
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies (was: Re: 2821bis server reply ABNF)
Sender: [email-masked]

On Fri, 6 Apr 2007, John C Klensin wrote:
>
> I don't know how many of them still exist in the wild, but there
> was a time that a multiline opening greeting would cause some
> clients to self-destruct.

There are a number of servers that have a multi-line greeting by default.
I've been running my server for years now with a multi-line greeting and
observed no interop problems (3M messages per week).

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTH TYNE DOGGER: MAINLY WEST OR NORTHWEST 3 OR 4. MODERATE, OCCASIONALLY
ROUGH. FAIR. MODERATE OR GOOD.

Date: Fri, 06 Apr 2007 18:50:49 -0400
From: John C Klensin <[email-masked]>
To: Tony Finch <[email-masked]>, [email-masked]
Subject: rfc2821bis-01 Issue 14 Continuation of 222 greeting and Issue 15 syntax for multiline replies (was: Re: 2821bis server reply ABNF)
Sender: [email-masked]

--On Thursday, 05 April, 2007 17:29 +0100 Tony Finch
<[email-masked]> wrote:

> 
> Section 4.2 says:
> 
>    In ABNF, server responses are:
> 
>    Greeting       = "220 " (Domain / address-literal) [ SP
> text ] CRLF
> 
>    Reply-line     = Reply-code [ SP text ] CRLF
> 
> This omits the multiline response cases, and it omits the
> definition of Reply-code, so it should be corrected to:
> 
>    Greeting       =   "220 " (Domain / address-literal) [ SP
> text ] CRLF                   /   "220-" (Domain /
> address-literal) [ SP text ] CRLF                    *( "220-"
> [ text ] CRLF )
>                       "220" [ SP text ] CRLF

I don't know how many of them still exist in the wild, but there
was a time that a multiline opening greeting would cause some
clients to self-destruct.  Frank's comments about trying to
confuse some bots may be consistent with that observed behavior.
And the greeting is definitely not a "reply to a command".
I'll do whatever people like but, unless someone can show a
reason why a multiline greeting is actually necessary, I'm a bit
reluctant to change that just for symmetry purposes.  Issue 14
is now "continuation on 220 Greeting".

>    Reply-line     = *( Reply-code "-" [ text ] CRLF )
>                        Reply-code [ SP text ] CRLF
> 
>    Reply-code     = %x31-35 %x30-35 %x30-39

That was, IMO, clearly a bug.  Will fix unless someone objects
RSN.  Issue 15 for tracking purposes.
> 
> Tony.

Date: Thu, 05 Apr 2007 19:12:17 -0400
From: "Eric A. Hall" <[email-masked]>
CC: [email-masked]
Subject: Re: draft-hall-prdr-00
Sender: [email-masked]

On 3/11/2007 2:48 PM, Eric A. Hall wrote:
> 
> I submitted the PRDR draft to the I-D editor a week ago, but it appears to
> have been put into a will-process queue that is on hold until the 19th.

The draft was cleared and is available from the I-D repositories at
http://www.ietf.org/internet-drafts/draft-hall-prdr-00.txt

I've got a handful of edits in my local queue but will wait another week
for more. Again, I'm mostly concerned about the behaviors in sections 4.6
and 4.7

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: 2821bis server reply ABNF
Date:  Thu, 05 Apr 2007 20:56:48 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Tony Finch wrote:
 
>    In ABNF, server responses are:

>    Greeting       = "220 " (Domain / address-literal) [ SP text ] CRLF
[...]
> This omits the multiline response cases

Yes, that could be an intentional omission for 821 compatibility (?)

| Only the EXPN and HELP commands are expected to result in multiline
| replies in normal circumstances, however multiline replies are 
| allowed for any command.

That text appears almost verbatim (adding EHLO) also in 2821.  Arguably
the greeting is no reply to a command.  And the client could intend to
use HELO after the greeting.

IIRC some mailers use multiline greetings to confuse naive direct-to-MX
bot-scripts, but probably that's already obsolete (?)

> it should be corrected to:
 
>    Greeting       =   "220 " (Domain / address-literal) [ SP text ] CRLF
>                   /   "220-" (Domain / address-literal) [ SP text ] CRLF
>                    *( "220-" [ text ] CRLF )
>                       "220" [ SP text ] CRLF

IMHO "Greeting =/" introducing the multiline case would be clearer.
The "strongly advised" grouping could muddy the waters here.

>    Reply-line     = *( Reply-code "-" [ text ] CRLF )
>                        Reply-code [ SP text ] CRLF
> 
>    Reply-code     = %x31-35 %x30-35 %x30-39

+1.  That's fortunately not the <text> in 2822, but apparently it's not
specified anywhere.  Maybe something in the direction of...

     text           = 1*VCHAR *( 1*WSP 1*VCHAR ) 

...but using a better name, e.g. <Reply-text>.

Frank

Date: Thu, 5 Apr 2007 17:29:46 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: 2821bis server reply ABNF
Sender: [email-masked]

Section 4.2 says:

   In ABNF, server responses are:

   Greeting       = "220 " (Domain / address-literal) [ SP text ] CRLF

   Reply-line     = Reply-code [ SP text ] CRLF

This omits the multiline response cases, and it omits the definition of
Reply-code, so it should be corrected to:

   Greeting       =   "220 " (Domain / address-literal) [ SP text ] CRLF
                  /   "220-" (Domain / address-literal) [ SP text ] CRLF
                   *( "220-" [ text ] CRLF )
                      "220" [ SP text ] CRLF

   Reply-line     = *( Reply-code "-" [ text ] CRLF )
                       Reply-code [ SP text ] CRLF

   Reply-code     = %x31-35 %x30-35 %x30-39

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HEBRIDES BAILEY: WESTERLY 4 OR 5. MODERATE BECOMING ROUGH. OUTBREAKS OF RAIN
OR DRIZZLE. GOOD, OCCASIONALLY MODERATE OR POOR.

Date: Tue, 3 Apr 2007 19:55:40 -0400
From: John Leslie <[email-masked]>
To: Tony Hansen <[email-masked]>
Cc: [email-masked]
Subject: Re: RFC2821bis-01 Issue 4: Client actions on receipt of 4yz and 5yz codes
Sender: [email-masked]

Tony Hansen <[email-masked]> wrote:
>> John C Klensin wrote:
>>> In RFC 2821bis-01, Section 4.2.1 (Reply Code Severities and Theory), in
>>> the description of the generic 4yz category, described the desired
>>> action in terms of what the client "is encouraged to" do.  I was
>>> persuaded in an off-list exchange that this can reasonably be changed to
>>> "SHOULD" and -01 reflects that change.   For 5yz, there was a similar
>>> construction that indicates that the client "is discouraged from"
>>> repeating the request.  I was less confident about changing that to
>>> "SHOULD NOT", so the "is discouraged" text remains in -01.
>>>
>>> Question: Should the 5yz description  be changed to use "SHOULD NOT".  
>>> If not, is "SHOULD" in the 4yz description reasonable?
> 
> <snip>
> 
> For 4yz, this does not show any consensus to change from SHOULD.
> 
> For 5yz, there is a trend to use SHOULD NOT instead of "is discouraged",
> but the numbers are too low to really call a consensus yet.

   I'd like to endorse that course of action. We can argue whether some
other wording is more technically accurate; but matching a SHOULD against
a SHOULD NOT is the easiest to read and technically correct. (Anybody
who can't figure out a few cases where it's time to stop beating the
dead horse, or the problem "has been fixed" needs to be reading a quite
different document!)

--
John Leslie <[email-masked]>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  2821 Dot-string (was: RFC2821bis-01 Issue 5: Syntax of ID clause in Received trace header)
Date:  Tue, 03 Apr 2007 18:07:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 32
Sender: [email-masked]

Tony Hansen wrote:

> Unless people can find examples where quoted-strings were actually used,
> I would like to close this issue out with the recommendation to change
> "string" to "atom".

You'd need <Dot-string> instead of <Atom> if you want to allow dots.

A 2821 <Dot-string> is exactly the same as a 2822 <dot-atom-text>, for
consistency with 2822 I'd prefer to replace s/Dot-string/dot-atom-text/g
everywhere:

[2822 3.2.4]:
| atom            =       [CFWS] 1*atext [CFWS]
| dot-atom        =       [CFWS] dot-atom-text [CFWS]
| dot-atom-text   =       1*atext *("." 1*atext)

[2821bis-01]:
| Dot-string     = Atom *("."  Atom)
| Atom           = 1*atext

It's already bad that 2821 and 2822 use similar terms for syntactically
different things, e.g. <Atom> vs. <atom>, or <Mailbox> vs. <mailbox>,
confusing some folks including me again and again.

The opposite case, different terms for syntactically identical concepts,
is IMO just a bad idea.  Maybe there are historical reasons for it, and
maybe DRUMS intentionally wanted to confuse future readers, but I think
it's wrong without further evidence.

Frank

Date: Tue, 3 Apr 2007 13:14:24 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: TINW was RFC2821b is-01 Issue 1: trailing dot in Domain
Cc: Tony Finch <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

Tony Finch writes:
> On Thu, 29 Mar 2007, Arnt Gulbrandsen wrote:
>>  Well... I ran a little xargs dig | sort | grep I wc loop now, and 
>>  see a depressingly large number of TLDs now have MX records. The 
>>  first time I counted that was in the early nineties: 1 TLD MX. Then 
>>  second was IIRC around the time 2821 was published: 0. And the 
>>  third today: 25.
>
> There are also several TLDs with A records - only a couple have both. 
> None have AAAA records :-)
>
>>  What does this mean? I'm not sure. I think I'd like to hear what 
>>  some of the people operating those MXes thing. Shall I spam 
>>  postmaster@ those and ask?
>
> You'll be disappointed :-)

On the contrary.

> I've done some non-intrusive probes (MAIL/RCPT/RSET) to see what I can 
> find. Most of them are horribly misconfigured - the MTAs don't think 
> the domain is local or aren't listening. Only five seem to work (ai, 
> io, tk, tt*, ua), in that they accept email to postmaster and not to 
> random addresses. Three of these are set up with the zone admin's 
> first name as a valid local part. Five accept anything (as, bj*, hr, 
> mh*, va*). The four I have marked with * have broken secondary MXs.

I read this as saying "2821 is okay - the extra permissiveness of 821 is 
worthless in practice". Fine with me.

The reason I am not disappointed: Since there are no AAAA records, the 
set of IPv6-using TLD operators and that of horribly clueless TLD 
operators may just possibly be disjoint.

(For your next exercise, find out how many of those 25/26 TLD operators 
have blogs.)

Arnt

Date: Tue, 3 Apr 2007 12:02:43 +0100
From: Tony Finch <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked], John C Klensin <[email-masked]>
Subject: Re: TINW was RFC2821b is-01 Issue 1: trailing dot in Domain
Sender: [email-masked]

On Thu, 29 Mar 2007, Arnt Gulbrandsen wrote:
>
> Well... I ran a little xargs dig | sort | grep I wc loop now, and see a
> depressingly large number of TLDs now have MX records. The first time I
> counted that was in the early nineties: 1 TLD MX. Then second was IIRC
> around the time 2821 was published: 0. And the third today: 25.

There are also several TLDs with A records - only a couple have both. None
have AAAA records :-)

> What does this mean? I'm not sure. I think I'd like to hear what some of
> the people operating those MXes thing. Shall I spam postmaster@ those
> and ask?

You'll be disappointed :-) I've done some non-intrusive probes
(MAIL/RCPT/RSET) to see what I can find. Most of them are horribly
misconfigured - the MTAs don't think the domain is local or aren't
listening. Only five seem to work (ai, io, tk, tt*, ua), in that they
accept email to postmaster and not to random addresses. Three of these
are set up with the zone admin's first name as a valid local part.
Five accept anything (as, bj*, hr, mh*, va*). The four I have marked
with * have broken secondary MXs.

http://fanf.livejournal.com/72486.html

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH UTSIRE SOUTH UTSIRE: NORTH BACKING NORTHWEST 5 TO 7, OCCASIONALLY GALE
8. MODERATE OR ROUGH INCREASING ROUGH OR VERY ROUGH. RAIN LATER. MODERATE OR
GOOD.

Date: Tue, 03 Apr 2007 06:55:44 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: RFC2821bis-01 Issue 3: EHLO parameter
Sender: [email-masked]

John C Klensin wrote:
> --On Sunday, 01 April, 2007 14:26 -0700 Dave Crocker
> 
> ...
> 
>     smtpout.example.com. AAAA ....
>     example.com.         MX 0 smtpin.example.com.
>     stmpin.example.com.  A  ....
>                          AAAA ....
> 
 > ...
> Now assume a mail transaction that looks something like:
>     EHLO smtpout.example.com
>     MAIL FROM:<[email-masked]>
>     RCPT TO:<[email-masked]>
>     ...
 >
 > ...
> The receiving server does a DNS lookup, type=A, on
> "smtpout.example.com".  It fails.  Do you want to _reject_ this
> message or encourage anyone to do so based on that information?
> Or suppose it does do the type=AAAA lookup as well as the type=A
> one.  It gets back the IPv6 record but, because of the
> packet-level address remapping gateway, that address is not the
> same as the address of the TCP connection to the SMTP port.   Do
> you want to _reject_ the message on that basis?
> 
> Bad idea, IMO.  Probably a worse idea than it was in the simpler
> world of 2000 (RFC 2821) or 1989 (RFC 1123).

With it comes to IPv6, I agree. IPv6 considerations for EHLO/HELO simply 
increases the potential for unreliability.

But there is something I am not quite following.  Isn't it pretty much 
long settled that anyone with an interest in IPv6 including us as long 
time SMTP vendors that has done early IPv6 implementation consideration 
R&D, that including IPv6 support at this time requires an awful amount 
of assumptions to be in place?  You cited many of the presumed possible 
requirements and needs.

But this is why I lean to removing all the subjective recommendations 
and just lay down the technical facts as we known it today and will be 
in the future.

> Do you even want to mark it "suspicious" before delivering it?
> Fortunately, 2821bis takes no position at all on that subject.

Right. I think the point here is that we are wasting time trying to mode 
or finesse a policy that IMV shouldn't be there.   Laying down the 
contemporary technical issues is good enough IMV.  Let implementations 
decide.  Who knows? Maybe tomorrow someone will invent, propose a more 
reliable concept for EHLO including IPv6 verification.  The problem? 
These specific new proposals will need to update 2821bis to remove its 
"MUST NOT/SHOULD NOT" restriction.

> Two added comments:
> 
> (1) It does seem to me that it might be useful to add, possibly
> to the Security Considerations section, a comment to the effect
> that the tests required or prohibited by SMTP in order to
> determine whether to reject or deliver mail and questions about
> how far to push the robustness principle in terms of what to
> accept and whether deviations that are still considered
> deliverable should be evaluated in message classification, are
> outside the scope of 2821bis.   That would, of course, require
> making up new text, so it would be useful to think about whether
> such text would be welcome before we start thinking about how to
> craft it.

This is just an opinion.  Maybe we should refrain from the terms "reject 
mail" or "mail rejection," etc. The SMTP receiver is not rejecting the 
mail, it is creating a negative response or condition that prohibits the 
continuance of the transaction.  Thats a big difference because SMTP 
level negative responses are blind to mail content interpretations.   Of 
course, in a POST SMTP or dynamic DATA "shim" consideration , the mail 
is subject to mail content subjective considerations.

But the difference is we are strictly talking about SMTP level state 
(machine) validation here I believe.

IMV, there are two areas in SMTP commands that fall under the same 
issues and consideration: The client domain (EHLO/HELO) and the return 
path (MAIL FROM).   IMV, the discussion of MAIL FROM is pretty "near" 
perfect:

   - We KNOW it must be valid

   - We make no statement how we prove that, we have an
     inherent presumption that it is valid for Bounce purposes.

   - But we also give some excellent technical guidance
     for implementations such as checking for RCPT first
     before applying any MAIL FROM verification ideas.

But just imagine if back in 2000 when you did 2821, if we had all the 
current LMAP generation verification ideas in place? such as DMP, 
CALLER-ID, SPF, SENDER-ID, and the growing SMTP based CallBack 
verification implementations?  I am willing to bet that someone would of 
suggested a similar restriction for MAIL FROM for the same reasons you 
have for EHLO/HELO with its DNS unreliable lookup issues:

    "Return Path Verification Ideas out of scope.....
     However, verification failures MUST NOT reject on the
     basis of LMAP  considerations or any other kind of dynamic
     SMTP return path domain DNS lookup methodology."

Isn't that the essentially the same thing we have for EHLO/HELO?

Fortunately, 2821 did not have such MAIL FROM restrictions and new 
technology was able to evolve, explored and implemented.

> (2) It seems to me that the text of 7.8 should be clarified to
> include something like "unless otherwise constrained above or by
> extensions that are in use" to eliminate any possible confusion
> about apparent contradictions and that this should be done
> regardless of what changes are made, if any, to the specific
> EHLO text.  Are there objections to my doing that, in spite of
> the fact that it is "new text" and adds to the word count?

Objectively, all this is already a reality. Subjectively, the only 
remaining question is how long will it take the editors (and WG) to 
finally start taking it into account.  I think you can add the extra 
text to cover the wider range of contemporary realities.

Speaking for myself, I have no feel for these "next text" or "word 
count" conflicts being wage between the principles here. :-) I more for 
"getting it right, the first time" (a metaphor) and move on.  If the 
extra text helps, I vote for it.

That said, in general, whether it is this issue or others, in principle 
I vote for removing any subjective, technical restrictions, and lay down 
what technical realities which isn't a philosophical debate or question.

I think what you have in 7.8 is a good example, for example, to me, you 
can remove and move the relay discussion into own 7.x section and add a 
forward pointer in the EHLO/HELO section.  Then the 7.8 "SCOPE" section 
is reduced to:

   --

    7.8.  Scope of Operation of SMTP Servers

    It is a well-established principle that an SMTP server may refuse to
    accept mail for any operational or technical reason that makes sense
    to the site providing the server.

    However, cooperation among sites and installations makes the
    Internet possible.  If sites take excessive advantage of the right
    to reject traffic, the ubiquity of email availability (one of
    the strengths of the Internet) will be  threatened; considerable
    care should be taken and balance maintained if a site decides to
    be selective about the traffic it will accept and process.

   --

The specifics about 550 or other response code was already discussed in 
other specific sections.

--
HLS

Date: Tue, 03 Apr 2007 02:34:43 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-01 Issue 5: Syntax of ID clause in Received trace header
Sender: [email-masked]

John C Klensin wrote:
> 
> In Section 4.4 on Trace Information, the "ID" clause of the Received
> field is defined as
> 
>     ID = CFWS "ID" FWS String / msg-id
> 
> This is claimed to be inconsistent with RFC 2822, which only permits
> msg-id.
> 
> "String" is there to provide additional flexibility for gateways, etc.,
> for which a message-ID format may not be appropriate, but it has been
> suggested that we remove it in the interest of consistency.
> 
> Question: Is the production correct as is, or should "String" be
> eliminated as a possibility?
> 
> Note that, if the answer is "ok as is", we may want to think about
> whether we are happy with the syntax in in 2822.

To summarize the comments that have been received onlist:

	eliminate string	1
	keep string		1
	change string to atom	3

If we change string to atom, from the ABNF productions, this would
remove quoted-string from the possibilities for use in Received, but
leave atoms in the production list. Given the examples of real world
data we've seen so far, showing both atoms and msg-ids but not
quoted-strings, this seems reasonable. This is actually a perfect
example where we're narrowing the definition to match what's actually
been implemented.

Unless people can find examples where quoted-strings were actually used,
I would like to close this issue out with the recommendation to change
"string" to "atom".

Your pseudo-chair,
	Tony Hansen

Date: Tue, 03 Apr 2007 02:12:07 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-01 Issue 4: Client actions on receipt of 4yz and 5yz codes
Sender: [email-masked]

> John C Klensin wrote:
>> In RFC 2821bis-01, Section 4.2.1 (Reply Code Severities and Theory), in
>> the description of the generic 4yz category, described the desired
>> action in terms of what the client "is encouraged to" do.  I was
>> persuaded in an off-list exchange that this can reasonably be changed to
>> "SHOULD" and -01 reflects that change.   For 5yz, there was a similar
>> construction that indicates that the client "is discouraged from"
>> repeating the request.  I was less confident about changing that to
>> "SHOULD NOT", so the "is discouraged" text remains in -01.
>>
>> Question: Should the 5yz description  be changed to use "SHOULD NOT".  
>> If not, is "SHOULD" in the 4yz description reasonable?

To summarize the comments that have been received onlist:

Wording for 4yz:
	leave as "is encouraged"	0
	leave as "SHOULD"		0
	MAY is better wording		1

Wording for 5yz:
	leave as "is discouraged"	0
	SHOULD NOT is better wording	2
	could be MUST NOT		1
	MUST NOT is better wording	1

For 4yz, this does not show any consensus to change from SHOULD.

For 5yz, there is a trend to use SHOULD NOT instead of "is discouraged",
but the numbers are too low to really call a consensus yet.

Your pseudo-chair,
	Tony Hansen
	[email-masked]

Date: Mon, 02 Apr 2007 16:34:37 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC2821bis-01 Issue 3: EHLO parameter
Sender: [email-masked]

John C Klensin wrote:
>> However, I hope that folks keep in mind that reality tends to
>> override working group consensus.
> 
> As long as we can agree on what the reality is, of course.

well, umm, err... yeah.

>> And that's all I am suggesting here, but by a minimizing
>> approach, rather than trying to create new language.
> 
> There isn't much "creating new language" implied in anything
> I've seen and understood.  There have been some suggestions
 > about changing a MUST to a SHOULD, which is certainly not new

Strictly speaking, changing words means making new words, although of course 
that's a quibble.  In reality, I was too terse and too academic.  Sorry.

I wanted to mark one end of a spectrum, and didn't make clear that I was 
talking more broadly than the immediate MUST->SHOULD proposal.

That said I think that "SHOULD" entails risks on a par with retaining MUST: 
Current reality is fuzzy and changing.  A core email spec making *any* 
statement in this arena is likely to find itself undermined relatively 
quickly, and that assumes (as above) that folks can agree that true community 
consensus -- at this moment -- is SHOULD.

> Note also that removing text that provides very strong guidance
> that a particular test is not especially useful is not, IMO, a
> "minimizing approach", unless your definition of minimizing is
> the trivial one of simply reducing the word count. 

It takes the core spec out of any attempt to assert correct policy for this 
particular issue.  Rather than try to walk a line, be subtle or otherwise 
finesse things, it simply treats the issue as out of scope.

Reducing scope certainly seems like a minimizing approach, to me.

  In all that
> has changed in the last several years, I've heard no suggestions
> that the value of this test _as a criterion for rejecting a
> message_ has significantly increased.  

I wonder how much of the email anti-abuse community has polled on this issue?

> The receiving server does a DNS lookup, type=A, on
> "smtpout.example.com".  It fails.  Do you want to _reject_ this
> message or encourage anyone to do so based on that information?
> Or suppose it does do the type=AAAA lookup as well as the type=A
> one.  It gets back the IPv6 record but, because of the
> packet-level address remapping gateway, that address is not the
> same as the address of the TCP connection to the SMTP port.   Do
> you want to _reject_ the message on that basis?

I think you have, as usual, done an excellent job of showing an example that 
underscores the complexities in this arena.

My point is that a core spec should try to avoid such complexities, if it can. 
  In this case, it can, merely by dropping the text.

It's not that the topic is not important. It is that it is messy, has become 
poorly understood, is part of a significantly changing landscape of policy 
formulations, and currently demonstrates a pretty wide range of community 
operational preferences, from what I can tell.

> Do you even want to mark it "suspicious" before delivering it?
> Fortunately, 2821bis takes no position at all on that subject.

Again, that's the point:  rejection vs. marking are points along a continuum 
of choices and the core mail transport spec need not cover it, nevermind cover 
it partially.

> I think I agree with your principle, but not with the
> application to this text, for the reasons above.

Well, heck, *that's* never happened before...

> Two added comments:

Noted.  Certainly a conciliatory effort.  My own view is that the real issue 
is with any normative text in this arena, but your text is a plausible fall-back.

d/

ps. We've explored our views pretty thoroughly.  Absent any sort of emerging 
view that my suggestion be pursued further -- and my observation is that the 
emergence is most definitely absent -- I'll retreat back into my corner.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 2 Apr 2007 18:16:28 -0400
To: Frank Ellermann <[email-masked]>
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: More references in 2821bis
Cc: [email-masked]
Sender: [email-masked]

At 20:06 +0200 on 04/02/2007, Frank Ellermann wrote about Re: More 
references in 2821bis:

>Robert A. Rosenberg wrote:
>
>  [Apparently-To]
>>  I seem to remember that it ALSO needed the absence of a Cc header
>>  to trigger this injected listing.
>
>Plausible, maybe RFC 2076 just didn't mention this detail.  RFC 2821
>7.2 doesn't discuss any details, and RFC 3552 6.1.1.2 is only a copy
>of RFC 2821 7.2.
>
>Somebody on the EAI list got me to dig out a stoneage bat book, with
>that at hand I find:
>
>  <time travel=1994>
>| If the header of a mail message lacks all recipient information
>| (lacks all of the To:, Cc:, Bcc: header lines), sendmail adds an
>| Apparently-To: header line and puts the recipient's address in
>| from the envelope into the field of that line.  This behaviour
>| is hardcoded into sendmail.
>|
>| The Apparently-To header name is not defined in RFC822.  It is
>| added by sendmail because RFC822 requires at least one To: or Cc:
>| header, and neither is present.
>  </time>
>
>Trying to fix missing header fields by adding a different header
>field was a rather odd idea... <g>   And "apparently" it did not an
>Apparently-To when there was (only) a Bcc, but it stripped the Bcc,
>again arriving at no RFC822 recipient at all.  Oh well...  And I'll
>fix my header field name review request, thanks for your hint.
>
>Frank

You're welcome. I tend to be a nit-picker and tend to want to see 
details of this type accurate.

My memory that a Cc would suppress the injection of the header, seems 
to be borne out by that quote above. As to the Bcc, it was/is 
normally never sent by the MUA (although some did, if I remember) but 
was only used to Mail-To list that was passed on by the MUA.

Date: Mon, 02 Apr 2007 15:52:13 -0400
From: John C Klensin <[email-masked]>
To: Lisa Dusseault <[email-masked]>
cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions (was: Re: IPv6)
Sender: [email-masked]

--On Monday, 02 April, 2007 09:32 -0700 Lisa Dusseault
<[email-masked]> wrote:

> 
> On Mar 31, 2007, at 5:24 PM, John C Klensin wrote:
> 
>> having to prohibit IPv6 (or IPv6-without-IPv4) in 2007
>> seems strange and very unfortunate.  RFC3974, which does
>> derive from operational experience, doesn't shed a lot of
>> light on this: a quick summary is that IPv4-only is ok,
>> IPv6-only is ok, but, for mixtures or unknown destinations,
>> any IPv6 host had better be running dual-stack with some IPv4
>> MX records.  That document contains a few statements that I
>> believe to be serious errors (perhaps the reason it was
>> published as an independent submission with strong
>> disclaimers), but it is at least a thoughtful attempt to
>> explore the problem.
>> 
>> I need consensus advice on this one.
> 
> Do we have the right IPv6 experience on the list for this one?
> I'm not sure who to consult, but there may be experts not on
> this list (please pipe up if you are an expert and on this
> list already, or if you have a suggestion who to consult).
> There's probably relevant operational experience out there
> already.

I know I'm not an expert, but, given Frank's response to my
note, I would claim that there is some small evidence that I've
thought about it enough to be able to identify some of the nasty
edge cases.

Certainly there is expertise out there, but the answers may not
be clear even then.  For example, the authors of RFC 3974
thought about the issues and apparently ran some production
systems.  They then produced a draft based on their experience
that they community  apparently didn't consider solid enough to
publish as a BCP and hence sent down the "informational" track
with a rather strong disclaimer.  

Let me offer a hypothesis: We could go out and round up the
necessary expertise and the people with the right experience to
upgrade 3974 into something that could be published as a BCP or
on the Standards Track.  It is probably worth doing, but it
wouldn't be quick because, probably, everyone with the
experience is busy (e.g., in my case, I don't think I could
spend significant energy on it until after we get EAI put to
bed, but I may not be critical).   That turns into a question to
you, Chris, and the people who are really anxious to get 2821bis
published: can we figure out a responsible way to finesse this
in RFC2821bis and move it forward?  Or do we need to derail
2821bis until we can resolve the IPv6 issues and write them up,
either in the document or in something that it can reference?

best,
    john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: RFC2821bis-01 Issue 3: EHLO parameter
Date:  Mon, 02 Apr 2007 20:57:59 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 42
Sender: [email-masked]

John C Klensin wrote:

> suppose that such a domain has DNS records as follows:

>     smtpout.example.com. AAAA ....
>     example.com.         MX 0 smtpin.example.com.
>     stmpin.example.com.  A  ....
>                          AAAA ....

> suppose also that the network that supports these hosts
> has a network layer IPv6-IPv4 conversion gateway, a
> gateway that makes its conversions at the packet layer,
> invisible to SMTP.

Arrggh.  Strike my comment trying to "improve" the proposed
SHOULD NOT, and strike my "IPv6 gateway considerations". :-(
I completely missed this "invisible" case.

The potential "IPv6" issues are really ugly.  We're near to 
a "MUST NOT IPv6 only" (but not exactly a "new" requirement
if it was always implied).

> It does seem to me that it might be useful to add, possibly
> to the Security Considerations section, a comment to the
> effect that the tests required or prohibited by SMTP in
> order to determine whether to reject or deliver mail and
> questions about how far to push the robustness principle
> in terms of what to accept and whether deviations that are
> still considered deliverable should be evaluated in message
> classification, are outside the scope of 2821bis.

IMO not discussing it would cause havoc when folks just try
to do the right thing, and miss a scenario like your example
in their considerations.

Outside of the security considerations we need a caveat or
note that the IP address of the client is not necessarily
the IP seen by the server, even if the obvious NAT cases are
excluded.  [[ And that caveat has to be copied to 4408bis. ]]

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: More references in 2821bis
Date:  Mon, 02 Apr 2007 20:06:31 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 33
Sender: [email-masked]

Robert A. Rosenberg wrote:

 [Apparently-To]
> I seem to remember that it ALSO needed the absence of a Cc header
> to trigger this injected listing.

Plausible, maybe RFC 2076 just didn't mention this detail.  RFC 2821
7.2 doesn't discuss any details, and RFC 3552 6.1.1.2 is only a copy
of RFC 2821 7.2.

Somebody on the EAI list got me to dig out a stoneage bat book, with
that at hand I find:

 <time travel=1994>
| If the header of a mail message lacks all recipient information
| (lacks all of the To:, Cc:, Bcc: header lines), sendmail adds an
| Apparently-To: header line and puts the recipient's address in
| from the envelope into the field of that line.  This behaviour
| is hardcoded into sendmail.
|
| The Apparently-To header name is not defined in RFC822.  It is
| added by sendmail because RFC822 requires at least one To: or Cc:
| header, and neither is present.
 </time>

Trying to fix missing header fields by adding a different header
field was a rather odd idea... :-)  And "apparently" it did not an
Apparently-To when there was (only) a Bcc, but it stripped the Bcc,
again arriving at no RFC822 recipient at all.  Oh well...  And I'll
fix my header field name review request, thanks for your hint.

Frank

Cc: Frank Ellermann <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Re: RFC2821bis-01 Issue 12: IPv6 MX records and transitions (was: Re: IPv6)
Date: Mon, 2 Apr 2007 09:32:47 -0700
To: John C Klensin <[email-masked]>
Sender: [email-masked]

--Apple-Mail-8--344127855
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

On Mar 31, 2007, at 5:24 PM, John C Klensin wrote:

> having to prohibit IPv6 (or IPv6-without-IPv4) in 2007
> seems strange and very unfortunate.  RFC3974, which does derive
> from operational experience, doesn't shed a lot of light on
> this: a quick summary is that IPv4-only is ok, IPv6-only is ok,
> but, for mixtures or unknown destinations, any IPv6 host had
> better be running dual-stack with some IPv4 MX records.  That
> document contains a few statements that I believe to be serious
> errors (perhaps the reason it was published as an independent
> submission with strong disclaimers), but it is at least a
> thoughtful attempt to explore the problem.
>
> I need consensus advice on this one.

Do we have the right IPv6 experience on the list for this one?  I'm  
not sure who to consult, but there may be experts not on this list  
(please pipe up if you are an expert and on this list already, or if  
you have a suggestion who to consult).  There's probably relevant  
operational experience out there already.

Lisa
--Apple-Mail-8--344127855
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=ISO-8859-1

<HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode: space; =
-khtml-line-break: after-white-space; "><BR><DIV><DIV>On Mar 31, 2007, =
at 5:24 PM, John C Klensin wrote:</DIV><BR =
class=3D"Apple-interchange-newline"><BLOCKQUOTE type=3D"cite"><P =
style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">having to prohibit IPv6 (or =
IPv6-without-IPv4) in 2007</FONT></P> <P style=3D"margin: 0.0px 0.0px =
0.0px 0.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px =
Helvetica">seems strange and very unfortunate.<SPAN =
class=3D"Apple-converted-space">=A0 </SPAN>RFC3974, which does =
derive</FONT></P> <P style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT =
face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px Helvetica">from =
operational experience, doesn't shed a lot of light on</FONT></P> <P =
style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">this: a quick summary is =
that IPv4-only is ok, IPv6-only is ok,</FONT></P> <P style=3D"margin: =
0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" size=3D"3" =
style=3D"font: 12.0px Helvetica">but, for mixtures or unknown =
destinations, any IPv6 host had</FONT></P> <P style=3D"margin: 0.0px =
0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: =
12.0px Helvetica">better be running dual-stack with some IPv4 MX =
records.<SPAN class=3D"Apple-converted-space">=A0 </SPAN>That</FONT></P> =
<P style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" =
size=3D"3" style=3D"font: 12.0px Helvetica">document contains a few =
statements that I believe to be serious</FONT></P> <P style=3D"margin: =
0.0px 0.0px 0.0px 0.0px"><FONT face=3D"Helvetica" size=3D"3" =
style=3D"font: 12.0px Helvetica">errors (perhaps the reason it was =
published as an independent</FONT></P> <P style=3D"margin: 0.0px 0.0px =
0.0px 0.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px =
Helvetica">submission with strong disclaimers), but it is at least =
a</FONT></P> <P style=3D"margin: 0.0px 0.0px 0.0px 0.0px"><FONT =
face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px =
Helvetica">thoughtful attempt to explore the problem.</FONT></P> <P =
style=3D"margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; =
min-height: 14.0px"><BR></P> <P style=3D"margin: 0.0px 0.0px 0.0px =
0.0px"><FONT face=3D"Helvetica" size=3D"3" style=3D"font: 12.0px =
Helvetica">I need consensus advice on this one.</FONT></P> =
</BLOCKQUOTE></DIV><BR><DIV>Do we have the right IPv6 experience on the =
list for this one?=A0 I'm not sure who to consult, but there may be =
experts not on this list (please pipe up if you are an expert and on =
this list already, or if you have a suggestion who to consult).=A0 =
There's probably relevant operational experience out there =
already.</DIV><DIV><BR =
class=3D"khtml-block-placeholder"></DIV><DIV>Lisa</DIV></BODY></HTML>=

--Apple-Mail-8--344127855--

Date: Mon, 02 Apr 2007 08:03:17 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: RFC2821bis-01 Issue 3: EHLO parameter
Sender: [email-masked]

--On Sunday, 01 April, 2007 14:26 -0700 Dave Crocker
<[email-masked]> wrote:

> John C Klensin wrote:
>>> Given the complexities, controversies, and resulting
>>> instability of community consensus about abuse-related
>>> reasons for rejecting mail, anything that touches that space
>>> should be omitted from the document.
>>> 
>>> Drop the text.
>> 
>> For the record, as editor, I'll do whatever consensus emerges
>> that I should do.
> 
> mais oui.
> 
> However, I hope that folks keep in mind that reality tends to
> override working group consensus.

As long as we can agree on what the reality is, of course.

>> However...  The specific name != address prohibition
>> originated, I believe, with 1123 after significant
>> discussion.  It was discussed again in DRUMS and reaffirmed.
>> 
> 
> Even as recently as DRUMS, the world was a very different
> place.  And remember that the exercise of 1123 was to
> synchronize with reality.
> 
> And that's all I am suggesting here, but by a minimizing
> approach, rather than trying to create new language.

There isn't much "creating new language" implied in anything
I've seen and understood.  There have been some suggestions
about changing a MUST to a SHOULD, which is certainly not new
language; Alex's recent suggestion to insert an explicit "other
rejections are not prohibited" which might be supplemented by a
forward pointer, which would normally fall into the category of
editorial smoothing and clarification rather than substantive
new language; and suggestions to somewhat tune the language of
7.8 (the "... may refuse to accept mail ...reason that makes
sense" material), again, something we would normally consider an
editorial matter rather than "creating new language".

Note also that removing text that provides very strong guidance
that a particular test is not especially useful is not, IMO, a
"minimizing approach", unless your definition of minimizing is
the trivial one of simply reducing the word count.  In all that
has changed in the last several years, I've heard no suggestions
that the value of this test _as a criterion for rejecting a
message_ has significantly increased.  I note that neither the
comments about that test, nor anything else in 2821 or 2821bis,
affect criteria for quarantining an message or assigning a
positive or negative statistical probability as to whether it is
wanted or unwanted traffic or whether it falls into some
particular category.  Under the "deliver or reject" model that
has been a property of Internet mail since the FTP days, those
classificatory or folder routing options all lie
post-final-SMTP-delivery.  
 
> There are many policy-related issues that rfc2821/821 could
> have chosen to make normative statements about, but didn't.
> That it chose to make a normative comment about the action to
> take, or not take, regarding a particularly validation
> exercise, steps into territory that was reasonable at the
> time, but now sits rather wildly in the middle of a very
> message, unstable, very energized/polarized topic.

The prohibition went into 1123 because, for a number of reasons
related to how the Internet worked, this was an expensive test
and tricky test that was subject to false positives for
rejecting mail.  As discussed above, no evidence has been
offered for that having changed.  

If the statement has said "this test is so bad that one should
not even use it to try to classify a message, or to contribute
to a score as to whether it is likely evil" then, certainly, it
would be desirable to remove it.  But it doesn't say that.
While it doesn't go into this much detail (maybe it should, but
that would require "creating new language") that simply trying
to forward-map DNS names and compare IP addresses is not a
useful test to enable message rejection.   In a way, that advice
is more useful today than it was when 2821 was completed.
Consider the separate discussion we have been having about IPv6.
Now, remembering that it has become common for administrative
domains to use different servers for sending and receiving mail,
suppose that such a domain has DNS records as follows:

    smtpout.example.com. AAAA ....
    example.com.         MX 0 smtpin.example.com.
    stmpin.example.com.  A  ....
                         AAAA ....

suppose also that the network that supports these hosts has a
network layer IPv6-IPv4 conversion gateway, a gateway that makes
its conversions at the packet layer, invisible to SMTP.  

Now assume a mail transaction that looks something like:
    EHLO smtpout.example.com
    MAIL FROM:<[email-masked]>
    RCPT TO:<[email-masked]>
    ...
and assume the simple case (the common one today): there are no
relays involved and the mail server for bogus.domain.name
doesn't support IPv6 even to the extent of looking up AAAA
records.  Assume that both the forward and reverse-pointing
addresses given are deliverable without difficulties.  Now what
happens:

The receiving server does a DNS lookup, type=A, on
"smtpout.example.com".  It fails.  Do you want to _reject_ this
message or encourage anyone to do so based on that information?
Or suppose it does do the type=AAAA lookup as well as the type=A
one.  It gets back the IPv6 record but, because of the
packet-level address remapping gateway, that address is not the
same as the address of the TCP connection to the SMTP port.   Do
you want to _reject_ the message on that basis?

Bad idea, IMO.  Probably a worse idea than it was in the simpler
world of 2000 (RFC 2821) or 1989 (RFC 1123).

Do you even want to mark it "suspicious" before delivering it?
Fortunately, 2821bis takes no position at all on that subject.

> It need not be anywhere near that territory, in order to do
> its basic job.
> 
> So I'll merely stress for the group:
> 
>       As folks consider what to do with the text, they should
> a) make sure it creates no new violence in the document, of
> course, and b) reflects reality by virture of matching
> existing practise.
> 
>       That's why I suggest completely dropping the text,
> rather than trying to tweak the normative status.

I think I agree with your principle, but not with the
application to this text, for the reasons above.

Two added comments:

(1) It does seem to me that it might be useful to add, possibly
to the Security Considerations section, a comment to the effect
that the tests required or prohibited by SMTP in order to
determine whether to reject or deliver mail and questions about
how far to push the robustness principle in terms of what to
accept and whether deviations that are still considered
deliverable should be evaluated in message classification, are
outside the scope of 2821bis.   That would, of course, require
making up new text, so it would be useful to think about whether
such text would be welcome before we start thinking about how to
craft it.

(2) It seems to me that the text of 7.8 should be clarified to
include something like "unless otherwise constrained above or by
extensions that are in use" to eliminate any possible confusion
about apparent contradictions and that this should be done
regardless of what changes are made, if any, to the specific
EHLO text.  Are there objections to my doing that, in spite of
the fact that it is "new text" and adds to the word count?

     john

Date: Mon, 2 Apr 2007 00:03:07 -0400
To: Frank Ellermann <[email-masked]>
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: More references in 2821bis
Cc: [email-masked]
Sender: [email-masked]

At 13:44 +0200 on 04/01/2007, Frank Ellermann wrote about Re: More 
references in 2821bis:

>[[ Oops, "Apparenly-To" is already mentioned in 7.2, I've added it
>    to <http://permalink.gmane.org/gmane.ietf.message-headers/37> ]]

You show the text as "This header field was never formally standardized, it
was used by a mailer in absence of a To: header field derived from 
the envelope RCPT TO addresses, thereby disclosing addresses of blind 
carbon copies."

While the lack of To: Header is accurate, I seem to remember that it 
ALSO needed the absence of a Cc header to trigger this injected 
listing. IOW: So long as SOME address (even only a comment in parens) 
was supplied in EITHER the To or Cc header it would not expose the 
Bcc (or MAIL-TO only) addresses. Someone please confirm or deny this 
behavior.

Date: Mon, 2 Apr 2007 03:25:03 +0200
From: Alex van den Bogaerdt <[email-masked]>
To: [email-masked]
Subject: Re: RFC2821bis-01 Issue 3: EHLO parameter
Reply-To: [email-masked]
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Sun, Apr 01, 2007 at 02:26:24PM -0700, Dave Crocker wrote:

>      As folks consider what to do with the text, they should a) make sure 
>      it creates no new violence in the document, of course, and b) reflects 
> reality by virture of matching existing practise.

In any case, the current text in 2821 is misinterpreted by some (many?)
as a sign that it is not okay to reject on bad hello.

>      That's why I suggest completely dropping the text, rather than trying 
>      to tweak the normative status.

Dropping the text would mean a change.  I am not against this (far from it).
If the MUST NOT should stay, at least use different wording so that it is
clear to all that the text only forbids one particular case.

2821bis (I've been looking at draft-klensin-rfc2821bis-01) is already 
much better, but I suggest adding a bit of text:

s/address only; s/address only; other rejections are not prohibited! S/

my 2ct
Alex

Date: Sun, 01 Apr 2007 14:26:24 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: RFC2821bis-01 Issue 3: EHLO parameter
Sender: [email-masked]

John C Klensin wrote:
>> Given the complexities, controversies, and resulting
>> instability of community consensus about abuse-related reasons
>> for rejecting mail, anything that touches that space should be
>> omitted from the document.
>>
>> Drop the text.
> 
> For the record, as editor, I'll do whatever consensus emerges
> that I should do.

mais oui.

However, I hope that folks keep in mind that reality tends to override working 
group consensus.

> However...  The specific name != address prohibition originated,
> I believe, with 1123 after significant discussion.  It was
> discussed again in DRUMS and reaffirmed.   

Even as recently as DRUMS, the world was a very different place.  And remember 
that the exercise of 1123 was to synchronize with reality.

And that's all I am suggesting here, but by a minimizing approach, rather than 
trying to create new language.

There are many policy-related issues that rfc2821/821 could have chosen to 
make normative statements about, but didn't.  That it chose to make a 
normative comment about the action to take, or not take, regarding a 
particularly validation exercise, steps into territory that was reasonable at 
the time, but now sits rather wildly in the middle of a very message, 
unstable, very energized/polarized topic.

It need not be anywhere near that territory, in order to do its basic job.

So I'll merely stress for the group:

      As folks consider what to do with the text, they should a) make sure it 
creates no new violence in the document, of course, and b) reflects reality by 
virture of matching existing practise.

      That's why I suggest completely dropping the text, rather than trying to 
tweak the normative status.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: More references in 2821bis
Date:  Sun, 01 Apr 2007 20:15:44 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 92
Sender: [email-masked]

John C Klensin wrote:

 [PCMAIL and IMAPv2]
> part of the point here is that it was those older protocols
> that marked the change being discussed, not the newer,
> consolidated, IMAP and the newer POP which is somewhat
> bloated relative to historical assumptions about its role.

> More generally, you are making an assumption with which I
> disagree.  I believe that, when a document is being produced
> that updates or summarizes a well-established protocol,
> information about the sources of differences and evolution is
> important.   Putting things in context that was may not be
> optimal for a completely new, "white room", implementation of
> SMTP but it is often useful in reviewing an older implementation
> or collection of modules to understand what must be updated and
> why.

> You may, of course, disagree.

Not sure, depends on what 2821bis tries to be.  Is it "only" a
specification of ESMTP as is today, where interested readers can
(try to) reconstruct the history by following "obsoletes NNN" in
referenced RFCs ?

Or does it also try to explain how SMTP used to work in the past
starting with 821 and 822 ?  You certainly know that I proposed
to extract the history resulting in two smaller documents, so I'm
not against documenting history, quite the contrary.  And if you
prefer to do it in one document it's also fine, as a consequence
2821bis will be rather long.  I can manage (from a reader's POV).

BUT - if 2821bis is supposed to contain a history, then it should
also explain why the original concept of the <reverse-path>s was
removed, how that affected forwarding to third parties, how it
affected the use and later forgery of envelope sender addresses,
and ultimately killed the accept-and-bounce strategy as used in
the first decades of SMTP.  IOW the disaster we face today, it
would be at least somewhat different with RFC 821 reverse-paths.

> Our definitions of "easy" also differ.

Maybe.  In the case of references I've adopted Scott's strategy
to use something like [I-D.2234bis] instead of [RFC 2234] when
it's likely that the referenced I-D gets its RFC number _before_
the referencing I-D.  Of course 2440bis is not yet approved, and
you can as well delay such cleanups until AUTH48.  I'd probably
forget such nits, and with an I-D as reference it's more obvious
that there was something needing a last minute check.  If you've
a different strategy to watch such details the [2440] is okay.

> If 2440bis is approved, assigned an RFC number, and published
> before 2821bis is published, it would, of course, be appropriate
> to update the reference, but it is not, IMO, appropriate to do
> so on the basis of an I-D.

I only happened to see that the IESG review for 2440bis started
on March 29, if you knew that already simply ignore my hint...

> point to an I-D instead of to RFC 2440 would be a cosmetic
> change and one that would move things in the wrong direction.

...it's not my idea to have references to _any_ I-D in the final
(published) 2821bis.  It's the SMTP specification.  It's one of
the most important IETF documents anybody could ever write.  It
is of utmost importance to get it right for readers today.

> If you still want to try to convince people that an upgrade to
> 2821 is not worth doing unless it can be completely restructured,
> I would personally recommend and request that you focus on that,
> rather than trying to gain that effect by exhaustion.

So far I focus on fixing bugs, ambiguities, unclear terminology,
and anything I might stumble over while reading the I-D looking
for something else.  There are lots of places were 2821 silently
assumed that it's always possible to report problems later, and
we know that this is generally not more true today.  It's not
impossible to identify such issues, and make them more obvious
in the specification.  Of course that's exhausting, but you did
not like my proposals to cut this work into smaller pieces.

>> <http://permalink.gmane.org/gmane.ietf.smtp/5395>
> These have been fixed (although some of the resulting
> constructions seem clumsy)

Thanks.  If the wording is clumsy I think you can always replace
the verb "bounce" by either "reject" or "return", and the noun
by either "reply code" or "non-delivery report" - it shouldn't
be too ugly, 821, 974, 1123, and 1869 don't use "bounce" at all.

Frank

From: [email-masked]
Cc: Frank Ellermann <[email-masked]>, [email-masked]
Date: Sun, 01 Apr 2007 07:57:09 -0700 (PDT)
Subject: Re: More references in 2821bis
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Sunday, 01 April, 2007 13:44 +0200 Frank Ellermann
> <[email-masked]> wrote:

> >
> > John C Klensin wrote:
> >
> >> [13] (RFC 1176, the original IMAPv2 spec) is used as part of a
> >> comment about historical development, for which [14] (RFC
> >> 3501, IMAPv4r1) would just be inappropriate.
> >
> > Okay, those are intentional references to obsolete RFCs:
> >
> >  The problems associated with ill-formed messages were
> > exacerbated by  the introduction of the split-UA mail reading
> > protocols (RFC937 [11],  RFC1939 [31], RFC1176 [13], RFC1056
> > [27]).
> >
> > How about writing something like this:
> >
> >  The problems associated with ill-formed messages were
> > exacerbated by  the introduction of the split-UA mail reading
> > protocols like POP and  IMAP (now POP3 [31] and IMAPv4r1 [14]).
> >
> > I haven't heard of PCMAIL before, apparently a kind of BBS.  I
> > fear that's not more relevant today.  With the proposed
> > wording you could remove the historical [11], [13], and [27]
> > from the references.

> Actually, PCMAIL was the original "disconnected mode" protocol.
> We had POP (strictly offline mode -- messages downloaded from
> server and deleted), PCMAIL (disconnected mode -- messages kept
> on server but cached on client), and IMAP (online or kiosk
> mode).  With IMAPv4, the other two modes were added to IMAP and,
> depending on the client and server, more or less well
> implemented.  And part of the point here is that it was those
> older protocols that marked the change being discussed, not the
> newer, consolidated, IMAP and the newer POP which is somewhat
> bloated relative to historical assumptions about its role.

> More generally, you are making an assumption with which I
> disagree.  I believe that, when a document is being produced
> that updates or summarizes a well-established protocol,
> information about the sources of differences and evolution is
> important.   Putting things in context that was may not be
> optimal for a completely new, "white room", implementation of
> SMTP but it is often useful in reviewing an older implementation
> or collection of modules to understand what must be updated and
> why.

> You may, of course, disagree.  If you do, you need to convince
> Tony (and others) that pulling the contextual text out is
> substantive and valuable and not just a cosmetic change that
> carries some risk.

I am strongly supportive of John in this. I will also observe that while a
"white room" implementation strategy is marginally less well served by having
such context in the document, implementations produced by such strategies tend
not to be the best when it comes to interoperability. Like it or not, context
is useful in getting things to work, especially when dealing with corner cases.

> >> tampering with the references is not completely
> >> straightforward...
> >
> > Yes. but in some cases it's easy, e.g. for [12] you'd need the
> > new draft-ietf-openpgp-rfc2440bis when it survived the IESG
> > evaluation, and if you keep it in chapter 7.1 -- I'd like to
> > defer "chapter 7.1" issues for some I-Ds, rewriting it from
> > scratch later (separately).

> Our definitions of "easy" also differ.  As far as I know, the
> normative references to documents that have been superceded were
> all updated before -01 was posted.  By referencing RFC2440 in
> [12], there is a reference to a published RFC.  To change that
> to a reference to an I-D --an I-D that the IESG has not yet
> signed off on-- raises all sorts of problems about people being
> able to actually find the relevant document.  If the reference
> to 2440 is retained, then anyone who looks at the RFC index will
> find out about updates when it is published and anyone who does
> not will get, from 2440, a general idea about what is going
> on... and the latter is all the informative reference is about.
> If 2440bis is approved, assigned an RFC number, and published
> before 2821bis is published, it would, of course, be appropriate
> to update the reference, but it is not, IMO, appropriate to do
> so on the basis of an I-D.

> Again, you may disagree and should certainly try to convince
> people if you do, but I personally believe that changing [12] to
> point to an I-D instead of to RFC 2440 would be a cosmetic
> change and one that would move things in the wrong direction.

This one's a nobrainer as far as I'm concerned. Any reference to a draft either
creates a dependency that has to be resolved before publication or, if it's
allowed in the final document, due to RFC Editor rules ends up creating a
pointer that is difficult for people to resolve. (I happen to disagree with the
way the RFC Editor handles these, but the process is what it is.)

> > [[ Oops, "Apparenly-To" is already mentioned in 7.2, I've
> > added it    to
> > <http://permalink.gmane.org/gmane.ietf.message-headers/37> ]]
> >
> >> If we were trying to do a more significant revision, I think
> >> I'd try to reduce the number of such references, rather than
> >> increasing them.
> >
> > That "If" belongs to the TINW.

> Those are the rules under which we got rfc2821bis out of the
> "indefinite hold/ there isn't sufficient energy to deal with
> this" state.  If you still want to try to convince people that
> an upgrade to 2821 is not worth doing unless it can be
> completely restructured, I would personally recommend and
> request that you focus on that, rather than trying to gain that
> effect by exhaustion.

Yep.

> > 2119-keywords and references in an ABNF comment are again a
> > matter of taste, I'd convert it to prose.  What does "Attdl"
> > stand for ?

> Your preferences is noted and classified as cosmetic unless
> others feel strongly about this.

Editor's choice as far as I'm concerned.

				Ned

Date: Sun, 01 Apr 2007 09:52:56 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: More references in 2821bis
Sender: [email-masked]

--On Sunday, 01 April, 2007 13:44 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> John C Klensin wrote:
>  
>> [13] (RFC 1176, the original IMAPv2 spec) is used as part of a
>> comment about historical development, for which [14] (RFC
>> 3501, IMAPv4r1) would just be inappropriate.
> 
> Okay, those are intentional references to obsolete RFCs:
> 
>  The problems associated with ill-formed messages were
> exacerbated by  the introduction of the split-UA mail reading
> protocols (RFC937 [11],  RFC1939 [31], RFC1176 [13], RFC1056
> [27]).
> 
> How about writing something like this:
> 
>  The problems associated with ill-formed messages were
> exacerbated by  the introduction of the split-UA mail reading
> protocols like POP and  IMAP (now POP3 [31] and IMAPv4r1 [14]).
> 
> I haven't heard of PCMAIL before, apparently a kind of BBS.  I
> fear that's not more relevant today.  With the proposed
> wording you could remove the historical [11], [13], and [27]
> from the references.

Actually, PCMAIL was the original "disconnected mode" protocol.
We had POP (strictly offline mode -- messages downloaded from
server and deleted), PCMAIL (disconnected mode -- messages kept
on server but cached on client), and IMAP (online or kiosk
mode).  With IMAPv4, the other two modes were added to IMAP and,
depending on the client and server, more or less well
implemented.  And part of the point here is that it was those
older protocols that marked the change being discussed, not the
newer, consolidated, IMAP and the newer POP which is somewhat
bloated relative to historical assumptions about its role.

More generally, you are making an assumption with which I
disagree.  I believe that, when a document is being produced
that updates or summarizes a well-established protocol,
information about the sources of differences and evolution is
important.   Putting things in context that was may not be
optimal for a completely new, "white room", implementation of
SMTP but it is often useful in reviewing an older implementation
or collection of modules to understand what must be updated and
why.

You may, of course, disagree.  If you do, you need to convince
Tony (and others) that pulling the contextual text out is
substantive and valuable and not just a cosmetic change that
carries some risk.

>> tampering with the references is not completely
>> straightforward... 
> 
> Yes. but in some cases it's easy, e.g. for [12] you'd need the
> new draft-ietf-openpgp-rfc2440bis when it survived the IESG
> evaluation, and if you keep it in chapter 7.1 -- I'd like to
> defer "chapter 7.1" issues for some I-Ds, rewriting it from
> scratch later (separately).

Our definitions of "easy" also differ.  As far as I know, the
normative references to documents that have been superceded were
all updated before -01 was posted.  By referencing RFC2440 in
[12], there is a reference to a published RFC.  To change that
to a reference to an I-D --an I-D that the IESG has not yet
signed off on-- raises all sorts of problems about people being
able to actually find the relevant document.  If the reference
to 2440 is retained, then anyone who looks at the RFC index will
find out about updates when it is published and anyone who does
not will get, from 2440, a general idea about what is going
on... and the latter is all the informative reference is about.
If 2440bis is approved, assigned an RFC number, and published
before 2821bis is published, it would, of course, be appropriate
to update the reference, but it is not, IMO, appropriate to do
so on the basis of an I-D.

Again, you may disagree and should certainly try to convince
people if you do, but I personally believe that changing [12] to
point to an I-D instead of to RFC 2440 would be a cosmetic
change and one that would move things in the wrong direction.

> [[ Oops, "Apparenly-To" is already mentioned in 7.2, I've
> added it    to
> <http://permalink.gmane.org/gmane.ietf.message-headers/37> ]]
> 
>> If we were trying to do a more significant revision, I think
>> I'd try to reduce the number of such references, rather than
>> increasing them.
> 
> That "If" belongs to the TINW.

Those are the rules under which we got rfc2821bis out of the
"indefinite hold/ there isn't sufficient energy to deal with
this" state.  If you still want to try to convince people that
an upgrade to 2821 is not worth doing unless it can be
completely restructured, I would personally recommend and
request that you focus on that, rather than trying to gain that
effect by exhaustion.

>  Some concepts are stated more
> than once, "tuning" them at one place could be inconsistent
> with their use in another place.  For an example see three
> cases of the old "bounce := reject" terminology in chapters
> 2.4, 3.4, and 3.5.3, inconsistent with the revised terminology
> in chapters 61. and 6.2:
> <http://permalink.gmane.org/gmane.ietf.smtp/5395>

These have been fixed (although some of the resulting
constructions seem clumsy), but this risk of catching some
places and not others is precisely why I was reluctant to try to
change that terminology.

>> I'm personally disinclined to start down the path of trying to
>> significantly upgrade, or even fine-tune, the external
>> references (although, if people really want it and supply
>> specifics, I'll try to find time).
> 
> For the POP and IMAP cases I think it's better to use the names
> instead of obsolete RFC numbers, with references to the actual
> RFCs, but that's mainly a matter of taste and editorial issue.

yes.  and see above.

> For the "missing" 3848 link there's a clear place _where_ I
> miss it, in section 4.4, in the ABNF comment for
> <Attdl-Protocol>:
> 
> -    ; Additional standard names for protocols are
> -    ; registered with the Internet Assigned Numbers
> -    ; Authority (IANA) in the "mail parameters"
> -    ; registry.  SMTP servers SHOULD NOT use
> -    ; unregistered names.
> +    ; Additional standard names for protocols are
> +    ; registered in the "mail parameters" registry
> +    ; as specified in [RFC 3848].  SMTP servers 
> +    ; SHOULD NOT use unregistered names.

Partially to demonstrate the advantages of supplying specific
context and textual suggestions, this reference has been added.
If anyone objects, say so.

> 2119-keywords and references in an ABNF comment are again a
> matter of taste, I'd convert it to prose.  What does "Attdl"
> stand for ?

Your preferences is noted and classified as cosmetic unless
others feel strongly about this.

It stands for "Additional"

     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: More references in 2821bis
Date:  Sun, 01 Apr 2007 13:44:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 71
Sender: [email-masked]

John C Klensin wrote:
 
> [13] (RFC 1176, the original IMAPv2 spec) is used as part of a
> comment about historical development, for which [14] (RFC 3501,
> IMAPv4r1) would just be inappropriate.

Okay, those are intentional references to obsolete RFCs:

 The problems associated with ill-formed messages were exacerbated by
 the introduction of the split-UA mail reading protocols (RFC937 [11],
 RFC1939 [31], RFC1176 [13], RFC1056 [27]).

How about writing something like this:

 The problems associated with ill-formed messages were exacerbated by
 the introduction of the split-UA mail reading protocols like POP and
 IMAP (now POP3 [31] and IMAPv4r1 [14]).

I haven't heard of PCMAIL before, apparently a kind of BBS.  I fear
that's not more relevant today.  With the proposed wording you could
remove the historical [11], [13], and [27] from the references.

> tampering with the references is not completely straightforward... 

Yes. but in some cases it's easy, e.g. for [12] you'd need the new
draft-ietf-openpgp-rfc2440bis when it survived the IESG evaluation,
and if you keep it in chapter 7.1 -- I'd like to defer "chapter 7.1"
issues for some I-Ds, rewriting it from scratch later (separately).

[[ Oops, "Apparenly-To" is already mentioned in 7.2, I've added it
   to <http://permalink.gmane.org/gmane.ietf.message-headers/37> ]]

> If we were trying to do a more significant revision, I think I'd
> try to reduce the number of such references, rather than
> increasing them.

That "If" belongs to the TINW.  Some concepts are stated more than
once, "tuning" them at one place could be inconsistent with their
use in another place.  For an example see three cases of the old
"bounce := reject" terminology in chapters 2.4, 3.4, and 3.5.3,
inconsistent with the revised terminology in chapters 61. and 6.2:
<http://permalink.gmane.org/gmane.ietf.smtp/5395>

> I'm personally disinclined to start down the path of trying to
> significantly upgrade, or even fine-tune, the external
> references (although, if people really want it and supply
> specifics, I'll try to find time).

For the POP and IMAP cases I think it's better to use the names
instead of obsolete RFC numbers, with references to the actual
RFCs, but that's mainly a matter of taste and editorial issue.

For the "missing" 3848 link there's a clear place _where_ I miss
it, in section 4.4, in the ABNF comment for <Attdl-Protocol>:

-    ; Additional standard names for protocols are
-    ; registered with the Internet Assigned Numbers
-    ; Authority (IANA) in the "mail parameters"
-    ; registry.  SMTP servers SHOULD NOT use
-    ; unregistered names.
+    ; Additional standard names for protocols are
+    ; registered in the "mail parameters" registry
+    ; as specified in [RFC 3848].  SMTP servers 
+    ; SHOULD NOT use unregistered names.

2119-keywords and references in an ABNF comment are again a
matter of taste, I'd convert it to prose.  What does "Attdl"
stand for ?

Frank