Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
From: Keith Moore <[email-masked]>
To: [email-masked] (Andrew Church)
Cc: [email-masked], [email-masked], [email-masked]
Date: Fri, 30 Aug 2002 02:17:08 -0400
List-Id: <ietf-smtp.imc.org>

> >and no, it's not necessary to determine the real sender's identity, and this
> >might not even be desirable.   an alternative proposal is forthcoming.
> 
>      When you have such a proposal I'd be interested in reading it.  Until
> then, I'm going to continue with mine.

of course.

actually I think there are some interesting ideas in your proposal
which might dovetail well with some of the ideas in mine. 

Keith

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Fri, 30 Aug 2002 15:10:21 JST
Sender: [email-masked]

>and no, it's not necessary to determine the real sender's identity, and this
>might not even be desirable.   an alternative proposal is forthcoming.

     When you have such a proposal I'd be interested in reading it.  Until
then, I'm going to continue with mine.

  --Andrew Church
    [email-masked]
    http://achurch.org/

From: Keith Moore <[email-masked]>
To: [email-masked] (Andrew Church)
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Fri, 30 Aug 2002 02:08:02 -0400
Sender: [email-masked]

> >and you really want return-path / MAIL FROM to point to the place where
> >the sender will get the nondelivery reports resulting from mail that he
> >sends.  (list mail excepted)
> 
>      This, I think, is inappropriate.  If you want a header for indicating
> where non-delivery reports should go, add one;

uh, there's been a standard header for that purpose for over 20 years.
it's called return-path.  and the DSN standards require that reports go there. 

> There needs to be _some_
> way to determine the real sender of a message, at least to domain level if
> not farther, 

it is probably a mistake to assume that the "real sender" has anything 
to do with either the From or Return-Path address, or perhaps, that the sender
can be identified with *any* email address.  (in fact this is what the Sender 
header was intended for, though it's been so widely misused that too late to 
save it, and there's a reason that Sender isn't required to have the syntax of 
an email address).  but email addresses are not distinguished names.  they can 
be shared, and Senders can have as many email addresses as they want.

and no, it's not necessary to determine the real sender's identity, and this
might not even be desirable.   an alternative proposal is forthcoming.

Keith

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Fri, 30 Aug 2002 14:20:27 JST
Sender: [email-masked]

>> I already have enough trouble with over-paranoid mail system admins.
>
>so do I.  which is why I don't want mail being filtered just because
>I happen to send mail from a dialup account (or my palm pilot)
>with my normal return address.

     Which is why I decided to go with a challenge/response instead of
hostnames.  Didn't we already cover this once?

>the mail standards specifically allow the sender to alter the From address,

     I'll accept that; even the draft points out that controlling From
probably isn't feasible, regardless of desirability.

>and you really want return-path / MAIL FROM to point to the place where 
>the sender will get the nondelivery reports resulting from mail that he
>sends.  (list mail excepted)

     This, I think, is inappropriate.  If you want a header for indicating
where non-delivery reports should go, add one; that also gives you the
ability to have errors go to more than one place (whether that's desirable
or not is a separate question).  CS researchers 25 years ago may have had
no problems with sending each other joke mails with fake addresses, but
that's not a reasonable attitude to take today.  There needs to be _some_
way to determine the real sender of a message, at least to domain level if
not farther, and I think it's pretty obvious that Received headers aren't
doing the trick.  I don't claim that my suggestion is a panacea either, but
I do think it's a good start.

  --Andrew Church
    [email-masked]
    http://achurch.org/

From: Keith Moore <[email-masked]>
To: [email-masked] (Andrew Church)
cc: [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Fri, 30 Aug 2002 01:03:28 -0400
Sender: [email-masked]

> I already have enough trouble with over-paranoid mail system admins.

so do I.  which is why I don't want mail being filtered just because
I happen to send mail from a dialup account (or my palm pilot)
with my normal return address.

the mail standards specifically allow the sender to alter the From address,
and you really want return-path / MAIL FROM to point to the place where 
the sender will get the nondelivery reports resulting from mail that he
sends.  (list mail excepted)

so I don't think it's reasonable to try to restrict which From addresses
or which return-path addresses can come from which MTAs.

Keith

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
Date: Fri, 30 Aug 2002 13:17:04 JST
Sender: [email-masked]

>Ah, I have got the idea. However, you might want to consider to discuss
>in your draft:
>
>(1) Section 1 is confusing since it is about "Forged Headers".
>
>(2) How serious is the problem of "Sender Address" compared to
>    the "Forged Headers"?

     These are the same problem.  "Forged headers" (okay, I admit it might
be clearer to say "forged sender addresses") refer to the sender creating
invalid data, not someone in the middle changing valid data to invalid
data.  (dictionary.com, from Webster's: forge, v.t. [...] 4. To make
falsely; to produce, as that which is untrue or not genuine; to fabricate;
to counterfit, as, a signature, or a signed document.)

>(3) Why use a complicated cryptographic challenge mechanism? AFAIK, it is 
>    quiet common for a current receiving MTA to reject emails that come 
>    from a host with no reverse "in-addr.arpa" information.

     Some valid MTAs (achurch.org, for example) don't have reverse DNS
mappings, and I already have enough trouble with over-paranoid mail system
admins.

>    why not use the MS RR to list the MTAs who are allowed to use the 
>    sender address of the domain?

     I think I went over this once, but this was the approach I took in
earlier versions, and I rejected it because the administrative burden
becomes significant with dynamic IP addresses and other cases in which
the IP address of an allowed MTA can change.

  --Andrew Church
    [email-masked]
    http://achurch.org/

Date: Fri, 30 Aug 2002 10:39:44 +0700
From: "Rahmat M. Samik-Ibrahim" <[email-masked]>
Organization: vLSM.org
To: Andrew Church <[email-masked]>
Cc: [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
Sender: [email-masked]

Ref: http://www.ietf.org/internet-drafts/draft-church-dns-mail-sender-02.txt

Andrew Church wrote:

>     "In cases where a message is forwarded through one or more MTAs before
>      reaching its final destination [...], the MTA at the message's
>      destination will typically reject the message when the forwarding MTA
>      fails to respond properly to an MS challenge for the sender's domain.
>      Thus, MTAs which forward messages should modify the sender address of
>      such messages to one which they can respond to an MS challenge for.
>      However, depending on circumstances, it may be appropriate for the MTA
>      to record elsewhere in the message the original sender address."

Ah, I have got the idea. However, you might want to consider to discuss
in your draft:

(1) Section 1 is confusing since it is about "Forged Headers".

(2) How serious is the problem of "Sender Address" compared to
    the "Forged Headers"?

(3) Why use a complicated cryptographic challenge mechanism? AFAIK, it is 
    quiet common for a current receiving MTA to reject emails that come 
    from a host with no reverse "in-addr.arpa" information. Furthermore, 
    why not use the MS RR to list the MTAs who are allowed to use the 
    sender address of the domain?
    
regards,

-- 
Rahmat M.  Samik-Ibrahim -- vLSM.org  -- http://rms46.vLSM.org --
One Debian GNU/Linux,with glibc,and justice for all - Pledge of A

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
Date: Thu, 29 Aug 2002 16:21:45 JST
Sender: [email-masked]

>>      As I said before in private mail, my goal is not to avoid modification
>> of the headers en route (though I agree that's not a bad idea), but to
>> ensure that the original sender doesn't give a false sender address.
>
>http://www.ietf.org/internet-drafts/draft-church-dns-mail-sender-02.txt
>
>In section 3.2: 
>   "which the sending MTA would encrypt with its private key;"
>
>In section 3.4: 
>   "as long as the relay MTA can properly respond to an MS challenge for 
>    the domain, the sender address will be accepted." 
>
>May I know, how a proper relay MTA (e.g. mailing list, bounce, forward)
>is able answer a MS challenge without having access to the private key?

     It can't, thus it will need to rewrite the envelope sender.  (As Keith
Moore pointed out earlier, mailing lists are expected to do this anyway,
and IMO this is proper behavior for forward/bounce as well.)  This is
discussed more thoroughly in revision 03 (currently being edited):

    "In cases where a message is forwarded through one or more MTAs before
     reaching its final destination [...], the MTA at the message's
     destination will typically reject the message when the forwarding MTA
     fails to respond properly to an MS challenge for the sender's domain.
     Thus, MTAs which forward messages should modify the sender address of
     such messages to one which they can respond to an MS challenge for.
     However, depending on circumstances, it may be appropriate for the MTA
     to record elsewhere in the message the original sender address."

  --Andrew Church
    [email-masked]
    http://achurch.org/

Date: Thu, 29 Aug 2002 10:57:37 +0700
From: "Rahmat M. Samik-Ibrahim" <[email-masked]>
Organization: vLSM.org
To: Andrew Church <[email-masked]>
Cc: [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
Sender: [email-masked]

Hello:

Thank you for your reply.

Andrew Church wrote:

>      As I said before in private mail, my goal is not to avoid modification
> of the headers en route (though I agree that's not a bad idea), but to
> ensure that the original sender doesn't give a false sender address.

http://www.ietf.org/internet-drafts/draft-church-dns-mail-sender-02.txt

In section 3.2: 
   "which the sending MTA would encrypt with its private key;"

In section 3.4: 
   "as long as the relay MTA can properly respond to an MS challenge for 
    the domain, the sender address will be accepted." 

May I know, how a proper relay MTA (e.g. mailing list, bounce, forward)
is able answer a MS challenge without having access to the private key?

My interest is actually not about "avoiding modification", but about
how to prevent abuses of a domain address. I believe, that there is a 
need for a STUB to be included in an email message.

regards,

-- 
Rahmat M.  Samik-Ibrahim -- vLSM.org  -- http://rms46.vLSM.org --
Wise men say:Can't Help Falling in Linux with GNU - LILO and GRUB

From: Keith Moore <[email-masked]>
To: [email-masked] (Andrew Church)
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Tue, 27 Aug 2002 10:38:58 -0400
Sender: [email-masked]

> >> >1. it is not so clear; on how to implement a mailing list exploder; which
> >> >   is neither (cmiiw) "3.3. Relays on the receiving end" nor
> >> >   "3.4. Relays on the sending end".
> >> 
> >>      Current mailing list software (at least as far as I'm aware) resends
> >> the message from the user and domain the software runs as/in; 
> >
> >no, that's not the case.  actually domains are not topological at all;
> >so there's no such thing as the "domain the software runs as/in".
> 
>      Perhaps my wording wasn't clear; what I meant to say was, the envelope
> sender (MAIL FROM: parameter) of the exploded message will be set by the
> mailing list software to an address under its own control.  For example, I
> would expect messages received from a list such as "[email-masked]" to
> have a Return-Path of "<something>@imc.org" (or possibly
> "<something>@somehost.imc.org", etc.).  Aside from simple mail aliases,
> most mailing list software I'm aware of does this, if nothing else to make
> bounce checking easier.  And in cases where this isn't true, frankly I
> think it ought to be.  

oh, okay.  yes, lists are expected to set mail from/return-path to an address
that can handle bounces - though strictly speaking it doesn't have to indicate
anything about where the list traffic "comes from".

Keith

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Tue, 27 Aug 2002 23:11:53 JST
Sender: [email-masked]

>> >1. it is not so clear; on how to implement a mailing list exploder; which
>> >   is neither (cmiiw) "3.3. Relays on the receiving end" nor
>> >   "3.4. Relays on the sending end".
>> 
>>      Current mailing list software (at least as far as I'm aware) resends
>> the message from the user and domain the software runs as/in; 
>
>no, that's not the case.  actually domains are not topological at all;
>so there's no such thing as the "domain the software runs as/in".

     Perhaps my wording wasn't clear; what I meant to say was, the envelope
sender (MAIL FROM: parameter) of the exploded message will be set by the
mailing list software to an address under its own control.  For example, I
would expect messages received from a list such as "[email-masked]" to
have a Return-Path of "<something>@imc.org" (or possibly
"<something>@somehost.imc.org", etc.).  Aside from simple mail aliases,
most mailing list software I'm aware of does this, if nothing else to make
bounce checking easier.  And in cases where this isn't true, frankly I
think it ought to be.  As the recipient, I ought to be able to find out
where my message was directly sent from; message sender information is
already available in the From: line.

  --Andrew Church
    [email-masked]
    http://achurch.org/

From: Keith Moore <[email-masked]>
To: [email-masked] (Andrew Church)
cc: [email-masked], [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt 
Date: Tue, 27 Aug 2002 08:57:49 -0400
Sender: [email-masked]

> >1. it is not so clear; on how to implement a mailing list exploder; which
> >   is neither (cmiiw) "3.3. Relays on the receiving end" nor
> >   "3.4. Relays on the sending end".
> 
>      Current mailing list software (at least as far as I'm aware) resends
> the message from the user and domain the software runs as/in; 

no, that's not the case.  actually domains are not topological at all;
so there's no such thing as the "domain the software runs as/in".

Keith

From: [email-masked] (Andrew Church)
To: [email-masked]
Cc: [email-masked]
Subject: Re: Comments (II) on draft-church-dns-mail-sender-02.txt
Date: Tue, 27 Aug 2002 18:51:14 JST
Sender: [email-masked]

>1. it is not so clear; on how to implement a mailing list exploder; which
>   is neither (cmiiw) "3.3. Relays on the receiving end" nor
>   "3.4. Relays on the sending end".

     Current mailing list software (at least as far as I'm aware) resends
the message from the user and domain the software runs as/in; this would
work without changes with MS.

>2. same as #1, for the standard ".forward" procedure.

     As #1, though in this case the original envelope sender would be lost.
If this is considered a problem, a 551 reply to the RCPT command can be
used.

>3. same as #1, for the "bounce" option of a mail reader (example pine or 
>   mutt);

     As #1.

>How about to verify the integrity of the email (checksum) to a server that 
>is pointed by the "MS" RR. I.e. not only to make sure that the headers 
>haven't been modified by anyone in the middle; but also as a confirmation 
>from the original sender (through MS RR).

     As I said before in private mail, my goal is not to avoid modification
of the headers en route (though I agree that's not a bad idea), but to
ensure that the original sender doesn't give a false sender address.

  --Andrew Church
    [email-masked]
    http://achurch.org/

Date: Tue, 27 Aug 2002 11:37:59 +0700
From: "Rahmat M. Samik-Ibrahim" <[email-masked]>
Organization: vLSM.org
To: [email-masked]
Cc: [email-masked]
Subject: Comments (II) on draft-church-dns-mail-sender-02.txt
Sender: [email-masked]

Hello:

Let me refine my comments on
http://www.ietf.org/internet-drafts/draft-church-dns-mail-sender-02.txt
I cc: this to ietf-smtp and apology if it is a wrong place to send.

I wrote:

1. it is not so clear; on how to implement a mailing list exploder; which
   is neither (cmiiw) "3.3. Relays on the receiving end" nor
   "3.4. Relays on the sending end".

2. same as #1, for the standard ".forward" procedure.

3. same as #1, for the "bounce" option of a mail reader (example pine or 
   mutt);

How about to verify the integrity of the email (checksum) to a server that 
is pointed by the "MS" RR. I.e. not only to make sure that the headers 
haven't been modified by anyone in the middle; but also as a confirmation 
from the original sender (through MS RR).

Example, sending an email from sender@FQDN1 (MTA1) to receiver@FQDN2 (MTA2):

- MTA1 at FQDN1 generates a STUB.

- A STUB consists of: the signed checksum of -- the email body; plus some 
  crucial email headers (eg. "From:", "Subject:", "Message-ID:", "Date:", 
  etc); plus its expire time. The whole STUB should be reasonable small 
  (say: 128-1024 bits?) so it could be processed by an ordinary CPU.

- When MTA1 sends the "MAIL FROM" stanza, it also includes the email STUB

- While still sending "250 OK", the receiver (MTA2) should query the 
  MS RR of FQDN1 to verify authentication of the STUB; 

- MTA2 verifies the integrity of the email.
  At the end, MTA2 should notify MTA1 if the message is accepted or rejected.

Drawback:

- The one who has got the STUB is able to spam the net. Therefore, 
  there should be a reasonable expire time of the STUB.

regards,

-- 
Rahmat M.  Samik-Ibrahim -- vLSM.org  -- http://rms46.vLSM.org --
Wise men say:Can't Help Falling in Linux with GNU - LILO and GRUB

From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: 2821 DATA timeouts
Date: Wed, 21 Aug 2002 19:18:33 +0530
Sender: [email-masked]

Now that I see my answer black on white, I see a decent answer to it, 
too: The phrasing is just another way of saying that an SMTP client 
SHOULD tolerate a network outage of up to three minutes. Which sounds 
reasonable.

--Arnt

From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: 2821 DATA timeouts
Date: Wed, 21 Aug 2002 18:58:05 +0530
Sender: [email-masked]

Hi,

RFC2821 states in section 4.5.3.2 that while the SMTP client is sending 
the messag body, it SHOULD accept up to a three-minute delay between 
each time it can send data.

With 1500-byte packets, that implies that an SMTP client SHOULD be happy 
with speeds as low as eight bytes per second (1500 bytes per three 
minutes), which strikes me as being a little on the low side... is that 
limit intentionally that low?

--Arnt

Date: Sat, 17 Aug 2002 21:08:19 -0500
From: "Eric A. Hall" <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 7:47 PM John C Klensin wrote:

> Eric, I don't disagree with this.  2821 doesn't disagree with
> this -- all it says is that "if you make the test, and it fails,
> don't reject the message for that reason".

You and I (and a couple of others) may agree that the lack of a
prohibition against subsequent policy decisions does not equate to a
prohibition against those decision. However, a quick reading through this
thread shows that there are people who have interpreted the existing text
as just that.

Therein lies the problem. Either you explicitly state it and resolve the
dilemma or you continue leaving the interpretation to the reader. My own
personal experience -- and the raison d'etre for this thread -- suggests
the latter has not worked particularly well when trying to convince people
that they are in fact allowed to filter on whatever they well want,
including the EHLO identifier.

At least I now have the editor of the RFC backing me up, which is good
enough for my purposes. Over and out.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Sat, 17 Aug 2002 20:47:46 -0400
From: John C Klensin <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

--On Saturday, 17 August, 2002 16:08 -0500 "Eric A. Hall"
<[email-masked]> wrote:

>...
> Are there valid reasons why an operator should not simply
> discard any lookup which results in a negative answer?
> Absolutely. There are also valid reasons for an operator
> within a specific network to do so. A closed server might want
> to reject any connection which cannot be validated through
> multiple means (including the EHLO identifier, the subsequent
> address lookup, and a tierterary comparison to an X.509
> subject), and a failure of any of these could be considered
> valid for that site. Most of us are just looking for something
> in between, such as refusing sessions from site claiming to be
> me but which I know are not me. Asking me to detail all of
> these is arbitrary to the point of seeming artificial.

Eric, I don't disagree with this.  2821 doesn't disagree with
this -- all it says is that "if you make the test, and it fails,
don't reject the message for that reason".  I even agree that
asking you to detail all of the motivations for refusing a
message would be arbitrary and artificial... and a terribly bad
idea at that.   But that is _exactly_ the problem with what you
are suggesting: it is arbitrary, artificial, and a bad idea to
single out the particular case that you are concerned about and
put text into 2821bis whose intent is to say "if you are unhappy
after looking at the identifier, whether you continue with the
SMTP session or not, and what else you might do, is up to you".
Similar text could be inserted, with at least equal
justification, with every SMTP command.  

E.g., suppose a delivery address in a RCPT command doesn't exist
on my system.  The language that is now present says that I
either generate a 5yz code in response to the RCPT command or I
accept it, and return an error message later.  But, if it does
exist, am I _required_ to accept the mail?  Of course not, and I
don't need to tell the sender why, either, although there are
some behaviors I'm strongly discouraged from exhibiting (like
closing the connection without warning).  That is what section
7.7 is all about.  Could we add prose to 4.1.4 that parallels
the text you propose for the EHLO argument case?  Of course we
could, but it doesn't strike me as either necessary or wise.

     john

Date: Sat, 17 Aug 2002 20:37:23 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

In the interest of reducing the number of nearly-identical
responses, I'm going to respond to several messages together...

--On Saturday, 17 August, 2002 14:43 -0500 "Eric A. Hall"
<[email-masked]> wrote:

>  An SMTP server MAY verify that the domain name parameter in
> the EHLO  command actually corresponds to the IP address of
> the client. If this  verification process is not performed, or
> if this process does not  complete successfully, the server
> SHOULD limit use of the identifier  to logging purposes only.
> If the verification process completes  without error (either
> in the positive or negative), the identifier  MAY be used for
> other purposes, as the server operator sees fit.

Eric, I think this makes text that is arguably already too
convoluted even worse.   In addition, it complicates section
7.7, which requires logging of this information in a Received
field if the message isn't rejected outright.  And, again, the
current text does not imply _any_ limitations on the use of the
domain name parameter, but _only_ on the results of trying to
match up the name with an address or vice versa.

--On Sunday, 18 August, 2002 00:04 +0300 Matti Aarnio
<[email-masked]> wrote:

> On Sat, Aug 17, 2002 at 04:33:44PM -0400, Keith Moore wrote:
>>  An SMTP server MAY verify that the domain name parameter in
>>  the EHLO command actually corresponds to the IP address of
>>  the client. 
>> 
>> the last thing we need is to give people more excuses to
>> implement stupid policies.
> 
> Indeed.  I would say there something like this:
> 
>   The experience with various desktop MUA softwares since
> early 1990es   shows that those generate all manners of
> variously faulty HELO/EHLO   parameter data making usefullness
> of the data testing highly   questionable.  Put the received
> parameter data into the trace header   (see n.n.n) along with
> literal IP address of the remote host.
>  
> (fill in the section number)

The section number is 4.4 in the existing RFC 2821 and this is
the first bullet.  It doesn't include the explanation -- in the
interest of not turning an 80-ish page document into a 200 page
one, few of those explanations are there.

--On Saturday, 17 August, 2002 15:39 -0500 "Eric A. Hall"
<[email-masked]> wrote:

>...
> The opportunity to be stupid comes in the text I added,
> although I will point out that it is a MAY not a SHOULD.
> Operators MAY be as dumb as they want to be.

Eric, section 7.7 of 2821, IMO, gives operators as much latitude
for intelligent (or stupid) filtering and other actions as they
could possibly need, without pointing out instances of
stupidity.  I have inserted a forward pointer to it into 4.1.4
of 2821bis on the theory that it can't be harmful (although note
my comment in an earlier message).  But I don't think that
filling up that document up with explicit endorsements of the
right to stupidity makes sense -- there are just too many
opportunities and the document is too long already.

Keith, the "you can do it, but don't reject on it" text is in
2821 because it was carried forward, with some rephrasing from
1123.  If I recall, the "use it only for logging" part was added
to make sure we were clear about what we were talking about
(although it obviously wasn't clear enough for Eric, who reads
into it all sorts of prohibitions that I don't see).  We could
just tear the whole business out, but the practice is (or was)
frequent enough, and seems dumb enough, that the warning/
restriction seems to be in order.

Now, by contrast, Abhijit suggested alternate text that, while
less convoluted, I think would get us into trouble:

--On Sunday, 18 August, 2002 01:45 +0530 Abhijit Menon-Sen
<[email-masked]> wrote:

>   However, the server MUST NOT refuse to accept a message
> because of a   verification failure: this information is for
> logging and tracing   only.

The difficulty is that rejecting on the basis of some _real_
verification failure strikes me as quite reasonable.  Part of
the point is that name(address(name-in-EHLO-argument)) isn't
good enough to be a "verification failure" -- it is just an
often-meaningless test.  Worse, if some test were used in
conjunction with real remote server authentication and failed,
one would almost certainly want to reject the message/
transaction.  So we would have to qualify this suggested
language to make it clear what was, and was not, a verification
failure.  I think that, by the time we got through with that,
we'd end up with text that was at least as convoluted as what is
there now, with little or no gain.

In response to a few other comments, I (for whatever my opinion
is worth), would favor a BCP document called "rational and
irrational things to do in an SMTP server to resist spammers" or
something to that effect.   As Ned and others have pointed out,
there are enough cases that such a document would be hard to
write, and it would get harder if one wanted to include only
techniques that the spammers couldn't work around once they
becamse part of a widely-circulated document.  But, if someone
wants to try...   You will all forgive me if I don't think
adding each and every one of those cases to 2821bis is a good
idea.

      john

Date: Sat, 17 Aug 2002 14:27:02 -0700 (PDT)
From: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
To: Keith Moore <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, [email-masked]
Sender: [email-masked]

> > We had a recent case where a large ISP was
> > being inundated with bogus messages. (I don't know if this was DOS attack or
> > merely spam.) The messages were coming from a variety of different addresses on
> > a variety of different networks. But the one thing they all had in common was
> > that they had the exact same domain listed as their EHLO argument. The ISP
> > actually went to the trouble of contacting the owner of this domain name and
> > was informed that the owner would under no circumstances actually use it as an
> > argument to EHLO.

> > Of course a block in such a case will only work for as long as it takes the
> > people sending the messages to realize what's going on and work around it. But
> > do you really think it is wrong to impose a temporary block of this one
> > argument value in this specific case?

> no.  and neither do I think that it's a good idea to categorically forbid
> implementations from providing hooks to do that - because of that rare
> specific case where it really is the right band-aid.

> at the same time, we recognize that there's a lot of ignorance out there, and
> some hooks are more easily misused than others.

Agreed. And this particular one is particularly easy to misuse.

> so what is the best way to discourage misuse of such hooks, or more generally,
> to discourage poorly-chosen filtering policies?

I think the current text actually does a pretty good job. It allows
verification but specifically prohibits using that particular result as a
filter. It says nothing that forbids filtering based on a specific string or
some other basis, however. I thought you were arguing for changing that, which
is why I spoke up.

I guess my position is that clarification of the text would be fine, but
changing what it says is allowed is not.

				Ned

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 17:21:57 -0400
Sender: [email-masked]

> We had a recent case where a large ISP was
> being inundated with bogus messages. (I don't know if this was DOS attack or
> merely spam.) The messages were coming from a variety of different addresses on
> a variety of different networks. But the one thing they all had in common was
> that they had the exact same domain listed as their EHLO argument. The ISP
> actually went to the trouble of contacting the owner of this domain name and
> was informed that the owner would under no circumstances actually use it as an
> argument to EHLO.
> 
> Of course a block in such a case will only work for as long as it takes the
> people sending the messages to realize what's going on and work around it. But
> do you really think it is wrong to impose a temporary block of this one
> argument value in this specific case?

no.  and neither do I think that it's a good idea to categorically forbid 
implementations from providing hooks to do that - because of that rare
specific case where it really is the right band-aid.

at the same time, we recognize that there's a lot of ignorance out there, and
some hooks are more easily misused than others.

so what is the best way to discourage misuse of such hooks, or more generally,
to discourage poorly-chosen filtering policies?

Keith

Date: Sat, 17 Aug 2002 13:51:31 -0700 (PDT)
From: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
To: Keith Moore <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> > EG, I'm not allowed to reject sessions from spew.spammer.com, even
> > if I know with certainty that the identifier is correct.

> it's not clear how you can know this for certain from EHLO.  you might
> know it for certain by other means, such as if you had a IPsec or TLS
> authenticated session.  the point is that the EHLO identifier (more precisely,
> the relationship between that identifier and the IP source address) really
> isn't reliable enough to be used as the basis of any kind of filter.

> to put it another way - nothing stops you from rejecting mail from
> any host you want.  but it's a violation of the protocol to use
> the EHLO identifier to decide whether the mail is from a host that you
> want to reject.  the only ways to "know with certainty" that this is
> from a particular host don't involve EHLO.

We have ample experience that says trying to match up the argument to EHLO with
IP address information in any way isn't sufficiently reliable. That's why
this is specifically prohibited. 

But I don't think this generalizes. We had a recent case where a large ISP was
being inundated with bogus messages. (I don't know if this was DOS attack or
merely spam.) The messages were coming from a variety of different addresses on
a variety of different networks. But the one thing they all had in common was
that they had the exact same domain listed as their EHLO argument. The ISP
actually went to the trouble of contacting the owner of this domain name and
was informed that the owner would under no circumstances actually use it as an
argument to EHLO.

Of course a block in such a case will only work for as long as it takes the
people sending the messages to realize what's going on and work around it. But
do you really think it is wrong to impose a temporary block of this one
argument value in this specific case?

				Ned

Date: Sat, 17 Aug 2002 16:08:32 -0500
From: "Eric A. Hall" <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 3:05 PM Arnt Gulbrandsen wrote:
> That's not sufficient. If you want that wording, you have to detail what 
> "postive" and "negative" mean.

Or remove it and leave it as "complete"

> You may also want to explain the implications, for example for SMTP 
> clients sitting behind NAT routers.

These demands strike me as being artificial.

Let me tell you a story. A few months back I made a typo while adding a
netblock range to my blacklist, and I ended up blocking something like 40
class A netblocks. By the arguments above, text permitting the use of
address ranges for policy decisions would need to be accompanied by "don't
make typos".

Are there valid reasons why an operator should not simply discard any
lookup which results in a negative answer? Absolutely. There are also
valid reasons for an operator within a specific network to do so. A closed
server might want to reject any connection which cannot be validated
through multiple means (including the EHLO identifier, the subsequent
address lookup, and a tierterary comparison to an X.509 subject), and a
failure of any of these could be considered valid for that site. Most of
us are just looking for something in between, such as refusing sessions
from site claiming to be me but which I know are not me. Asking me to
detail all of these is arbitrary to the point of seeming artificial.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: 	Sun, 18 Aug 2002 00:04:11 +0300
From: Matti Aarnio <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

On Sat, Aug 17, 2002 at 04:33:44PM -0400, Keith Moore wrote:
>  An SMTP server MAY verify that the domain name parameter in the EHLO
>  command actually corresponds to the IP address of the client. 
> 
> the last thing we need is to give people more excuses to implement
> stupid policies.

Indeed.  I would say there something like this:

  The experience with various desktop MUA softwares since early 1990es
  shows that those generate all manners of variously faulty HELO/EHLO
  parameter data making usefullness of the data testing highly
  questionable.  Put the received parameter data into the trace header
  (see n.n.n) along with literal IP address of the remote host.

(fill in the section number)

-- 
/Matti Aarnio	<[email-masked]>

From: Keith Moore <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: Keith Moore <[email-masked]>, John C Klensin <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 16:57:14 -0400
Sender: [email-masked]

> The excerpt you cited is extracted verbatim from current 2821 text.
> 
> The opportunity to be stupid comes in the text I added, although I will
> point out that it is a MAY not a SHOULD. Operators MAY be as dumb as they
> want to be.

not in the all caps sense, i.e. not as a matter of protocol compliance.  

while we've traditionally given a fair amount of latitude to operators
to reject messages for almost any reason, this is something that should 
probably be revisited in light of recent experience.  poorly chosen 
countermeasures for spam and viruses are now a huge problem.

we shouldn't pretend that we can stop operators from doing whatever they
want, but neither should we be encouraging implementors to give them hooks
to do stupid things - and in some cases we would do well to discourage 
specific hooks that have no practical use. 

checking EHLO for validity is just one of those - others include 
checking whether there is an A record for the domain in the return-path,
refusing to accept mail with a return-path of <>, and refusing to accept
mail with more than N recipients (as opposed to just rejecting those
recipients with a 4xx code).

Keith

Date: Sat, 17 Aug 2002 15:39:31 -0500
From: "Eric A. Hall" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 3:33 PM Keith Moore wrote:
>  An SMTP server MAY verify that the domain name parameter in the EHLO
>  command actually corresponds to the IP address of the client. 
> 
> the last thing we need is to give people more excuses to implement
> stupid policies.

The excerpt you cited is extracted verbatim from current 2821 text.

The opportunity to be stupid comes in the text I added, although I will
point out that it is a MAY not a SHOULD. Operators MAY be as dumb as they
want to be.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

From: Keith Moore <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 16:33:44 -0400
Sender: [email-masked]

 An SMTP server MAY verify that the domain name parameter in the EHLO
 command actually corresponds to the IP address of the client. 

the last thing we need is to give people more excuses to implement
stupid policies.

Date: Sun, 18 Aug 2002 01:45:21 +0530
From: Abhijit Menon-Sen <[email-masked]>
To: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

At 2002-08-17 14:43:38, [email-masked] wrote:
> 
>  An SMTP server MAY verify that the domain name parameter in the EHLO
>  command actually corresponds to the IP address of the client. If this
>  verification process is not performed, or if this process does not
>  complete successfully, the server SHOULD limit use of the identifier
>  to logging purposes only. If the verification process completes
>  without error (either in the positive or negative), the identifier
>  MAY be used for other purposes, as the server operator sees fit.

I think that's unnecessarily convoluted. The wording should be left as
it is; or, at most, changed to:

  However, the server MUST NOT refuse to accept a message because of a
  verification failure: this information is for logging and tracing
  only.

- ams

From: Arnt Gulbrandsen <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
Subject: Re: clarification re 2821, s4.1.4
Cc: John C Klensin <[email-masked]>, [email-masked]
Date: Sun, 18 Aug 2002 01:35:06 +0530
Sender: [email-masked]

That's not sufficient. If you want that wording, you have to detail what 
"postive" and "negative" mean.

You may also want to explain the implications, for example for SMTP 
clients sitting behind NAT routers.

--Arnt

Date: Sat, 17 Aug 2002 14:43:38 -0500
From: "Eric A. Hall" <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 2:27 PM John C Klensin wrote:

> If anyone has textual suggestions that would make this more
> clear and less convoluted, I'm listening.

Here:

 An SMTP server MAY verify that the domain name parameter in the EHLO
 command actually corresponds to the IP address of the client. If this
 verification process is not performed, or if this process does not
 complete successfully, the server SHOULD limit use of the identifier
 to logging purposes only. If the verification process completes
 without error (either in the positive or negative), the identifier
 MAY be used for other purposes, as the server operator sees fit.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

From: Arnt Gulbrandsen <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
Subject: Re: clarification re 2821, s4.1.4
Cc: Keith Moore <[email-masked]>, [email-masked]
Date: Sun, 18 Aug 2002 01:02:00 +0530
Sender: [email-masked]

What do you mean, information which is not in dispute?

In which cases does the HELO argument carry indisputable meaning? I 
wasn't aware of any.

--Arnt

Date: Sat, 17 Aug 2002 15:27:56 -0400
From: John C Klensin <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

--On Saturday, 17 August, 2002 13:04 -0500 "Eric A. Hall"
<[email-masked]> wrote:

> on 8/17/2002 12:54 PM Arnt Gulbrandsen wrote:
>> You most certainly are allowed to reject connections from 
>> www.spammer.com.
>> 
>> You can reject the connection because the SMTP client's IP
>> corresponds  to www.spammer.com. Perfectly legal by the RFC.
>> The HELO/EHLO argument  never enters the picture.
> 
> I'm aware of that, but it's not what I'm asking about. I'm
> specifically asking about using the EHLO identifier as a
> policy trigger. Under the interpretations of the section as
> provided here, this is prohibited.

--On Saturday, 17 August, 2002 11:53 -0500 "Eric A. Hall"
<[email-masked]> wrote:

> ps--according to the statements made here, I am not allowed to
> filter based on a HELO identifier which has verified in the
> positive form, either. EG, I'm not allowed to reject sessions
> from spew.spammer.com, even if I know with certainty that the
> identifier is correct.
> 
> So, clarification on allowable policy when verification
> succeeds is what I'm after, although frankly, I don't know why
> 2821 addresses policy issues at all.

Eric,

Speaking as the victim^H^H^H^H^H^H editor, this sort of stuff is
one of the things that caused DRUMS to take as long as it did
and that is causing motivation problems with a revision to 2821.

The sequence of events is that someone reads something into the
text that isn't there (usually omitted quite deliberately).
Then people comment on the subject, usually on the matter of why
the text is really quite specific and adequate, even if it could
be better written.  After the discussion goes on for a while,
and more misinterpretations are posted, it becomes clear that
the text should be clarified to eliminate any ambiguity.  So,
someone, usually the editor but ideally someone else, suggests a
patch and, often because everyone is tired of the discussion by
that time, the patch is accepted by general silence and
indifference.  It then goes in, and, by further complicating
things, creates an additional opportunity for readings that were
never intended.  I don't know how to fix that, and it is clearly
why we pay WG Chairs and editors the big bucks, but...

To repeat what others have said, the text you cite, as it now
appears in the middle of section 4.1.4, does not prevent you
from  looking at the argument to EHLO (or HELO) and immediately
generating a 550 response.  It is addressed to one case only,
which is your mapping a FQDN you find there to an IP address (or
determining the IP address by some other mechanism), then
determining whether the IP address is that of the proported
client, and then rejecting the message if it does not, _because_
it does not.  And, as others have pointed out, the main reason
for that prohibition (which comes from RFC 1123) is that the
behavior is dumb.  

To say this again in other words, that paragraph doesn't
prohibit you from rejecting a session from spew.spammer.com,
either by determining that you don't like the string, or by
noticing that the IP address is that of a known spammer
(regardless of whether it matches the EHLO argument string or
not), or by some other method.  It just doesn't address those
issues.

Those issues _are_ addressed in section 7.7, which gives fairly
broad latitude for SMTP servers to be configured to reject
problematic traffic while still conforming to the standard.

And the latter is the reason why 2821 must "address policy
issues".  If standards are to be useful, it is important, IMO,
to be able to make meaningful statements about whether something
conforms to them or not.  And that, in turn, makes it important
to distinguish between non-conforming behavior and behavior that
is consistent with adoption of policies that are sensible and
necessary under various conditions.   Policy decisions permit
you to reject, implementations must have ways to not reject,
and, if you reject, the standard specifies that you use a 550
code.

If anyone has textual suggestions that would make this more
clear and less convoluted, I'm listening.  Otherwise, can we
bury this particular dead horse.

     john

Date: Sat, 17 Aug 2002 14:26:15 -0500
From: "Eric A. Hall" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 1:45 PM Keith Moore wrote:

> apparently you seem to think there is something wrong with a protocol
> specification saying "don't use an unreliable identifier as the basis
> for policy decisions".

Wrong. I have no qualms with text regarding identifiers that cannot be
verified, or where the verification process itself fails.

OTOH, information which is not in dispute or where the verification
process has completed without error is a different ballgame.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

From: Keith Moore <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 14:45:11 -0400
Sender: [email-masked]

> I'm aware of that, but it's not what I'm asking about. I'm specifically
> asking about using the EHLO identifier as a policy trigger. Under the
> interpretations of the section as provided here, this is prohibited.

right.  but apparently you seem to think there is something wrong with 
a protocol specification saying "don't use an unreliable identifier as the 
basis for policy decisions".

Keith

From: Keith Moore <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 14:40:51 -0400
Sender: [email-masked]

> EG, I'm not allowed to reject sessions from spew.spammer.com, even
> if I know with certainty that the identifier is correct.

it's not clear how you can know this for certain from EHLO.  you might
know it for certain by other means, such as if you had a IPsec or TLS
authenticated session.  the point is that the EHLO identifier (more precisely, 
the relationship between that identifier and the IP source address) really 
isn't reliable enough to be used as the basis of any kind of filter.

to put it another way - nothing stops you from rejecting mail from
any host you want.  but it's a violation of the protocol to use 
the EHLO identifier to decide whether the mail is from a host that you
want to reject.  the only ways to "know with certainty" that this is
from a particular host don't involve EHLO.

Keith

Date: Sat, 17 Aug 2002 11:40:21 -0700
From: Gregory Neil Shapiro <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
Cc: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

> I'm aware of that, but it's not what I'm asking about. I'm specifically
> asking about using the EHLO identifier as a policy trigger. Under the
> interpretations of the section as provided here, this is prohibited.

The current text says:

   An SMTP server MAY verify that the domain name parameter in the EHLO
   command actually corresponds to the IP address of the client.
   However, the server MUST NOT refuse to accept a message for this
                                                           ^^^^^^^^
   reason if the verification fails: the information about verification
   ^^^^^^
   failure is for logging and tracing only.

It doesn't say that the EHLO/HELO command can not be rejected for
a *different* reason, such as policy.  You just can't reject based
on the verification of IP address -> HELO/EHLO claimed name.

In fact, section 7.7 states:

   of mail.  Some sites have decided to limit the use of the relay
   function to known or identifiable sources, and implementations SHOULD
   provide the capability to perform this type of filtering.  When mail
   is rejected for these or other policy reasons, a 550 code SHOULD be
   used in response to EHLO, MAIL, or RCPT as appropriate.

Thereby giving permission to reject EHLO based on policy.

Date: Sat, 17 Aug 2002 13:04:02 -0500
From: "Eric A. Hall" <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 12:54 PM Arnt Gulbrandsen wrote:
> You most certainly are allowed to reject connections from 
> www.spammer.com.
> 
> You can reject the connection because the SMTP client's IP corresponds 
> to www.spammer.com. Perfectly legal by the RFC. The HELO/EHLO argument 
> never enters the picture.

I'm aware of that, but it's not what I'm asking about. I'm specifically
asking about using the EHLO identifier as a policy trigger. Under the
interpretations of the section as provided here, this is prohibited.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

From: Arnt Gulbrandsen <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
Subject: Re: clarification re 2821, s4.1.4
Cc: [email-masked]
Date: Sat, 17 Aug 2002 23:24:00 +0530
Sender: [email-masked]

You most certainly are allowed to reject connections from 
www.spammer.com.

You can reject the connection because the SMTP client's IP corresponds 
to www.spammer.com. Perfectly legal by the RFC. The HELO/EHLO argument 
never enters the picture.

--Arnt

From: Keith Moore <[email-masked]>
To: "Eric A. Hall" <[email-masked]>
cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4 
Date: Sat, 17 Aug 2002 12:57:42 -0400
Sender: [email-masked]

> What it says is that I cannot "refuse to accept [...] if verification
> fails". It does not mention rules for when verification has succeeded,
> especially in the negative form. 

I see what you are saying, and I agree that it could be worded more clearly.

But it is intended to mean that the fact that EHLO name and the address 
do not match is not a good reason for rejecting the mail.   There are too
many situations where a client can legitimately be using a name in EHLO
but where the server's query of that name will not produce the host's
source IP address, and/or the server's query of that source IP address 
will not produce the name used in EHLO.

Keith

Date: Sat, 17 Aug 2002 11:53:22 -0500
From: "Eric A. Hall" <[email-masked]>
To: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 11:27 AM Eric A. Hall wrote:

> It does not mention rules for when verification has succeeded,
> especially in the negative form.

ps--according to the statements made here, I am not allowed to filter
based on a HELO identifier which has verified in the positive form,
either. EG, I'm not allowed to reject sessions from spew.spammer.com, even
if I know with certainty that the identifier is correct.

So, clarification on allowable policy when verification succeeds is what
I'm after, although frankly, I don't know why 2821 addresses policy issues
at all.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Sat, 17 Aug 2002 11:27:20 -0500
From: "Eric A. Hall" <[email-masked]>
CC: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

on 8/17/2002 5:19 AM Kai Henningsen wrote:

>> The other way of reading it says that an identifier which has been
>> verified as wrong can be rejected.
>
> I do not see how you can justify that reading

What it says is that I cannot "refuse to accept [...] if verification
fails". It does not mention rules for when verification has succeeded,
especially in the negative form. I know that localhost is 127.0.0.1. Any
other address which claims to be localhost can be verified as incorrect.
The spec doesn't address that branch in the logic tree, thus my request
for clarification.

Separately, avoiding transfer loops by rejecting mail which appears to be
coming from the local MTA (or claims to be anyway) seems to be a good
idea, regardless of whether the identifier has been verified or not.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Sat, 17 Aug 2002 11:52:42 -0400
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Sender: [email-masked]

--On Saturday, 17 August, 2002 16:43 +0530 Arnt Gulbrandsen
<[email-masked]> wrote:

> 
> What the standard says is: If the EHLO parameter doesn't match
> the client, you can write scary messages in the response, and
> you can tarpit the conversation (sending 100-line multiline
> responses at 20 seconds per line) to slow down the presumed
> spammer, and you can do lots of other things, but you can't
> refuse the mail outright.

I would add "for that reason" after "outright", but, otherwise,
this seems to me to be a _very_ good summary.

    john

From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Date: Sat, 17 Aug 2002 16:43:19 +0530
Sender: [email-masked]

What the standard says is: If the EHLO parameter doesn't match the 
client, you can write scary messages in the response, and you can 
tarpit the conversation (sending 100-line multiline responses at 20 
seconds per line) to slow down the presumed spammer, and you can do 
lots of other things, but you can't refuse the mail outright.

--Arnt

Date: 17 Aug 2002 12:34:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked]  wrote on 13.07.02 in <[email-masked]>:

> I think this is a fine piece of theoretical analysis which unfortunately has
> no applicability to the real world. In the real world directory/database
> systems that let sites provide Internet-wide access to information about
> their users do not deploy.
>
> I'm not sure why this is the case, and indeed I'm fairly sure that there is
> no good reason for it, but a vast tract of experience tells me it for-sure
> is the case. (Now is not the time to get into whys in more detail.)

I'm pretty sure that *one* important reason is that a "directory service"  
actually serves two rather distinct functions, and trying to do both with  
the same protocol works fine in an intranet but is insane in the Internet.

Those two functions are

(1) Find out attributes of a particular user (this seems to be what RESCAP  
does)

(2) Search for a particular user matching some conditions about his  
attributes ("Who is the CEO for Apple?" "Give me a list of all people at  
Microsoft named John Smith")

Note that the first does not necessarily (depending on protocol design)  
give you any method to, for example, locate valid user addresses to spam.  
The second, OTOH, does so by design.

Obviously, you may (in the Internet, as opposed to an intranet) want to be  
the restrictions for (1) to be radically different from those for (2). For  
example, (1) may be configured explicitely for a few mailboxes and just  
use the equivalent of a wildcard MX for the rest ("most accounts here  
don't want to receive HTML, but it is ok for our press officer"); whereas  
(2) might be configured to just list those few role addresses you want  
public contacts to go to (presumably the exact same addresses you list on  
your web pages), and not advertize anything at all about any other  
address.

MfG Kai

Date: 17 Aug 2002 12:19:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
cc: [email-masked]
Subject: Re: clarification re 2821, s4.1.4
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Eric A. Hall)  wrote on 16.08.02 in <[email-masked]>:

>  | 4.1.4 Order of Commands
>
>  | An SMTP server MAY verify that the domain name parameter in the EHLO
>  | command actually corresponds to the IP address of the client.
>  | However, the server MUST NOT refuse to accept a message for this
>  | reason if the verification fails: the information about verification
>  | failure is for logging and tracing only.
>
> There are two possible ways to read that.
>
> The first way says that a legitimately-formed identifier cannot be
> refused, even if it is verified as wrong. This prevents me from rejecting
> sessions which provide my own domain name in the identifier.
>
> The other way of reading it says that an identifier which has been
> verified as wrong can be rejected.

I do not see how you can justify that reading, in light of the standard  
explicitely disclaiming it.

> Which is the intended interpretation? and can we get a clarification when
> this is revisited.

The clarification is right there in your quote of the standard. It seems  
completely unambiguous to me.

MfG Kai

Date: Fri, 16 Aug 2002 15:28:57 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: drums2
Sender: [email-masked]

In ietf-822, I think we're coming to the conclusion that it IS time to 
open 2821 and 2822 up again. It has been suggested that 2822 is ready to 
move to draft standard, but 2821 has too many needed changes to let it 
do so at this time.

And it is not clear whether a working group is required to do this work.

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> 
> [I recently posted the following query in ietf-822. I should have posted 
> it here as well. - Tony]
> 
> 2821/2822 are "Proposed Standards" and have been out for 1.5 years now. 
> What will it take to move them forward to Draft Standard? We've had 
> considerably longer than 1.5 years to get operational experience with 
> them, and I KNOW that there are lots of interoperational instances of each.
> 
>     Tony Hansen
>     [email-masked]
> 
> P.S. If 2821 and 2822 need to be reissued, it might also be nice if they 
> could be done in time to get the numbers 3821 and 3822. My estimate is 
> that those numbers will be reached in about 23 months. That should give 
> us enough time to work on them, if we started now. :-)
> 
> 

Date: Fri, 16 Aug 2002 15:09:16 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: drums2
Sender: [email-masked]

[I recently posted the following query in ietf-822. I should have posted 
it here as well. - Tony]

2821/2822 are "Proposed Standards" and have been out for 1.5 years now. 
What will it take to move them forward to Draft Standard? We've had 
considerably longer than 1.5 years to get operational experience with 
them, and I KNOW that there are lots of interoperational instances of each.

     Tony Hansen
     [email-masked]

P.S. If 2821 and 2822 need to be reissued, it might also be nice if they 
could be done in time to get the numbers 3821 and 3822. My estimate is 
that those numbers will be reached in about 23 months. That should give 
us enough time to work on them, if we started now. :-)

Date: Fri, 16 Aug 2002 10:57:57 -0500
From: "Eric A. Hall" <[email-masked]>
To: [email-masked]
Subject: clarification re 2821, s4.1.4
Sender: [email-masked]

 | 4.1.4 Order of Commands

 | An SMTP server MAY verify that the domain name parameter in the EHLO
 | command actually corresponds to the IP address of the client.
 | However, the server MUST NOT refuse to accept a message for this
 | reason if the verification fails: the information about verification
 | failure is for logging and tracing only.

There are two possible ways to read that.

The first way says that a legitimately-formed identifier cannot be
refused, even if it is verified as wrong. This prevents me from rejecting
sessions which provide my own domain name in the identifier.

The other way of reading it says that an identifier which has been
verified as wrong can be rejected.

Which is the intended interpretation? and can we get a clarification when
this is revisited.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

To: [email-masked], [email-masked]
Subject: an attempt to make CONNEG-over-SMTP work reasonably over mail relays
cc: [email-masked]
From: Keith Moore <[email-masked]>
Date: Tue, 06 Aug 2002 15:15:44 -0400
Sender: [email-masked]

this is an attempt to outline what it will take to make CONNEG-over-SMTP
work reasonably over mail relays

goals:

- make it possible for a client to request recipient capabilities, 
  for whatever use, provided it can communicate directly with 
  recipient's SMTP servers.

- make it possible for senders to request "assuredly deliver in a 
  form the recipient can use, else fail" semantics.

- work with relaying

- no change to current SMTP functioning unless explicitly requested
  by sender

1. providing recipient capability information

for general-purpose mail reception, recipient capability information 
MUST be under the control of the recipient.

for fixed-function devices which have exclusive use of one or more 
mailboxes, the recipient capabilities associated with those mailboxes 
MAY be set by such devices to reflect their capabilities.  the mechanism 
by which this is done, in the case where the SMTP server and the 
receiving device are separate, is currently unspecified.

posting of recipient capability information DOES NOT imply that the
recipient is unwilling to accept contents that are outside of those
capabilities.  (if this functionality is desirable this needs to be a 
separate capability - because it's clearly useful to state preferences 
while still being willing to accept the occasional attachment in an
unpreferred format)

2. obtaining recipient capability information

recipient capability information (if available) may be obtained by any 
party in the following way:

 - do a SRV lookup in DNS for {prefix-tbd}._tcp.{recipient-domain}.
   if any servers are returned, contact them using the SMTP protocol.
   if server supports CONNEG in the EHLO response. issue RCPT 
   <recipient> CONNEG=OPTIONAL, record results, issue QUIT, and 
   close connection.  

this allows senders to potentially determine recipient capability
prior to sending the message.

there is no requirement that the SMTP servers advertised in this
manner actually be willing to accept mail, unless they are also
advertised as MXes for that domain.  e.g. they can respond to all 
DATA and similar commands with 5xx errors.

any CONNEG information returned by such servers MUST be consistent
with CONNEG information that would be obtained by MTAs relaying 
mail to the recipient, though the information is allowed to change
over time.

3. originating mail with "deliver recipient-usable content or fail"
   semantics

this extension attempts to provide a new transport option, known as
"deliver recipient-usable content or fail" (DRUCOF for short) semantics

- DRUCOF semantics MUST be explicitly requested by the sender 

- in SMTP, the sender's request is communicated with a DRUCOF option
  to RCPT (which is advertised in EHLO in the usual way, details 
  to be determined)

- server advertisement of DRUCOF in EHLO asserts that the server is 
  willing to convey any sender's request for DRUCOF semantics 
  and to implement them as provide in section 4.

DRUCOF applies only to critical portions of a message.  By default 
all portions of a message are assumed to be critical.   This can be
specified by the sender on a per-body part basis by including an 
appropriate body part header.  details to be determined.  

it should be possible to mark a body-part in any of several ways:
conversion-prohibited, conversion-permitted, omission-permitted;
it should also be possible to mark a body-part as to whether
ability to provide the specified integrity level is critical
to successful message delivery.

the DRUCOF extension is defined as a mechanism to allow the sender's
request for "deliver recipient-usable content or fail" semantics
to be communicated through relays.  if initial submission of the 
message (from the UA) is directly to the recipient's primary MX, 
DRUCOF is not needed, and the behavior of the sender is as specified
in the next section.

4. relaying mail with the DRUCOF option

when an SMTP client attempts to relay mail to a recipient for which
the DRUCOF option was specified, or for which the SMTP client knows
for certain that the sender requests DRUCOF semantics, the following
applies:

- if the SMTP server advertises CONNEG:

  - if an attempt to request a CONNEG string for the recipient succeeds {

    - if the client can perform conversions to make the message 
      acceptable to the recipient, or if no conversions are
      necessary to make the message acceptable, perform conversions
      as necessary and relay the message without asserting DRUCOF 
      or CONNEG.

    - else, if the server supports DRUCOF, relay the message with
      the DRUCOF option  (if the client can't perform the conversion,
      perhaps a later MTA can)

    - else, the delivery fails (conversion required but not supported)

    }

  - else (request for CONNEG string for a recipient fails
    due to a permanent or temporary error) {

    - if the server supports DRUCOF, relay the message with 
      the DRUCOF option to RCPT

    - else, the delivery fails (DRUCOF specified but capabilities unavailable)

    }

- else, if the SMTP server advertises DRUCOF

   - relay the message with DRUCOF set

- else

   - the delivery fails (no way to relay sender's preference for DRUCOF)

note: the above assumes that it's better to convert as early as possible,
and (more to the point) to provide as many opportunities for conversion
as possible.  one could argue that it's better to convert as late as 
possible so that the conversion is closer to the recipient and more likely
to be done according to the recipient's preferences.  I'm not sure about 
which is better.

note 2: it should be feasible to extend this to allow the sender to
optionally request conversion when available and appropriate, but to 
deliver intact if conversion cannot be performed.  

5. conversions

conversions MUST NOT be performed unless requested by the sender

conversions MUST NOT be performed in the absence of specific information
about the recipient

conversions MUST NOT be performed on content marked conversion-prohibited.

conversions performed by MTAs other than the sender or recipient's
SHOULD be standardized by some appropriate body

conversions SHOULD be chosen in such a way as to minimize loss of
information

when multiple conversions are possible, the least lossy conversion
SHOULD be employed.

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Fwd: I-D ACTION:draft-ietf-fax-esmtp-conneg-03.txt 
Date: Tue, 06 Aug 2002 14:13:00 -0400
Sender: [email-masked]

> Keith,
> 
> Responding to those items that I can:
> 
> At 03:19 PM 8/5/2002 -0400, Keith Moore wrote:
> >- it is still overbroad, claiming applicability for general email
> >   when it really works only for a limited subset of content
> >   which has an obvious ordering relation for "quality".
> 
> Please feel free to provide real-world examples of usage that is likely to 
> be attempted and that will not work properly.

I have already done so, on multiple occasions.  Perhaps we have different 
ideas of what is 'likely' or what is 'real-world'.  To me it seems likely 
and consistent with the real world that senders will send around a wide
variety of content, that recipients will want to specify what kinds of 
content they are willing to accept to keep from having their mailboxes
cluttered with useless content, and that senders will at least occasionally 
want to specify that the message should not be delivered unless the recipient 
is willing to accept the content.  All of which is fine.  But the assumption 
that some intermediate MTA can then be expected to "transform" the sender's 
content into something that the recipient is willing to accept - that does
not seem either fine, likely, or consistent with the real world.

At any rate, I don't think it's acceptable for an extension to cause
delivery failure or unrequested munging of messages even in 'unlikely' 
cases - when whether the extension is employed isn't under control of 
the sender.

> >      perhaps illustrating this, all of the examples are for fax.
> 
> You believe that making examples be for something other than fax will 
> materially affect the adequacy of the normative specification?

I think that the examples being all fax illustrates that other cases 
have not been adequately considered.  Because as soon as you start to
consider other cases of how SMTP is used to send content, you find 
that the existing proposal doesn't suffice.
 
> >- it still permits unauthorized parties to make arbitrary and
> >   unspecified alterations to mail.
> 
> Sorry, no.  It makes no alterations to the power of intermediaries.  It 
> gives them better information, not different capabilities.

No, it's not just giving better information - because the SMTP client is 
*required* to do something with that information.  And there are words 
and examples in the current proposal that have MTAs doing transformations 
without any authorization from the sender or recipient to do so.

> In any event, the "permissions" assigned to intermediaries are a matter of 
> administration policy, not technical specification.  If an administrative 
> authority does not want to permit use of this option, it will not use it.

I disagree that it is appropriate to assign such authority to administrators.
Certainly the transformations will not be performed (i.e. the software will
not be installed) without the administrators' consent, but the authority
to transform a particular message needs to be given by the user, not the
administrator.  The administrator is rarely in a position to know whether
the transformation is appropriate.

> 
> >- worse, the new version permits unauthorized parties to make
> >   arbitrary assertions of recipient capabilities in the absence
> >   of explicit information.
> 
> That is not what the discussion says.  It describes a way of having the 
> receiving SMTP server take on the responsibilities for matching 
> capabilities that already reside with the sending SMTP server.

I think you have that backwards, or something.  It doesn't make any 
sense to me.

> Remember that these servers about which you are so suspicious already have 
> "control" over the semantics of the recipient's address.  You can't get 
> more powerful than that.

Yes, but the function of SMTP intermediaries has generally been limited to
relaying of the message intact - not performing arbitrary transformations on 
the message.  We've accepted that minor transformations might be necessary
in a gateway situation, but we've never blessed the idea that arbitrary MTAs 
should change the content of messages within the SMTP world.

> And, by the way, please remember that the appendix is not normative.  It 
> does not change the specification.  It "permits" and "prohibits" nothing.

Perhaps it is only misleading or confusing, but it still affects the 
quality of the document.
 
> >- it still allows SMTP servers to make assertions of recipient capabilities
> >   without actual knowledge of those capabilities.
> 
> What is the problem this causes?  How does it make this service fail?

If the message is not delivered intact when sender did not request
transformations and the receiver did not request transformations,
that's a failure.  SMTP isn't supposed to mung messages.
 
> >- I still think overloading of RCPT is a pessimal way to do this
> 
> A number of separate analyses of the design choices suggest pretty strongly 
> that a separate command is actually worse than an RCPT parameter.

A lot of that analysis depends on how you assume that this is used - and 
in particular whether it is used only by special-purpose devices or if 
it's used between UAs in general.   I don't share those assumptions.

> A separate command changes the SMTP state machine.  That is not a minor 
> thing.  An example of the impact of this is probable failure in getting 
> through some firewalls.

A separate command doesn't have to change the state machine in the sense
that it adds either new states or new state transitions -  and I'd argue
that the command should not do so.  I'll grant that firewalls might have 
trouble with a new command.  But I've also seen firewalls have trouble 
with new parameters, so I don't know how much the existence of firewalls 
favors a new parameter over a new command. 

And RCPT with CONNEG most definitely changes the state machine of the sender,
even in cases where the sender sends at most one RCPT per session, because
there is at least new one state transition necessary for "cannot adapt
content to recipient's capabilities".  And in general MTAs will need to deal
with the multiple RCPT cases, which adds a fair amount of complexity to the
sender's state machine.

> >   since the command will often need to be issued twice anyway -
> 
> Your use of the word "often" is contrary to the likelihoods expected by 
> others.

Yes, I understand that, though I think "some others" would be more accurate.
People will disagree about what they think are probable usage scenarios.  

> >recommendations for near-term publication:
> >
> >- this feature should be renamed and the document reworded to make it
> >   clear that it is fax-specific rather than general purpose.
> 
> There is nothing in the normative text that is fax specific.

The expectation that it's possible to downgrade to a common format is 
specific to fax.  It might work with voice mail assuming no proprietary 
codecs were used, but that seems like a stretch.  Also, the assumptions 
about how this will be used, that governed design decisions like 
overloading RCPT vs a new command, seem to be heavily based on fax.
 
> >- misleading text should be fixed, unworkable recommendations should be
> >   removed.
> 
> Please offer whatever text changes you think helpful.

To some degree I have tried to do so already, though perhaps not bracketed
as such.  But I don't think we're quite to the point we can solve the 
problems by mere wordsmithing. 

> >- the document should be published as experimental rather than on the
> >   standards track, since it doesn't meet the requirements for proposed
> >   standard.
> 
> You have so far offered no details of failure scenarios that warrant the 
> fears you are expressing.

That's false. I've done so on multiple occasions.

> >things that need to be fixed before the document should be considered
> >as standards-track:
> >
> >- needs to be examined for a wide range of content
> 
> Again, you are offering such a generic suggestion, there is no way to know 
> what problems you fear or how to satisfy them.

Well, I've already stated them on multiple occasions.  And when the problem
is one of a failure to consider some important factor early in the design 
phase (as this seems to be), the fix is to reconsider that factor. 
Until that is done, all protocol fixes will be suspect.

> >- needs to have a well-defined protocol by which senders can request
> >   conneg "convert-or-fail" semantics on a per-message, and probably
> >   per-bodypart, basis.   if they are not requested, such semantics
> >   must not be applied.
> 
> So, SMTP is not appropriate to standardize without having submit and pop 
> also specified at the same time?

I didn't say anything about the mechanism, and in fact I expect that 
senders would specify transformation permissions in message and body part 
headers.  I don't know if it's necessary to standardize the mechanisms 
by which recipients specify conneg parameters, but I do think it's 
necessary that recipients consent to such rather than having some 
unrelated other party make assertions on their behalf.

> That is a barrier that is equivalent to the one you wish to impose before 
> standardizing the current option.  It means that you believe that an entire 
> suite of specifications must be offered at the same time, before any one of 
> the components may be standardized.  This is contrary to typical IETF 
> practise.

I think you have this backwards.  It's contrary to IETF practice to 
standardize a protocol that has known technical omissions - which this
proposal certainly does.  Your claim that it's appropriate to advance
an incompete specification because 'an entire suite of specifications
must be offered at the same time' is baseless and not supported by
either the process standards or by 'typical IETF practise'.  

> >- needs to explicitly specify which conversion algorithms are permissible
> >   and how new conversion standards may be added, since there can be
> >   multiple ways to perform a conversion and some are worse than others.
> 
> Conneg is not new to this option.  The requirement that you wish to impose 
> has not been imposed on other uses of Conneg.  It appears that you are 
> confusing a carriage scenario with a conversion mechanism.

The difference here, I believe, is that with CONNEG over SMTP arbitrary
SMTP servers are both allowed and expected *as a matter of protocol 
conformance* to perform conversions without consent of either sender or 
receiver.
 
> >- needs to state that conneg features are only advertised with the
> >   recipient's consent and control
> 
> If you have text to suggest for the draft, please offer it, for working 
> group review.  (It would have been nice to have had that text offered 
> earlier.  The danger is that there will be a running stream of 
> non-normative modifications of this nature.)

I could offer text on this one specific point, but there are other points
for which offering text seems counterproductive.  In general, it seems 
premature to offer text when we don't even have a vague agreement on what 
specific mechanisms (e.g. for indicating consent) should look like.
 
> >- needs to state exactly which parties are authorized to perform
> >   conversion.
> 
> Huh?
> 
> 
> >- needs to prevent multiple conversions.
> 
> What does that mean?  What problems are you concerned about?

this is mentioned below in the detailed comments.
 
> >problem: this proposal doesn't provide any way to "enforce" content
> >negotiation past a single hop.
> 
> You believe that there does not already exist a requirement for relays to 
> enforce content quality?  You believe that the concern about enforcing 
> content quality is new to this option?

I don't know what it means to "enforce content quality", partially since
I think it's the job of MTAs to "maintain content integrity" and partially
since the MTA has no notion of what "quality" means to the recipient.   
I believe there is now a requirement for relays to relay mail intact - in 
the current world, any relay that doesn't do so is nonstandard.  The proposed 
extension tries to change that.

> >nit:  It is not clear whether it is desirable for the server to give
> >CONNEG errors priority over all other sources of error.
> 
> The nature of any complex system is that there can be multiple errors at 
> approximately the same time.
> 
> Hence your concern for prioritization of error reporting is not specific to 
> this option.

No, but the proposal says that it MUST take precedence, and that seems 
overbroad.  I did attempt to suggest better wording.
 
> >problem: the client cannot be compelled to perform transformations that
> >are impossible or for which the client lacks the capability.  no client
> >can reasonably convert video to audio.
> 
> Please provide a detailed example of Conneg use that could lead to the 
> problem of needing to convert video to audio.

Someone sends mail with a video attachment to a recipient that advertises
that it can only accept audio.  Somebody (whether the sender or an 
intermediate MTA) adds the CONNEG option for that recipient.  The conversion 
is not possible, yet it is required by the current spec.
 
> > >      (2) Common Content
> > >
> > >      For a single ESMTP session, issue RCPT commands that obtain
> > >      content capabilities information for each recipient.  With the
> > >      DATA command, send the best content form that can be processed by
> > >      ALL of the recipients.  Some recipients will receive content that
> > >      is below their best capabilities.  However this approach also
> > >      consumes the least bandwidth and has the fewest cross-network
> > >      protocol exchanges
> >
> >problem: this scenario does not work because the sender has no way to know a
> >priori whether it is possible to produce a common form that is acceptable
> >to all recipients.
> 
> Feel free to provide a real-world example of being unable to produce a 
> common form within the context of conneg capabilities information.  You 
> have raised this concern before and we have, so far, seen nothing concrete 
> to substantiate the concern.

As far as I can tell, this is only because you believe that nobody will try 
to use CONNEG except in those cases that you've anticipated or where it has
been observed to work well.   It's not as if I haven't supplied concrete 
examples as to how things can fail, but you dismiss them as not 'real-world'.  
In my "real world" users' interests and habits are quite diverse and are 
not so predictable.

At any rate, making things work in your idea of the real world is not 
sufficient.  Email must function properly as a matter of design, so that
if the specifications are followed then predictable and useful behavior 
is assured.  Speculation about what users are likely to do and what 
administrators are likely to impose is not sufficient.

Keith

Date: Tue, 06 Aug 2002 01:21:23 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-ietf-fax-esmtp-conneg-03.txt 
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Keith,

Responding to those items that I can:

At 03:19 PM 8/5/2002 -0400, Keith Moore wrote:
>- it is still overbroad, claiming applicability for general email
>   when it really works only for a limited subset of content
>   which has an obvious ordering relation for "quality".

Please feel free to provide real-world examples of usage that is likely to 
be attempted and that will not work properly.

>      perhaps illustrating this, all of the examples are for fax.

You believe that making examples be for something other than fax will 
materially affect the adequacy of the normative specification?

>- it still permits unauthorized parties to make arbitrary and
>   unspecified alterations to mail.

Sorry, no.  It makes no alterations to the power of intermediaries.  It 
gives them better information, not different capabilities.

In any event, the "permissions" assigned to intermediaries are a matter of 
administration policy, not technical specification.  If an administrative 
authority does not want to permit use of this option, it will not use it.

>- worse, the new version permits unauthorized parties to make
>   arbitrary assertions of recipient capabilities in the absence
>   of explicit information.

That is not what the discussion says.  It describes a way of having the 
receiving SMTP server take on the responsibilities for matching 
capabilities that already reside with the sending SMTP server.

Remember that these servers about which you are so suspicious already have 
"control" over the semantics of the recipient's address.  You can't get 
more powerful than that.

And, by the way, please remember that the appendix is not normative.  It 
does not change the specification.  It "permits" and "prohibits" nothing.

>- it still allows SMTP servers to make assertions of recipient capabilities
>   without actual knowledge of those capabilities.

What is the problem this causes?  How does it make this service fail?

>- I still think overloading of RCPT is a pessimal way to do this

A number of separate analyses of the design choices suggest pretty strongly 
that a separate command is actually worse than an RCPT parameter.

A separate command changes the SMTP state machine.  That is not a minor 
thing.  An example of the impact of this is probable failure in getting 
through some firewalls.

>   since the command will often need to be issued twice anyway -

Your use of the word "often" is contrary to the likelihoods expected by others.

>recommendations for near-term publication:
>
>- this feature should be renamed and the document reworded to make it
>   clear that it is fax-specific rather than general purpose.

There is nothing in the normative text that is fax specific.

>- misleading text should be fixed, unworkable recommendations should be
>   removed.

Please offer whatever text changes you think helpful.

>- the document should be published as experimental rather than on the
>   standards track, since it doesn't meet the requirements for proposed
>   standard.

You have so far offered no details of failure scenarios that warrant the 
fears you are expressing.

>things that need to be fixed before the document should be considered
>as standards-track:
>
>- needs to be examined for a wide range of content

Again, you are offering such a generic suggestion, there is no way to know 
what problems you fear or how to satisfy them.

>- needs to have a well-defined protocol by which senders can request
>   conneg "convert-or-fail" semantics on a per-message, and probably
>   per-bodypart, basis.   if they are not requested, such semantics
>   must not be applied.

So, SMTP is not appropriate to standardize without having submit and pop 
also specified at the same time?

That is a barrier that is equivalent to the one you wish to impose before 
standardizing the current option.  It means that you believe that an entire 
suite of specifications must be offered at the same time, before any one of 
the components may be standardized.  This is contrary to typical IETF practise.

>- needs to explicitly specify which conversion algorithms are permissible
>   and how new conversion standards may be added, since there can be
>   multiple ways to perform a conversion and some are worse than others.

Conneg is not new to this option.  The requirement that you wish to impose 
has not been imposed on other uses of Conneg.  It appears that you are 
confusing a carriage scenario with a conversion mechanism.

>- needs to state that conneg features are only advertised with the
>   recipient's consent and control

If you have text to suggest for the draft, please offer it, for working 
group review.  (It would have been nice to have had that text offered 
earlier.  The danger is that there will be a running stream of 
non-normative modifications of this nature.)

>- needs to state exactly which parties are authorized to perform
>   conversion.

Huh?

>- needs to prevent multiple conversions.

What does that mean?  What problems are you concerned about?

>problem: this proposal doesn't provide any way to "enforce" content
>negotiation past a single hop.

You believe that there does not already exist a requirement for relays to 
enforce content quality?  You believe that the concern about enforcing 
content quality is new to this option?

>nit:  It is not clear whether it is desirable for the server to give
>CONNEG errors priority over all other sources of error.

The nature of any complex system is that there can be multiple errors at 
approximately the same time.

Hence your concern for prioritization of error reporting is not specific to 
this option.

>problem: the client cannot be compelled to perform transformations that
>are impossible or for which the client lacks the capability.  no client
>can reasonably convert video to audio.

Please provide a detailed example of Conneg use that could lead to the 
problem of needing to convert video to audio.

> >      (2) Common Content
> >
> >      For a single ESMTP session, issue RCPT commands that obtain
> >      content capabilities information for each recipient.  With the
> >      DATA command, send the best content form that can be processed by
> >      ALL of the recipients.  Some recipients will receive content that
> >      is below their best capabilities.  However this approach also
> >      consumes the least bandwidth and has the fewest cross-network
> >      protocol exchanges
>
>problem: this scenario does not work because the sender has no way to know a
>priori whether it is possible to produce a common form that is acceptable
>to all recipients.

Feel free to provide a real-world example of being unable to produce a 
common form within the context of conneg capabilities information.  You 
have raised this concern before and we have, so far, seen nothing concrete 
to substantiate the concern.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Fwd: I-D ACTION:draft-ietf-fax-esmtp-conneg-03.txt 
Date: Mon, 05 Aug 2002 15:19:46 -0400
Sender: [email-masked]

[bcc'ed to iesg - on the assumption that they might not want to see the 
resulting replies]

> The SMTP Conneg draft has been revised, with the primary change being 
> addition of two Appendices.  One discusses use of the option when there are 
> multiple recipients.  The other discusses use of the option when there is 
> relaying.

I think the two appendices help slightly.  However they also introduce 
new problems not present in the previous draft.  Also, the new version of 
the document fails to correct a number of problems (some of them serious) 
in the previous version.

In particular,

- it is still overbroad, claiming applicability for general email
  when it really works only for a limited subset of content 
  which has an obvious ordering relation for "quality".   
  perhaps illustrating this, all of the examples are for fax.

- it still permits unauthorized parties to make arbitrary and
  unspecified alterations to mail.  

- worse, the new version permits unauthorized parties to make
  arbitrary assertions of recipient capabilities in the absence
  of explicit information.

- the new version also permits multiple conversions to take place,
  potentially degrading the message considerably more than a
  single conversion.

- it still mandates use of particular reply and status codes in cases
  where they are not always appropriate

- it still has the server reporting status codes for conditions that
  are best detected on the client

- it still allows SMTP servers to make assertions of recipient capabilities
  without actual knowledge of those capabilities.

- I still think overloading of RCPT is a pessimal way to do this
  since the command will often need to be issued twice anyway -
  either because of a mixture of recipient capabilities, or because
  the conversion could not be performed on-the-fly.

recommendations for near-term publication:  

- this feature should be renamed and the document reworded to make it 
  clear that it is fax-specific rather than general purpose.  

- misleading text should be fixed, unworkable recommendations should be
  removed.

- the document should be published as experimental rather than on the 
  standards track, since it doesn't meet the requirements for proposed 
  standard.

things that need to be fixed before the document should be considered
as standards-track:

- needs to be examined for a wide range of content

- needs to have a well-defined protocol by which senders can request
  conneg "convert-or-fail" semantics on a per-message, and probably
  per-bodypart, basis.   if they are not requested, such semantics
  must not be applied.

- needs to explicitly specify which conversion algorithms are permissible
  and how new conversion standards may be added, since there can be 
  multiple ways to perform a conversion and some are worse than others.

- needs to state that conneg features are only advertised with the 
  recipient's consent and control

- needs to state exactly which parties are authorized to perform
  conversion.

- needs to prevent multiple conversions.

some details are below.  not everything above can be seen from a close-up
view, so the list of details below should be considered separately from
the overall problems listed above.

> 1.   INTRODUCTION
> 
>      When a data source and a receiver have interactive access to 
>      each other, the receiver often informs the source of its 
>      capabilities, to permit optimized performance or functionality for 
>      the interaction.  Classic telephone-based facsimile is an example, 
>      as are voice over IP and ESMTP, among Internet applications.  The
>      store-and-forward nature of Internet mail is usually assumed to 
>      preclude such capabilities exchanges, although the sender in a 
>      store-and-forward scenario could benefit from knowing precise 
>      details about the receiver.  In some configurations, direct 
>      email-based interactions -- with the originating ESMTP client 
>      and the destination ESMTP server able to have direct TCP 
>      connect -- are possible, such as over an intranet. In addition 
>      an end-to-end exchange can ESMTP for hop-by-hop enforcement.   

nit: s/can/can use/ ???

problem: this proposal doesn't provide any way to "enforce" content
negotiation past a single hop.  there is no requirement that the CONNEG
option be relayed from one hop to another.  even if there were, it would
be problematic - it would require earlier SMTPs to advertise capabilities
of recipients for which they did not have information, and it would 
require (and trust) later SMTPs to perform arbitrary and undefined 
conversions on content to satisfy recipient-specific constraints.

>      This document defines an SMTP-based service extension [ESMTP1, 
>      ESMTP2] for content negotiation, whereby an SMTP client may 
>      request information about content capabilities of the target 
>      device or system that is serviced by an SMTP server.  

nit: the capability request is on a per-recipient-mailbox basis rather
than a 'target device or system' basis.  of course being able to request
per-recipient-mailbox capabilities is sufficient to request target
device capabilities when all recipients accessed via a single SMTP server 
have the same capabilities.  but the fact that it's a per-recipient
capability makes the proposal more complex to handle for the sender,
and it would be better to make this clearer from the start.

> The SMTP 
>      server may report the target's content capabilities back to the 
>      client. This process can be used to emulate a classic facsimile
>      start-of-session capabilities negotiation, as well as being used 
>      for other email-based services.  The extension is primarily 
>      intended for "direct" SMTP transfers, although relayed scenarios 
>      are permitted through a series of SMTP servers and are discussed 
>      in Appendix B.
> 
> 
> 2.   CONVENTIONS
>      
>      In examples, "C:" and "S:" indicate lines sent by the
>      client and server respectively.
>      
>      The key words "MUST", "MUST NOT", "SHOULD", "SHOULD
>      NOT", and "MAY" in this document are to be interpreted
>      as defined in "Key words for use in RFCs to Indicate
>      Requirement Levels" [KEYWORDS].
> 
> 
> 
> 3.   CONTENT NEGOTIATION SERVICE EXTENSION
>      
>      (1)  The name of the SMTP service extension is
>           "Content_Negotiation"
> 
>      (2)  The EHLO keyword value associated with this extension
>           is "CONNEG"
> 
>      (3)  A parameter using the keyword "CONNEG" is added to the
>           RCPT command
> 
>      (4)  The server responds with a report of the content
>           capabilities of the device or system that embodies the
>           target RCPT address.
> 
> 
> 
> 4.   CONNEG PARAMETER TO RCPT
>      
> 4.1  Parameter
> 
>      Keyword:
>           
>           CONNEG
>      
>      Argument:
>           
>           REQUIRED
>             The client requires support for the capability. If the 
>             target does not support the CONNEG parameter, the target 
>             MUST reject the RCPT command with a 504 reply.
> 
>             If the target can not support the capability due to a 
>             temporary problem, it MUST reject the RCPT command with 
>             a 404 reply. 

nit:  It is not clear whether it is desirable for the server to give 
CONNEG errors priority over all other sources of error.   For instance,
if a recipient address is invalid, reporting *that* probably results in
a more useful error message than reporting that the CONNEG feature is
not supported for that particular recipient.  

suggestion:  reporting of errors with the CONNEG=REQUIRED option SHOULD 
only occur if the RCPT command would otherwise have a successful result.  
If the RCPT command would fail for other reasons, the error and status
codes should reflect the nature of that failure.  In most cases this will
result in a more useful error message.   An reasonable exception might 
exist if the server checks several potential error conditions concurrently, 
and it determines that the command would fail the CONNEG=REQUIRED criteria
before it determines that the command would fail due to other reasons.

>           OPTIONAL
>             The client requests the target to use the capability. If 
>             the target can not support the capability at this time, the 
>             target MUST process the address and message as if the 
>             requested CONNEG capabilities had not been specified.
>             
>           If the argument does not exist, the default is "REQUIRED".
>           
>           When a capability is REQUIRED by the client but can not 
>           currently be supported by the target, an error response will 
>           have significant performance impact to overall SMTP 
>           processing.  Use of the OPTIONAL parameter will ensure high 
>           SMTP performance.

nit: s/ensure high SMTP performance/facilitate better SMTP performance/

even better would be to say that REQUIRED SHOULD only be used when the
sender has specified that the message delivery should fail unless it
is known that the recipient has the necessary capability.

even better would be to omit REQUIRED entirely, since it increases the
complexity of the server implementation and the protocol, for essential
zero benefit to the client.  REQUIRED doesn't remove the need for the
client to detect errors due to content negotiation, since it is always
possible that the server/recipient does not support any way of handling
the content at all, or even if such conversion might be possible, the 
client does not have the capability to perform that specific conversion.  

> 4.2 Client Action
>           
>      If the server issued a 250-CONNEG, as part of its
>      EHLO response for the current session, the client MAY
>      issue the CONNEG parameter with RCPT.
>           
>      If the client issues the CONNEG parameter with
>      RCPT, then it MUST honor the capabilities specified
>      in the CONNEG RCPT reply, and transform data that is
>      sent, so that the server can accept the data. The
>      client SHOULD transform the data to the "highest" level
>      of capability of the server.

problem: the client cannot be compelled to perform transformations that
are impossible or for which the client lacks the capability.  no client
can reasonably convert video to audio. few clients could convert audio
or fax to text, and then only if the audio or fax contained words.
if taken literally this provision would prevent any client from asserting 
CONNEG.

suggestion: rather than saying "MUST honor" say that the client
MUST NOT send forward which is inconsistent with the recipient's
stated capabilities.  the client SHOULD, if capable, perform 
transformations which are likely to preserve the recipient's ability
to use most of the information in the content.  if the client is unable
to perform such transformations, it MUST bounce the message.

(there probably needs to be some interaction with critical content
here - it might be acceptable for a SMTP to discard conntent marked
as non-critical rather than bouncing the message.  but under no
circumstances should the message be delivered to the recipient when
the recipient has stated via CONNEG that he/she cannot receive it)

>      If the server rejects the RCPT command with a 404 reply, the 
>      client may later reissue the RCPT with the CONNEG parameter
>      in a different SMTP session.

nit: presumably "may" should be capitalized here.

>      If the server returns an EHLO 250 code without CONNEG 
>      capabilities, the client MUST NOT issue a CONNEG parameter 
>      with RCPT.
> 
>      Methods of using of this option with multiple addressees, for 
>      the same content, are discussed in Appendix A.
> 
>      
> 4.3 Server Action
>           
>      If the client specifies "CONNEG=REQUIRED" in the RCPT, 
>      but the server does not support the CONNEG parameter, the server
>      MUST reject the RCPT command with a 504 reply. 
>      
>      If the server supports the CONNEG parameter, but can not return 
>      the recepient's capability temporarily, the server MUST reject the 
>      RCPT command with a 404 reply. For example, if the server gets 
>      the capability information from a directory, but its connection is 
>      offline, the server MUST reject the RCPT command with a 404 
>      reply. 
> 
>      If the client specifies "CONNEG=OPTIONAL" in the RCPT, 
>      but the server does not support the CONNEG parameter or can not 
>      return the recipient's capability temporarily, the server MUST 
>      process the address and message as if the requested CONNEG
>      capabilities had not been specified.
> 
>      Regardless of the value of the parameter, if the server does 
>      support the CONNEG parameter and the address is acceptable, 
>      then it MUST issue a 250 reply, followed by the capabilities
>      of the server that is specified by  the RCPT address.
>      Successful responses to CONNEG RCPT requests will always be
>      multiple SMTP lines.  The first line is the normal RCPT response,
>      and subsequent lines beginning with the exact string
>      "250-CONNEG " and "250 CONNEG " are the CONNEG responses.
>      The last line begins with "250 CONNEG ".  
> 
>      If the SMTP server supports ENHANCEDSTATUSCODES [RFC1893], the 
>      response strings for a success are "250-2.1.5 CONNEG" and 
>      "250 2.1.5 CONNEG". 

nit: I think you want these to be "250-2.1.5 CONNEG " and "250 2.1.5 CONNEG ".

nit: it is out of scope for this document to specify the status codes for
a successful RCPT command.

>      The response strings for indicating a
>      permanent failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
>      The response strings for a temporary failure are "404-4.3.3 
>      CONNEG" and "404 5.3.3 CONNEG".

problem: this overloads existing status codes for new error conditions.
there needs to be new status codes for "sender insisted on not delivering
the message unless it knew for sure that the recipient could deal with
it and the recipient's capabilities could not be obtained".

>      All CONNEG-capable clients and CONNEG-capable servers MUST
>      be able to successfully process CONNEG lines that are up to 512
>      characters long, as required by RFC2821. If the length of CONNEG
>      lines is greater than 512 characters, the server MUST insert
>      line breaks and make next CONNEG line. 

problem: you need to explain more about how this is done.  e.g. can
"<CR><LF>250-2.1.5 CONNEG " be inserted in the middle of a token?
are spaces at the end of a reply line significant?

>      The contents of the capability listing MUST conform to 
>      the specifications in "Content Feature Schema for Internet Fax". 
>      [RFC2879]

problem: this restriction seems inappropriate if the content is not a fax.

> 
> 
> 5.   SYNTAX
>      
>      Command with "CONNEG":
>       "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
>                    / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
>                  "OPTIOANL") CRLF

nit: OPTIONAL is misspelled

nit: "CONNEG =" should be "CONNEG="

nit: "<Postmaster@" domain ">" is redundant with Forward-Path

problem: grammar excludes other RCPT options

suggestion: the RCPT command syntax should not be formally described 
in this document; it is adequately described elsewhere and CONNEG
doesn't change the syntax.

>      Reply:
>       ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
>         ("250 CONNEG" capability CRLF) )/
>       ( ("250-2.1.5" CRLF) 
>           *("250-2.1.5 CONNEG" capability CRLF)
>            ("250 2.1.5 CONNEG" capability CRLF) )/
>       ("504" CRLF) /
>       ("504 5.3.3" CRLF) /
>       ("404" CRLF) /
>       ("404 4.3.3" CRLF) /
> 
>       capability  =  <<as per [RFC2879]>>

problem: this grammar excludes the possibility of other kinds of 
responses besides CONNEG.

problem: need to specify how line wrapping works.

radical suggestion: 

1. server encodes the recipient's conneg string in quoted-printable, with 
the line length limits and wrapping rules per the MIME spec.

2. server then prefixes each of the resulting quoted-printable lines with 
 
"250-2.1.5 CONNEG "

(perhaps adding 2.1.5 even if ENHANCEDSTATUSCODES are not supported)

3. client collects all lines from the response that begin with
"250-2.1.5 CONNEG ", removes that prefix from each line, and 
processes the resulting set of lines as a quoted-printable encoded 
file in order to obtain the conneg string for the recipient.

> 
> 6.   EXAMPLES
> 
> 6.1  Success Response
> 
>      An example of ESMTP sequence with successful RCPT response
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      S: 250-<[email-masked]> recipient ok
>      S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

nit: example shows inappropriate line wrapping, without explanation.

>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per:
>           (  image-file-structure=TIFF-minimal
>              dpi=400
>              image-coding=JBIG
>              size-x=2150
>            )
>         >>

nit: example shows invalid content-type

nit: example assumes fax.

>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 6.2  Long Response
> 
>      An example of successful RCPT response when the length of 
>      capability is greater than 512 characters. 
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250-2.1.5 CONNEG (&(image-file-structure=TIFF-minimal) ...
>      S: 250-2.1.5 CONNEG   .....
>      S: 250 2.1.5 CONNEG   (color=Binary)

okay, that helps, but it still doesn't show whether line wrapping
can occur in the middle of a conneg token.

> 
>      An example of succssful RCPT response when CONNEG-capable 
>      server supports ENHANCEDSTATUSCODES.
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

nit: example assumes fax.

> 
> 6.3  Failure Response
> 
>      An example of ESMTP sequence with parmanent failure RCPT 
>      response. 
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      
>      S: 504 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 6.4  Temporary Failure Response
> 
>      An example of an ESMTP sequence with temporary failure RCPT 
>      response when the value of parameter is "REQUIRED":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      
>      S: 404 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
>      .
>      .
>      .
>      retry according to implementation
> 
> 
> 6.5  Temporary Failure with Optional handling
> 
>      An example of an ESMTP sequence with temporary failure RCPT 
>      response when the value of parameter is "OPTIONAL":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = OPTIONAL

nit: should be "CONNEG=OPTIONAL"

>      
>      S: 250 <[email-masked]> recipient ok
>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per "A Simple Mode of Facsimile Using Internet Mail"
>           RFC2305
>         >>

nit: example assumes fax

>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
> 7.   IANA CONSIDERATIONS
>      
>      On publicatiom of this document by the RFC Editor, the IANA 
>      shall register the Content_Negotiation ESMTP extension defined 
>      in section 3.
> 
> 
> 
> 8.   SECURITY CONSIDERATIONS
>      
>      This ESMTP option calls for a respondent to disclose
>      its capabilities.  Mechanisms for determining the
>      requestor's authenticated identity are outside the
>      scope of this specification.  It is intended that this
>      mechanism permit disclosure of public information;
>      hence there is no particular need for security
>      measures.
>      
>      However there is nothing to prevent disclosure of
>      sensitive information that should receive restricted
>      distribution.  It is, therefore, the responsibility of
>      the disclosing ESMTP server to determine whether
>      additional security measures should be applied to the
>      use of this ESMTP option.
> 
>      A man-in-the-middle attack might change the capabilities reported 
>      for a given recipient. For example: Suppose the sender knows the 
>      recipient has the ability to view color documents so they mark 
>      some things in red in what is otherwise a black and white 
>      document. But someone interferes with the returned capabilities, 
>      indicating that the recipient only supports black and white. The 
>      document is duly downgraded, with the result that the recipient 
>      doesn't see what the sender marked. 
>      
>      An indirect exposure can occur when the report of a capability 
>      implies use of specific software.  If that software is known to 
>      have security weaknesses, the capabilities report effectively 
>      advertises the associated opportunity to exploit the security 
>      weakness.
>      
>      For target SMTP servers that require security mechanisms to be in 
>      force at the start of the session, the target SHOULD refrain from 
>      including the CONNEG parameter in an EHLO response until the 
>      requisite security mechanisms are in force.
> 
>      For digitally signed content, the use of this option poses a
>      special challenge.  Digitally signing content relies on that
>      content to be in a particular form.  Use of this option changes
>      that form.  Hence an SMTP client that uses this option on
>      digitally signed content MUST be able to recompute the digital
>      signature of the content. 
> 
> 
> 
>  9.  ACKNOWLEDGEMENTS
>      
>      Graham Klyne provided useful suggestions to an earlier
>      draft.
> 
> 
> 
> 10.  NORMATIVE REFERENCES
>      
>      [ESMTP1]  Klensin, J., Freed, N., Rose, M., Stefferud,
>                E. and D. Crocker, "SMTP Service Extensions",
>                RFC 1869, November 1995
>      
>      [ESMTP2]  Klensin, J., "Simple Mail Transfer Protocol",
>                RFC 2821, April 2001.
>      
>      [RFC2879] McIntyre, L. and G. Klyne, "Content Feature
>                Schema for Internet Fax", RFC 2531, August
>                2000
> 
>      [RFC2305] Toyoda, K., Ohno, H., Murai, J. and  D. Wing, 
>                "A Simple Mode of Facsimile Using Internet Mail", 
>                RFC 2305, March 1998.
> 
> 
> 
> 11.  AUTHORS' ADDRESSES
>      
>      Kiyoshi Toyoda
>      Matsushita Graphic Communication Systems,Inc
>      2-3-8 Shimomeguro, Meguro-Ku
>      Tokyo 153  JAPAN
>      
>      +81.3.5434.7161
>      [email-masked]
>      
>      
>      Dave Crocker
>      Brandenburg InternetWorking
>      675 Spruce Drive
>      Sunnyvale, CA  94086  USA
>      
>      +1.408.246.8253
>      [email-masked]
> 
> 
> 
> 12.  FULL COPYRIGHT STATEMENT
>      
>      Copyright (C) The Internet Society (2001).  All Rights
>      Reserved.
>      
>      This document and translations of it may be copied and
>      furnished to others, and derivative works that comment
>      on or otherwise explain it or assist in its
>      implementation may be prepared, copied, published and
>      distributed, in whole or in part, without restriction
>      of any kind, provided that the above copyright notice
>      and this paragraph are included on all such copies and
>      derivative works.  However, this document itself may
>      not be modified in any way, such as by removing the
>      copyright notice or references to the Internet Society
>      or other Internet organizations, except as needed for
>      the purpose of developing Internet standards in which
>      case the procedures for copyrights defined in the
>      Internet Standards process must be followed, or as
>      required to translate it into languages other than
>      English.
>      
>      The limited permissions granted above are perpetual and
>      will not be revoked by the Internet Society or its
>      successors or assigns.
>      
>      This document and the information contained herein is
>      provided on an "AS IS" basis and THE INTERNET SOCIETY
>      AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL
>      WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
>      LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
>      HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
>      WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
>      PARTICULAR PURPOSE.
> 
> 
> 
> APPENDIX
> 
> A.   USAGE FOR MULTIPLE RECIPIENTS
> 
>      ESMTP permits specification of multiple recipients for the same
>      content.  This option specified here can produce information that
>      different recipients have different content capabilities.  How 
>      this differential information is used will depend upon the 
>      requirements of the sender.  This specification does not mandate 
>      particular choices. Three scenarios are possible:
> 
>      (1) Single Recipient
>      
>      For each recipient, conduct a separate ESMTP session.  This
>      ensures that each content transfer can be tailored to the
>      capabilities of each recipient.  This also consumes the largest
>      amount of bandwidth and the largest number of cross-network SMTP
>      protocol exchanges
> 
>      (2) Common Content
> 
>      For a single ESMTP session, issue RCPT commands that obtain
>      content capabilities information for each recipient.  With the
>      DATA command, send the best content form that can be processed by
>      ALL of the recipients.  Some recipients will receive content that
>      is below their best capabilities.  However this approach also 
>      consumes the least bandwidth and has the fewest cross-network 
>      protocol exchanges

problem: this scenario does not work because the sender has no way to know a 
priori whether it is possible to produce a common form that is acceptable
to all recipients.  so any sender that sends multiple RCPT commands in a
single session MUST be prepared to terminate the session (either via RSET
or QUIT) and retry sending only to those recipients that DO have a common 
format.

suggestion: delete scenario 2, as it is not workable. 
 
>      (3) Partial Batching
>      
>      This scenario begins the same as the "Common Content" scenario.
>      Content capabilities information is obtained during a single ESMTP
>      session, with all of the RCPT commands issued together.  The
>      difference for this scenario is that the client SMTP then
>      terminates the session with an RSET and begins one or more new,
>      separate sessions.  Addressees are processed in batches, according
>      to the similarity of their capabilities.  This option balances
>      consumption of bandwidth with optimization of content, at the cost 
>      of a terminated session. 
> 
> 
> B.   SCENARIOS FOR RELAYING
> 
>      ESMTP is a direct transfer mechanism, using a single TCP 
>      connection.  It supports Internet mail store-and-forward through 
>      such characteristics as global addressing and ESMTP enforcement of 
>      global features.  

nit: I'm not at all sure what this last sentence means.

> Use of the ESMTP Conneg option is 
>      straightforward when the originating ESMTP client is able to 
>      directly connect to the destination ESMTP server.  The nature of 
>      this option is to return address-specific information that will 
>      affect content transmission.  Hence the use of this option in the 
>      presence of the in-direct effects of store-and-forward is not 
>      obvious.
>      
>      This Appendix discusses some styles of use for the ESMTP Conneg 
>      option when messages are being relayed.  The Appendix is intended
>      only to provide discussion.  It is neither intended to be 
>      exhaustive nor restrictive.  Other scenarios are likely and 
>      encouraged.
>      
>      
> B.1  Relay Server Modes
> 
>      The key challenge for use of the ESMTP Conneg option is the 
>      requirement the receiving ESMTP server be able to return 
>      capabilities on behalf of a target addressee.  This requirement 
>      translates into two, basic styles of operation for the receiving 
>      ESMTP server:
>      
>      (1) Server Knows Addressee Capabilities
>      
>      Here the receiving server has the necessary details about an 
>      addressee, at the time the sending server issues the RCPT 
>      response.  Typically, this information will be obtained through a 
>      direct, real-time query mechanism, either to a directory 
>      containing addressee information or to a service run directly on 
>      behalf of the individual addressee and possibly on the addressee's 
>      system.  
>      
>      One form of query to the addressee's system is to stack a 
>      cascading sequence of ESMTP sessions and RCPT commands 
>      together, all the way to the destination ESMTP server. Although 
>      theoretically possible, this attempt to turn a multi-hop scenario 
>      into a real-time, pseudo-direct query is not practical.  It 
>      will most likely result in response delays for the RCPT command 
>      that are not acceptable, particularly over the public Internet.
>      
>      (2) Server Ensures Conversion
>      
>      Fundamental to the use of a capabilities exchange is the 
>      requirement that the receiver of capabilities information be able 
>      to convert content into a form that is more capable than the 
>      default form, when the receiver indicates that it can support the 
>      superior form.  That is, use of this option presumes that the 
>      sender is holding a "more capable" form of the content and will 
>      map it to a "less capable form" if the receiver does not support 
>      the superior form.  

problem: there is inherently no way for the sender to know in advance 
whether it can convert the message in a form that is  suitable for the
recipient.

problem: for most kinds of content there is not an obvious ordering
which determines "more capable" or "less capable".  fax images containing
text are not inherently more or less capable than text - each has different
benefits and drawbacks.  HTML is not inherently more or less capable than 
PDF.   it depends on how the recipient will use the message.  so the 
assumption by which relaying is justified is not valid.

>      Hence another style of relaying configuration is to have a relay 
>      SMTP respond to an RCPT capability query by indicating that it 
>      supports the most capable form.  The sender will pass the best 
>      version it can.  The receiver has then taken responsibility for 
>      performing later conversions, as necessary, to the next hop in the 
>      sequence.  That is, the receiving ESMTP server inherits the same 
>      level of responsibility already being held by the sending ESMTP 
>      client.

problem: this is unworkable unless the SMTP server already knows the
recipient's capabilities, since without knowing those capabilities
the SMTP server cannot be sure that it can convert the message to
something the recipient can accept.  and if the SMTP server already
knows the recipient's capabilities, there is no need for it to lie 
about them.

problem: this introduces the possibility of multiple conversions.  multiple 
conversions can cause considerable lossage due to interference between the 
conversions - e.g. aliasing when audio or image sampling rates are changed, 
interference caused when artifacts in the the output of one audio codec 
appear in  the input of another.

problem: this provision almost certainly allows SMTPs to make assertions
about recipient capabilities without the permission of the recipient.

> B.2  Some End-to-End Scenarios
> 
>      This section suggests methods of using ESMTP Conneg for achieving 
>      an end-to-end service that uses knowledge of recipient 
>      capabilities for modifying the content or the handling of the 
>      content.  This section is intended only to explore possible 
>      scenarios.  Others are feasible and likely. The choice of scenario 
>      to support will depend upon particular service policies chosen for 
>      a relay.  This specification provides no constraint or guidance 
>      about which policies to choose.
> 
>      (1) Basic Tranfer betweeen Organizations
>      
>      Again noting that alternative configurations and support 
>      environments are permitted, a simple example of combining the two 
>      modes of receiving server style can be helpful.  The scenario to 
>      consider is sending from one organization's email system to 
>      another organization's email system, across the Internet.  
>      
>      This scenario presumes two, independent email services, one for 
>      each organization.  In this simple example, the user has a local 
>      ESMTP server and it talks with the organization's Internet email 
>      gateway.  Hence there are four ESMTP servers.  The originating 
>      server takes the message from the originating user.  It then 
>      relays the message to the originating organization's ESMTP 
>      gateway.  This gateway, in turn, relays the message to the 
>      destination organization's gateway, which finally delivers it to 
>      the recipient's server.
>      
>      In this simple example, the originating ESMTP server includes a 
>      CONNEG option on the RCPT command 

problem: for the originating server to include CONNEG on outgoing mail
without any per-message authorization from the sender presumes that it 
is never desirable to deliver mail unless the recipient capabilities are 
known.  this is a radical departure from current practice and is probably 
not feasible.  

if the example assumes that the organizations have a mutual agreement to 
support CONNEG for all recipients and the outgoing server only asserts 
CONNEG when sending to recipients at that organization, this needs to be
stated explicitly.

>      to the originating 
>      organization's Internet gateway and the gateway always responds 
>      that it supports the highest capabilities.  It is then given the 
>      most capable form of the content.
>      
>      The originating gateway then performs a RCPT CONNEG exchange with 
>      the destination organization's gateway.  This gateway can operate 
>      in the same "Ensure Conversion" mode as the originating gateway, 

problem: no it cannot, at least not without more assumptions than are stated.

>      or it can perform a real-time query about the addressee's 
>      capabilities.  The former mode defers resolving recipient 
>      capabilities until the final step.  The real-time query requires 
>      both that the query mechanism be timely, in order to avoid RCPT 
>      response delays, and that it be accurate, with correct information 
>      about the addressee.
>      
>      The final exchange is, of course, "direct".  The recipient's ESMTP 
>      server is presumed to have easy access to the necessary 
>      capabilities information.
> 
>      (2) Integration with Multipart/Alternative
>      
>      Basic support for Conneg will typically involve a Conneg client 
>      that has a single, high-quality version of content and then maps 
>      is "down" to the best quality that can be supported by the 
>      responding Conneg server.  That is, the sending system will do a 
>      conversion.  In another mode of operation the Conneg client 
>      already holds a number of different mappings of content at 
>      different levels of quality.  The client will use the Conneg 
>      response to choose the content quality that is "closest" to the
>       capabilities of the receiver.
>      
>      MIME's Multipart/Alternative provides a means for a content 
>      originator to send a fixed set of content quality choices.  An 
>      ESMTP relay in the sequence can then choose to use Conneg as a 
>      means of selecting one of the alternatives to transmit, rather 
>      than transmitting the full set.  Such a scenario is particularly 
>      useful when the communication path changes from high-bandwidth to 
>      low.
>      
>      (3) Content Staging on Retrieval Server
>      
>      Rather than converting content to a lower quality, an ESMTP Conneg 
>      client might choose to use a Conneg response for choosing not to 
>      send the content directly.  It might, instead, remove the content 
>      from the message and store the content on a retrieval server (for 
>      access through HTTP, FTP or the like.)  It can then put a citation 
>      into the message, which points to the stored content.
>      
>      For the retrieval server, access security, as well as life-time 
>      management of the content on the retrieval server, should be 
>      considered. A full discussion of these considerations is out of 
>      the scope of this memo."
> 
>      (4) Alternate Addressing
>      
>      If an ESMTP Conneg client has access to multiple addresses for the 
>      same recipient, it might use Conneg to determine which is most 
>      capabable for particular content and send the content to that one.
>      
>      This scenario requires a number of infrastucture features for 
>      which there currently are no standards.

Date: Mon, 05 Aug 2002 08:47:53 -0700
To: [email-masked], [email-masked]
From: Dave Crocker <[email-masked]>
Subject: Fwd: I-D ACTION:draft-ietf-fax-esmtp-conneg-03.txt
Sender: [email-masked]

Folks,

The SMTP Conneg draft has been revised, with the primary change being 
addition of two Appendices.  One discusses use of the option when there are 
multiple recipients.  The other discusses use of the option when there is 
relaying.

d/

>A New Internet-Draft is available from the on-line Internet-Drafts 
>directories.
>This draft is a work item of the Internet Fax Working Group of the IETF.
>
>         Title           : SMTP Service Extension for Content Negotiation
>         Author(s)       : K. Toyoda, D. Crocker
>         Filename        : draft-ietf-fax-esmtp-conneg-03.txt
>         Pages           :
>         Date            : 02-Aug-02
>
>This document defines a content negotiation service
>extension for SMTP [ESMTP1, ESMTP2] whereby an SMTP
>client may request information about content
>capabilities of the target device or system that is
>serviced by an SMTP server.  The SMTP server may report
>the target's content capabilities back to the client.
>This process emulates a classic facsimile start-of-
>session capabilities negotiation, although it can be used
>for a broad range of email-based scenarios.  This service
>extension is primarily intended for 'direct', one-hop,
>originator/recipient SMTP transfers, although relayed
>scenarios through multiple SMTP servers are permitted.
>
>A URL for this Internet-Draft is:
>http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-03.txt

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850