Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
From: [email-masked]
To: "Robert A. Rosenberg" <[email-masked]>
Cc: Keith Moore <[email-masked]>, John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Date: Wed, 17 Jul 2002 20:15:06 -0700 (PDT)
List-Id: <ietf-smtp.imc.org>

> At 01:39 -0700 on 07/15/2002, [email-masked] wrote about Re:
> Keywords for "SMTP Service Extension for Content Ne:

> >  > Frankly the method of returning capabilities in a bounce
> >>  message seems more effective and more reliable because it doesn't rely on
> >>  intermedaries.
> >
> >The problem I see with depending on notifications is another one of those
> >"looks good on paper" things: For whatever reason, support for sophisticated
> >handling of notifications has been slow to materialize in user agents.

> Agreed. OTOH, since the client is going to have to handle CONNEG in
> any case, adding a notification parser to the mix as part of the
> CONNEG implementation is a different issue since it is not adding the
> notification parser to existent support but adding new support that
> INCLUDES a notification parser as part of its implementation
> requirements. Note that I am not advocating or rejecting a
> notification parser solution but only pointing out what I perceive as
> a fallacy in your analysis of the issue.

I'm not entirely clear on what you're driving at here, but regardless: My point
was and is that resistance against doing certain things in certain ways can be
a lot higher than you'd ever expect it to be, to the point where things just
don't get implemented or deployed.

There are all sorts of good reasons to parse DSNs. But despite the multiple
advantages this capability provides it hasn't materialized in a lot of places
where you'd expect it to. Indeed, what has happened in some of those cases is
that some quite extraordinary kludges have been adopted instead.

I'm guilty of this myself to some extent, I guess: MIXER-compliant X.400
gateways require DSN parsing, but back when I coded such things this capability
was the absolute last thing I added.

I still cannot explain the reluctance. This just isn't rocket science.

				Ned

Date: Wed, 17 Jul 2002 22:35:21 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Cc: Keith Moore <[email-masked]>, [email-masked], [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 01:39 -0700 on 07/15/2002, [email-masked] wrote about Re: 
Keywords for "SMTP Service Extension for Content Ne:

>  > Frankly the method of returning capabilities in a bounce
>>  message seems more effective and more reliable because it doesn't rely on
>>  intermedaries.
>
>The problem I see with depending on notifications is another one of those
>"looks good on paper" things: For whatever reason, support for sophisticated
>handling of notifications has been slow to materialize in user agents.

Agreed. OTOH, since the client is going to have to handle CONNEG in 
any case, adding a notification parser to the mix as part of the 
CONNEG implementation is a different issue since it is not adding the 
notification parser to existent support but adding new support that 
INCLUDES a notification parser as part of its implementation 
requirements. Note that I am not advocating or rejecting a 
notification parser solution but only pointing out what I perceive as 
a fallacy in your analysis of the issue.

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked]
Subject: Re: conneg for better mailing list formats 
Date: Tue, 16 Jul 2002 11:31:59 -0400
Sender: [email-masked]

> References to the relationship between multipart/alternative and use of
> Conneg have triggered a possibility that I'd like to explore:
> 
> We currently have many mailing lists that permit html content and many that
> do not.  Most of the time, the enforcement of that rule is entirely social,
> ie, non-computer.  There is no protocol support for telling senders what
> content format is preferred or what format(s) are prohibited.
> 
> Conneg can fix that.  Conneg permits the recipient system to state and
> enforce content format "permissions".

I assume you're talking about the proposed CONNEG SMTP extension rather
than just the conneg format for describing recipient capabilities. 

Conneg might be able to fix that problem.  But first it is necessary to
fix conneg so that the information can be obtained by the sender's UA
(not the sender's MTA), and not just by the MTA that eventually relays the 
message to the recipient's MX.  Otherwise, it is expecting an unrelated
party to perform a potentially arbitrary and unspecified conversion.

For example, in the case of html to text, I've seen this done really well
and I've seen it done really badly - so badly that the resulting text was
almost unreadable.  I don't think we want to give license to intermediate
MTAs to perform arbitrary conversions to messages, particularly in the
absence of any explicit authorization from senders.

Keith

To: Dave Crocker <[email-masked]>
Cc: [email-masked]
Subject: Re: conneg for better mailing list formats 
From: [email-masked]
Date: Tue, 16 Jul 2002 00:42:42 -0400
Sender: [email-masked]

--==_Exmh_2067402031P
Content-Type: text/plain; charset=us-ascii

On Tue, 16 Jul 2002 13:07:55 +0900, Dave Crocker <[email-masked]>  said:

> The originator sends multipart/alternative content, such as html and 
> text/plain.  The sending MTA invokes conneg and sends only the content 
> format that the receiving mailing list server is willing to accept.

This becomes bletcherously messy if you also want to support multipart/signed.

Rather than have this:

multipart/signed
   multipart/alternative
      text/html
      text/plain
   application/signature

you need to do this:

multipart/alternative
   multipart/signed
      text/plain
      application/signature
   multipart/signed
      text/html
      application/signature

(I'll overlook the fun if it's a multipart/encrypted to some previously
distributed key (software distribution, etc), and the fact that this means
you have to poke around *two* levels down to see which alternative you want).

This gets *particularly* interesting if posting to two lists that have
distinct policies - whereas in the first case you have one signature that
binds the two text/ parts together as an entity, in the second case you have
two separate signatures over different objects.  This leads to an interesting
"talking out both sides of your mouth" attack, where you can appear to post
the same thing to different lists, but different parts get rejected by
different lists leaving each constituency with a different signed object that
they think is the SAME as the other list got. (Imagine a mailing list of
shareholders that accepts HTML, and a mailing list of upper management that
accepts text/plain, and run with that a bit.. ;)

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_2067402031P
Content-Type: application/pgp-signature

--==_Exmh_2067402031P--

Date: Tue, 16 Jul 2002 13:07:55 +0900
To: [email-masked]
From: Dave Crocker <[email-masked]>
Subject: conneg for better mailing list formats
Sender: [email-masked]

References to the relationship between multipart/alternative and use of 
Conneg have triggered a possibility that I'd like to explore:

We currently have many mailing lists that permit html content and many that 
do not.  Most of the time, the enforcement of that rule is entirely social, 
ie, non-computer.  There is no protocol support for telling senders what 
content format is preferred or what format(s) are prohibited.

Conneg can fix that.  Conneg permits the recipient system to state and 
enforce content format "permissions".

Even better, this can combine with multipart/alternative very constructively.

The originator sends multipart/alternative content, such as html and 
text/plain.  The sending MTA invokes conneg and sends only the content 
format that the receiving mailing list server is willing to accept.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Mon, 15 Jul 2002 08:45:37 -0400
Sender: [email-masked]

Regarding caching: I see no problem with MTAs that obtain conneg strings 
from a database caching those lookups.  OTOH I think caching is extremely 
dubious as a mechanism for propagating conneg strings in the direction
from the recipient's MTA to the sender's UA, at the rate of one hop per 
email message transmitted.

> The bottom line is that we're dealing with partial solutions in every case
> here. This makes it easy to criticize any given scheme. IMO this also makes it
> very much the WG's call as to how to proceed.

I think this is quite a reasonable statement if we are talking about a
fax-specific extension; less reasonable if we are talking about a generic
SMTP extension.

And regardless, I think more discussion is needed in the document about 
which parties have the responsibility for conversion (presumably the
sender's UA), and which parties have responsibility for providing
recipient information (presumably *all* of the listed MXes for that domain) -
this in addition to addressing several other issues that have been raised.

Keith

Date: Mon, 15 Jul 2002 01:39:06 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: Keith Moore <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > The case I was thinking of was the one where caching of conneg information by
> > intermediaries eventually brings the information back to the server adjacent to
> > the originating client. So the transformation is still done on the originating
> > client or on a server very close to it.

> I think it's a stretch to believe that caches are going to solve this problem.
> It's true that if A keeps sending messages to B then eventually A's MTA will
> be able to cache B's recipient information.   But this assumes that the
> intervening MTAs support CONNEG (and cache CONNEG information) - which itself
> seems like a stretch because, as you say, there's no business case for them
> doing conversion.

What's a stretch is your assumption that caching implies conversion
capabilities. They are completely different activities. Conversion by an
intermediary is a nasty mess, whereas caching conneg information is trivial.

> It's also a stretch to believe that this is going to work well
> for arbitrary pairs of senders and recipients - it might work for those pairs who
> exchange lots of messages, not so well for those pairs that occasionally exchange
> messages.

That's certainly a possible outcome. But you can game this in all sorts of ways
and reach various different conclusions.

> And of course there's the problem that CONNEG information can be
> expected to change over time, but there's no mechanism in the current proposal
> to timeout caches.

I've already indicated that I believe conneg information caches are something
that needs to be discussed.

> Frankly the method of returning capabilities in a bounce
> message seems more effective and more reliable because it doesn't rely on
> intermedaries.

The problem I see with depending on notifications is another one of those
"looks good on paper" things: For whatever reason, support for sophisticated
handling of notifications has been slow to materialize in user agents.

> The only way I see to make the CONNEG SMTP proposal work end-to-end is for
> all of the intermediate MTAs (those that don't have direct knowledge of the
> recipient capabilities) to open up SMTP connections to the next hop *in real time* -
> so when the sender asserts CONNEG in a RCPT command then the MTA has to stop
> acting like a relayer and start acting like a proxy - finding a MTA for the
> next hop, opening up a connection, sending EHLO and seeing if it supports
> CONNEG, sending MAIL and all previous RCPTs, and finally sending the RCPT
> that has the CONNEG option, just so that it can propagate the CONNEG response
> back to the sender's UA.  Even then it will be very slow, it introduces
> race conditions due to timeouts, and it still requires all of the intermediate
> MTAs to support CONNEG in proxy mode.

This scheme was actually discussed and I believe rejected in the FAX WG
some time ago. Regardless, I disagree with your assessment that this is
a requirement for the present proposal to be effective.

> I haven't followed the fax discussions, but weren't the fax people looking for
> a mechanism that would work predictably and reliably - e.g. if sender and
> recipient both have (say) high-res capability then the document is always
> transferred in high-res - and a mechanism that would give the sender the option
> of requiring that the recipient have certain minimum capabilities before sending
> the message?  (e.g. don't bother sending if recipient doesn't have color?)
> I just don't see a way to make that work through the existing SMTP infrastructure.

I'm sure there are plenty of places in the present infrastructure where this
scheme won't work well. I'm also sure there are plenty of places where it will.
My assessment of the notification scheme is roughly the same. And experience
should inform us that our ability to predict what portions of the
infrastructure will be upgraded is pretty limited.

The bottom line is that we're dealing with partial solutions in every case
here. This makes it easy to criticize any given scheme. IMO this also makes it
very much the WG's call as to how to proceed.

				Ned

Date: Mon, 15 Jul 2002 00:06:57 -0400
To: [email-masked], [email-masked], [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content   Negotiation"
Sender: [email-masked]

At 06:50 +0900 on 07/14/2002, Dave Crocker wrote about Re: Keywords 
for "SMTP Service Extension for Content  N:

>  >if the message being transmitted is a fax and all recipients are capable
>>of fax,
>>then perhaps it is acceptable to downgrade the message to a lower quality fax
>>format which is acceptable to all.
>
>Ever since multi-recipient addressing was introduced, we have had
>downgrading, by virtue of users having to choose the least common
>denominator approach to sending attachments.
>
>So the fact that this new mechanism allows that downgrading to happen later
>in the transmission sequence should not confuse anyone into thinking that
>downgrading is a new or unusual requirement.

?

There are ways of delaying the downgrading until the recipient has 
actually received the content (and thus gets the message they would 
have received if they were the sole addressee). One that is obvious 
is the use of Multi-Part/Alternative. The sender creates the 
"downgraded" versions that are acceptable/required and hands the full 
set off to EACH OF THE RECIPIENTS. Each one then selects the version 
THEY WANT from the supplied versions without any heavy-handed LCD 
interference by an intermediary just due to one of the addressees 
being unwilling/unable to accept one of the "Better" versions.

I differentiate this from the type of on-the-fly downgrading that 
occurs with 8-Bit->QP transforms during relay since the original 
message CONTENT gets delivered even after the transform (so long as 
the receiving MUA understands QP) unlike the LCD transform where the 
original content is replaced with a lower quality LCD version.

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: John C Klensin <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sun, 14 Jul 2002 22:15:06 -0400
Sender: [email-masked]

> At 02:22 PM 7/14/2002 -0400, John C Klensin wrote:
> >First of all, I don't find, anywhere in the document, a statement
> >that the originator, in attempting to use CONNEG, is authorizing
> >any intermediate MTA that might touch the message to alter its
> >content.
> 
> For a specification that is designed to permit a sending agent to know how 
> to tailor the form of a content, it is difficult to imagine what benefit 
> there can be in having that specification  contain a statement saying that 
> an implementation is "authorized" to tailor the form of that content.

Dave, that may be what your specification is *designed* to do.  It is
not what your specification *does*, except in the rare case where the
sender's UA is able to communicate directly with the recipient's MX.
And the sender's UA is the only "agent" that is typically under control of
the sender.

> >   Nor do I find in the "security considerations" an
> >explanation of that authorization and the associated risks.
> 
> You clearly have a better sense of this issue than I do, since I do not see 
> that there is a problem.
> 
> So, feel free to offer some text.

How about this:

This extension MUST NOT be used by any SMTP client which is not authorized
to perform conversion by either the sender or the recipient of the message.

> >   We have
> >typically tried to keep that transmission stream as opaque to
> >message content as possible.
> 
> Yup.  It sure is scary to try to do new things.
> 
> Except when there is a well-established basis for doing them.
>
> I am not inclined to dismiss that established basis solely because it came 
> from a non-IETF effort, particularly given the scale of success of that 
> prior work.

Nor, apparently, are you inclined to cite that established basis.
Perhaps it is because it is not applicable?

> >But, unless I read the above incorrectly, you are suggesting that
> >         * Hence intermediate/relay MTAs have the right/authority
> >         to make content changes based on their inferences about
> >         what the receiver wants.
> 
> Let's try to stay on the current topic.  

Yes, lets.  Why don't you actually address these technical criticisms
of your proposal rather than disingenenously pretending that they don't 
apply?

> What I mean is very simple:
> 
>          The  behavior that is specified in esmtp/conneg is a careful 
> increment in functionality and is based on extensive prior experience and 
> is known to be useful.

Dave, unless you can actually cite relevant experience in the field of
multi-hop, store-and-forward multimedia messaging, I see no reason why we 
should not conclude that you have no basis whatsoever for claiming that your 
current proposal is worth any more consideration.  

Keith

Date: Mon, 15 Jul 2002 09:56:00 +0900
To: John C Klensin <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 02:22 PM 7/14/2002 -0400, John C Klensin wrote:
>First of all, I don't find, anywhere in the document, a statement
>that the originator, in attempting to use CONNEG, is authorizing
>any intermediate MTA that might touch the message to alter its
>content.

For a specification that is designed to permit a sending agent to know how 
to tailor the form of a content, it is difficult to imagine what benefit 
there can be in having that specification  contain a statement saying that 
an implementation is "authorized" to tailor the form of that content.

>   Nor do I find in the "security considerations" an
>explanation of that authorization and the associated risks.

You clearly have a better sense of this issue than I do, since I do not see 
that there is a problem.

So, feel free to offer some text.

>   We have
>typically tried to keep that transmission stream as opaque to
>message content as possible.

Yup.  It sure is scary to try to do new things.

Except when there is a well-established basis for doing them.

I am not inclined to dismiss that established basis solely because it came 
from a non-IETF effort, particularly given the scale of success of that 
prior work.

>But, unless I read the above incorrectly, you are suggesting that
>         * Hence intermediate/relay MTAs have the right/authority
>         to make content changes based on their inferences about
>         what the receiver wants.

Let's try to stay on the current topic.  Arguing about generic principles 
and then trying to dismiss current work based on some subtle variation in 
interpretations of that abstract discussion is not going to be productive.

You might want to review prior work on interprocess communication, which 
explored trade-offs between a "sender makes right" model and a "receiver 
makes right system".

The significant difference between esmtp/connect and multipart/alternative 
is exactly this distinction.

(Hint:  sender makes right has nearly always been the preferred IETF 
choice.  esmtp/conneg is sender makes right.)

>Could you explain what you did mean?

What I mean is very simple:

         The  behavior that is specified in esmtp/conneg is a careful 
increment in functionality and is based on extensive prior experience and 
is known to be useful.

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

Date: Sun, 14 Jul 2002 13:20:41 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: [email-masked]
Cc: [email-masked], [email-masked]
Sender: [email-masked]

My apologies, I intended to edit the cc list but neglected to do so.

				Ned

> Private note since this is now entirely off topic...

> > > I think this is a fine piece of theoretical analysis which unfortunately has no
> > > applicability to the real world. In the real world directory/database systems
> > > that let sites provide Internet-wide access to information about their users do
> > > not deploy.
> > >
> > > I'm not sure why this is the case, and indeed I'm fairly sure that there is no
> > > good reason for it, but a vast tract of experience tells me it for-sure is the
> > > case. (Now is not the time to get into whys in more detail.)

> > I'll just add 2 words to the "why" and leave it at that:

> > "datamining" "spammers".

> This is the opposite of an explanation, I'm afraid. Techniques for protecting
> against datamining are well known and widely implemented. In fact it is almost
> certainly easier to implement them in LDAP than in, say, a Web interface. Yet
> the former is rarely done, the latter pretty common.

> 				Ned

Date: Sun, 14 Jul 2002 12:48:22 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

Private note since this is now entirely off topic...

> > I think this is a fine piece of theoretical analysis which unfortunately has no
> > applicability to the real world. In the real world directory/database systems
> > that let sites provide Internet-wide access to information about their users do
> > not deploy.
> >
> > I'm not sure why this is the case, and indeed I'm fairly sure that there is no
> > good reason for it, but a vast tract of experience tells me it for-sure is the
> > case. (Now is not the time to get into whys in more detail.)

> I'll just add 2 words to the "why" and leave it at that:

> "datamining" "spammers".

This is the opposite of an explanation, I'm afraid. Techniques for protecting
against datamining are well known and widely implemented. In fact it is almost
certainly easier to implement them in LDAP than in, say, a Web interface. Yet
the former is rarely done, the latter pretty common.

				Ned

To: Dave Crocker <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
From: [email-masked]
Date: Sun, 14 Jul 2002 15:28:15 -0400
Sender: [email-masked]

--==_Exmh_962177079P
Content-Type: text/plain; charset=us-ascii

On Sun, 14 Jul 2002 06:50:34 +0900, Dave Crocker said:

> Ever since multi-recipient addressing was introduced, we have had 
> downgrading, by virtue of users having to choose the least common 
> denominator approach to sending attachments.
> 
> So the fact that this new mechanism allows that downgrading to happen later 
> in the transmission sequence should not confuse anyone into thinking that 
> downgrading is a new or unusual requirement.

My MUA is authorized to make decisions regarding downgrading and choice of
C-T-Es for attachments.  It listens to my choices regarding base64 versus QP,
For the most part, said conversions are lossless (and a few, such as
PGP-signing are even information-added), but I have authorized it to do lossy
conversions under a a few sets of circumstances, and I *know* about my MUA's
actions *before* I hit 'send'.  This is not news, and hasn't been for at least
a decade.

On the other hand, once my MUA hands it to my MTA, *I DO NOT EXPECT THE MTA TO
PERFORM LOSSY CONVERSIONS*. Period. End Of Discussion.

To quote the 'Security Considerations' of the original draft:

     A man-in-the-middle attack might change the capabilities reported 
     for a given recipient. For example: Suppose the sender knows the 
     recipient has the ability to view color documents so they mark 
     some things in red in what is otherwise a black and white 
     document. But someone interferes with the returned capabilities, 
     indicating that the recipient only supports black and white. The 
     document is duly downgraded, with the result that the recipient 
     doesn't see what the sender marked. 

Hey hey... What happens if this happens by *ACCIDENT*?  Currently, if I
think the recipient has color, and I do this, the recipient hopefully
gets at least a *warning* "trying to open a color image, converting to
gray scale", and can notify me that there was a problem.  However, the spec
as written doesn't even require that the MTA add a 'X-Downgraded-by:' header,
and the recipient has *no clue* that what he received wasn't what I sent.

And that, Dave, *IS* a "new and unusual" state of affairs.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_962177079P
Content-Type: application/pgp-signature

--==_Exmh_962177079P--

To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
From: [email-masked]
Date: Sun, 14 Jul 2002 14:54:11 -0400
Sender: [email-masked]

--==_Exmh_847081899P
Content-Type: text/plain; charset=us-ascii

On Sat, 13 Jul 2002 08:55:41 PDT, [email-masked] said:

> I think this is a fine piece of theoretical analysis which unfortunately has no
> applicability to the real world. In the real world directory/database systems
> that let sites provide Internet-wide access to information about their users do
> not deploy.
> 
> I'm not sure why this is the case, and indeed I'm fairly sure that there is no
> good reason for it, but a vast tract of experience tells me it for-sure is the
> case. (Now is not the time to get into whys in more detail.)

I'll just add 2 words to the "why" and leave it at that:

"datamining" "spammers".
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_847081899P
Content-Type: application/pgp-signature

--==_Exmh_847081899P--

Date: Sun, 14 Jul 2002 14:22:49 -0400
From: John C Klensin <[email-masked]>
To: Dave Crocker <[email-masked]>, Keith Moore <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Sender: [email-masked]

--On Sunday, 14 July, 2002 06:50 +0900 Dave Crocker
<[email-masked]> wrote:

>...
> Ever since multi-recipient addressing was introduced, we have
> had downgrading, by virtue of users having to choose the least
> common denominator approach to sending attachments.
> 
> So the fact that this new mechanism allows that downgrading to
> happen later in the transmission sequence should not confuse
> anyone into thinking that downgrading is a new or unusual
> requirement.
>...

Dave, I want to be sure I understand what you are saying here,
because, if I am reading it correctly, I find it astonishing.

First of all, I don't find, anywhere in the document, a statement
that the originator, in attempting to use CONNEG, is authorizing
any intermediate MTA that might touch the message to alter its
content.  Nor do I find in the "security considerations" an
explanation of that authorization and the associated risks.  If
such text is actually there, then some of what follows may be
irrelevant (although it wouldn't convince me it is a good idea).

Certainly, originating MUAs downgrade and change formats based on
what they, on behalf of the user, believe the recipient will be
able to process and have done that for a long time.  But they do
so before the message enters a transmission stream.  We have
typically tried to keep that transmission stream as opaque to
message content as possible.   In recent years, we have made some
formal allowances for fixups by originating MTA for
low-capability MUAs, but we have rather carefully delimited them
(see sections 3.7, 6.1, and 6.3 of RFC 2821 (the references to
section 2.4.1 are incorrect and should refer to 6.3)).

We also permit downgrading through ESTMP options like 8BITMIME,
which is quite explicit that an initiator who asks for this
feature is agreeing to a (very specific, known-result) downgrade
by relaying systems.

But, unless I read the above incorrectly, you are suggesting that

	* Since the originating MUA clearly has the ability and
	authority to alter content to reflect its assumptions
	about receiver capability (before the content is handed
	off to an SMTP server and placed into the transport
	stream)
	
	* Other alterations to content ("downgrades") are just
	moving the process "later in the transmission sequence"
	
	* Hence intermediate/relay MTAs have the right/authority
	to make content changes based on their inferences about
	what the receiver wants.

I hope that isn't what you meant, because it is pretty scary.  It
also creates a contradiction between this draft and RFC 2821 (as
well as the way most of us have read 821 for years), raises a
collection of security issues (e.g., in-transit downgrading would
seem to me to be quite hard on signed content), and so on.

Could you explain what you did mean?

   john

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sun, 14 Jul 2002 13:12:53 -0400
Sender: [email-masked]

> Ever since multi-recipient addressing was introduced, we have had 
> downgrading, by virtue of users having to choose the least common 
> denominator approach to sending attachments.
> 
> So the fact that this new mechanism allows that downgrading to happen later 
> in the transmission sequence should not confuse anyone into thinking that 
> downgrading is a new or unusual requirement.

Dave, it's a gross exaggeration to claim that the fact that human senders have
long found it advantageous to send messages to multiple recipients using a
common format of their choosing, that this is a good justification for 
standardizing the idea that MTAs that are responsible to neither the sender 
nor the recipient should be able to "downgrade" a message, using arbitrary
and unspecified methods, without any knowledge of the consequence of applying
conversion to the content, and without any explicit authorization.

> >the use of the RCPT response to tell the client of the recipient's 
> >capabilities
> >is odd because it asserts that the client intends to deliver a message to
> >each of several recipients before the client actually knows whether it can
> >deliver the same version of the message to each recipient, or whether it can
> >deliver any version of that message to the recipient.
> 
> "or whether it can"???  There has been no posting that provides a practical 
> basis for such a conclusion.

Yes there has, but perhaps you missed it.  Regardless you should be able to
figure out that your proposal provides no assurance that downgrading is even 
possible.  just to take a trivial example - the message might contain only 
audio while one recipient accepts only fax.  Another example - the recipient 
accepts only HTML and plain text, the message is in microsoft word, and while 
conversion is possible the MTA doesn't know how to do it.  

so yes, "whether it can" is entirely appropriate wording.

> Therefore, any implications of this unproved -- and frankly 
> counter-intuitive and highly unlikely -- assessment has nothing to do with 
> the current specification.

Dave, the most charitable thing I can say is that you must be living in a 
different universe.  You have consistently responded to valid, well-explained,
and defensible technical criticisms by flat-out stating that they don't 
exist or that they don't make any sense, without any defense.  You seem 
to think that everyone who uses this extension you're propsing will have
the same assumptions that you do, and adhered to a set of constraints
that exists only in your head.  And if you don't understand your own 
proposal well enough to make intelligent responses to technical criticism
then frankly you need to withdraw it and stop wasting our time.

Keith

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sun, 14 Jul 2002 12:40:58 -0400
Sender: [email-masked]

> I'm probably a bit biased since I wrote a bunch of the text, but I think we did
> a reasonable job of documenting the risks of active content back when MIME was
> first specified. And yet active content in the form of email viruses is one of
> the major operational problems, if not THE operational problem, we face in
> email today. We did our bit and nobody listened.

I have a slightly different take - we did our bit and the major operating
system vendor listened and irresponsibly chose to ignore us, putting their
desire to have their applications' data formats (including executable content)
transferred over email ahead of their customers' need for security.

I was explicitly told this by a former program manager for that major 
operating system vendor, though he didn't seem to think it was irresponsible.
He called it a marketing decision.

> > ah, but either the client has a direct connection to the MX or it has to
> > relay the message to some MTA which may or may not be under control of the
> > client.  so the relaying case introduces the same problem that OPES had
> > when it was first proposed - that of having an intermediary make transformations
> > to the content that aren't authorized by either party of the conversation.
> 
> The case I was thinking of was the one where caching of conneg information by
> intermediaries eventually brings the information back to the server adjacent to
> the originating client. So the transformation is still done on the originating
> client or on a server very close to it.

I think it's a stretch to believe that caches are going to solve this problem.
It's true that if A keeps sending messages to B then eventually A's MTA will 
be able to cache B's recipient information.   But this assumes that the 
intervening MTAs support CONNEG (and cache CONNEG information) - which itself 
seems like a stretch because, as you say, there's no business case for them
doing conversion.  It's also a stretch to believe that this is going to work well 
for arbitrary pairs of senders and recipients - it might work for those pairs who 
exchange lots of messages, not so well for those pairs that occasionally exchange 
messages.  And of course there's the problem that CONNEG information can be
expected to change over time, but there's no mechanism in the current proposal
to timeout caches.  Frankly the method of returning capabilities in a bounce
message seems more effective and more reliable because it doesn't rely on
intermedaries.

The only way I see to make the CONNEG SMTP proposal work end-to-end is for
all of the intermediate MTAs (those that don't have direct knowledge of the
recipient capabilities) to open up SMTP connections to the next hop *in real time* -
so when the sender asserts CONNEG in a RCPT command then the MTA has to stop
acting like a relayer and start acting like a proxy - finding a MTA for the
next hop, opening up a connection, sending EHLO and seeing if it supports
CONNEG, sending MAIL and all previous RCPTs, and finally sending the RCPT
that has the CONNEG option, just so that it can propagate the CONNEG response
back to the sender's UA.  Even then it will be very slow, it introduces
race conditions due to timeouts, and it still requires all of the intermediate 
MTAs to support CONNEG in proxy mode.

I haven't followed the fax discussions, but weren't the fax people looking for 
a mechanism that would work predictably and reliably - e.g. if sender and 
recipient both have (say) high-res capability then the document is always 
transferred in high-res - and a mechanism that would give the sender the option 
of requiring that the recipient have certain minimum capabilities before sending 
the message?  (e.g. don't bother sending if recipient doesn't have color?)
I just don't see a way to make that work through the existing SMTP infrastructure.

Keith 

Date: Sun, 14 Jul 2002 15:19:10 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Cc: [email-masked], [email-masked]
Sender: [email-masked]

At 08:55 AM 7/13/02 -0700, [email-masked] wrote:
>I think the availabilty of capabilities information has the potential to be as
>important a step as MIME was. As such, I don't begrudge healthy debate about
>how and where it is done. But let's not forget that the development of 
>MIME was
>informed by the catastrophic failure of various other schemes for doing
>multimedia messaging -- some of them more elegant than MIME -- leading to
>compromises in the design intended to insure deployability. I think the
>discussion of where and how to do content negotiation needs to be similarly
>informed by the catastrophic failure of Internet-wide directory services to
>deploy.

(Not to disagree with anything you say...)

One of the considerations informing the UA-driven negotiation design was 
the idea that information obtained from any source other than the final 
recipient was potentially tentative, and that some kind of closing of the 
loop was needed to make negotiation reliable.  The better the initial 
information that is available, the more efficient the 
negotiate-and-transfer process (one round trip).  But incorrect initial 
information doesn't cause the transfer to fail (except possibly for reasons 
of timing).

#g

-------------------
Graham Klyne
<[email-masked]>

Date: Sun, 14 Jul 2002 10:19:46 +0900
To: John C Klensin <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Cc: John C Klensin <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 05:11 PM 7/13/2002 -0400, John C Klensin wrote:
>You are, I think, equating packet fragmentation and reassembly to
>email messages.

No I was not.

I DID, however, note that the issue of fragmentation was a) not part of the 
definition of packet switching, and b) irrelevant to the use of the term in 
the current discussion.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

Date: Sun, 14 Jul 2002 06:50:34 +0900
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 09:51 AM 7/13/2002 -0400, Keith Moore wrote:

> > Moore-san prefers to have the capabilities query be in a separate
> > command.  For convenience, he has given this command a name, RCAP.  So, 
> the
> > name refers to an idea that Moore-san has.  There is no written
> > specification for that command.  In my opinion, the current specification
> > is satisfactory.
...

>if the message being transmitted is a fax and all recipients are capable 
>of fax,
>then perhaps it is acceptable to downgrade the message to a lower quality fax
>format which is acceptable to all.

Ever since multi-recipient addressing was introduced, we have had 
downgrading, by virtue of users having to choose the least common 
denominator approach to sending attachments.

So the fact that this new mechanism allows that downgrading to happen later 
in the transmission sequence should not confuse anyone into thinking that 
downgrading is a new or unusual requirement.

>the use of the RCPT response to tell the client of the recipient's 
>capabilities
>is odd because it asserts that the client intends to deliver a message to
>each of several recipients before the client actually knows whether it can
>deliver the same version of the message to each recipient, or whether it can
>deliver any version of that message to the recipient.

"or whether it can"???  There has been no posting that provides a practical 
basis for such a conclusion.

Therefore, any implications of this unproved -- and frankly 
counter-intuitive and highly unlikely -- assessment has nothing to do with 
the current specification.

>so as the proposal is written, any client which attempts to deliver a message
>to multiple recipients in a single transaction must be prepared to abort the
>current SMTP transaction (presumably by using RSET) after sending multiple 
>RCPTs,

It is NOT required to do that.  Nor is the ability to send an RSET during a 
session a big deal.  In fact, that's what the command is there for.

>it's been claimed  that using RCPT in this way is faster that using a 
>separate
>command,

Faster for single-recipient use, yes.  Faster for multi-recipient use, with 
common format, yes.

In both cases there are fewer commands and fewer round-trips than when 
using a separate capabilities-query language.  It does not take deep 
analysis to know that the current proposal is faster.

Faster for the particular scenario you used, with an RCPT? No.  That has 
not been claimed. Using RSET is expensive.  A design that calls for its use 
makes sense only if that use is expected to be atypical.

As has been discussed previously use of the current specification is based 
on the expectation that use of RSET will not be a typical.

>another problem with RCPT is that it expects the server to return status codes
>for error conditions that can only be distinguished on the client.

There is no such expectation:

         "If the SMTP server supports ENHANCEDSTATUSCODES [RFC1893], the..."

Apparently you missed the "If".

In any event, i have no idea why the codes are somehow uninterpretable by 
any system other than the client.

At 10:16 AM 7/13/2002 -0400, Keith Moore wrote:
> > The UA-based CONNEG mechanism does query/response between two UAs that are
> > connected only by the store-and-forward email service.  ...
>
>perhaps not, but once a message is relayed, the conversion (and the decision
>of whether to deliver the message) is no longer under control of the sender -

I have no idea what you mean by "under control of the sender" nor do I have 
any idea how the current proposal has any practical difference in control, 
from any other proposal or idea for content negotiation.

>actually the most likely scenario when using the conneg proposal with the
>current infrastructure is that the message will be delivered, without
>conversion, regardless of recipient capabilities.

And I have no idea where this assessment is based on, nor does it make any 
obvious sense.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

Date: Sun, 14 Jul 2002 06:14:20 +0900
To: [email-masked], [email-masked]
From: Dave Crocker <[email-masked]>
Subject: Piggybacked, adjunct services
Cc: [email-masked]
Sender: [email-masked]

Folks,

Ned's note explains some operational realities and some architectural 
implications.  His note  is the clearest, the most pragmatic, and the most 
compelling statement on this issue that I have seen.

It is easy to miss just how important his explanation is.  So I am taking 
the slightly theatrical step of re-posting it, in order to stress the 
importance of its insight:

At 08:55 AM 7/13/2002 -0700, [email-masked] wrote:

>In the real world directory/database systems
>that let sites provide Internet-wide access to information about their 
>users do
>not deploy.
>
>I'm not sure why this is the case, and indeed I'm fairly sure that there is no
>good reason for it, but a vast tract of experience tells me it for-sure is the
>case. (Now is not the time to get into whys in more detail.)
>
>The IETF has tried to tackle this problem several times, and indeed continues
>to do so now in the RESCAP WG. But I don't think anyone seriously believes 
>that
>even if RESCAP produces some specifications it is going to succeed at its
>intended scale. Frankly, the silence is pretty deafening over there.
>
>Lacking direct capabilities to find capabilties information necessarily leads
>to various piggybacking options, that is, deploy this facility on top of or
>inside other protocols. This has some of the same characteristics as the
>everything-over-HTTP approach, but with several important differences: 
>There is
>a close relationship between the piggybacked data and the intended purpose of
>the underlying protocols (a relationship that is often lacking with stuff
>travelling over HTTP) and the operational case for doing this is far more
>compelling (getting the OK to drill a hole in a firewall for a new protocol is
>triviality itself compared to the difficulty in setting up even restricted
>access to  directory information).
>
>I think the availabilty of capabilities information has the potential to be as
>important a step as MIME was. As such, I don't begrudge healthy debate about
>how and where it is done. But let's not forget that the development of 
>MIME was
>informed by the catastrophic failure of various other schemes for doing
>multimedia messaging -- some of them more elegant than MIME -- leading to
>compromises in the design intended to insure deployability. I think the
>discussion of where and how to do content negotiation needs to be similarly
>informed by the catastrophic failure of Internet-wide directory services to
>deploy.
>
>                                 Ned

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

Date: Sat, 13 Jul 2002 14:22:19 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: Keith Moore <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > > The relevance of HTTP is even less obvious to me because HTTP and directory
> > > services (LDAP or otherwise) tend to be used in very different ways.  I don't
> > > see many sites exporting their company directories via either HTTP or LDAP.
> >
> > Then you haven't looked very hard. The former is quite popular; the latter
> > practically nonexistant.  And the fact that the same information is exposed
> > either way yet one approach is preferred is directly relevant to the matter
> > at hand.

> what can I say - most of the company directories I've seen are not publically
> accessible.  I don't doubt that many companies use HTTP to provide internal
> access; whether they make them available to the general public (which is what
> I meant by "export") is a different question.

I speak as someone who used to sell and support software that did this sort of
thing, both via HTTP and LDAP. I saw plenty of examples of public export of
directory information, including but not limited to user information, through
HTTP, very few through LDAP.

I really can't explain the difference. My assessment was and is that a
carefully designed LDAP->LDAP gateway is likely to be more secure than a random
Web server lashed up to an LDAP server.

> as to whether rescap would be branded a directory - the number of people who
> kept insisting that it should be LDAP was fairly significant, indicating
> that many people couldn't readily distinguish the two problems.  this
> even though one of the major reasons for making rescap a separate protocol was
> to distinguish it from directories and specifically to have a mechanism
> tailor-made for exporting public information.  but frankly part of that problem
> was that some LDAP people were seeing rescap as a competitor/threat to LDAP.

As an aside, I think we actually foster this attitude given the degree and tone
of pushback that's often present in the IETF. For what may seem like the best
of reasons people often have to fight to get their work through the process.
But once that's done you're left with people trained to fight.

> as you say, it's not a sane world.  URNs have an even worse mis-perception
> problem, they always have, and it's not clear what to do about it.

> > You can talk till you're blue in the face about how this is capabilities
> > discovery, how that's intrinsicly different from a general directory
> > protocol, and so on and so forth. It doesn't change the perception that
> > a protocol intended solely for looking up stuff, no matter how innocuous
> > that stuff is, no matter how divorced from the cporpoate directory or
> > whatever it can be made, is somehow more problematic than piggybacking
> > the exact same information on top of other protocols.

> okay, I'll concede the point that there's this dark cloud over things
> that (in the minds of network administrators) resemble directories.
> I don't think that should by itself stop working groups from using things
> that resemble directories if that's the right technical solution,
> but it's useful to be aware of it.  it does I think illustrate that good
> technical design is not always sufficient - unless we somehow convey
> to users an understanding of new protocols, how they're meant to be used,
> the risks associated with using them, etc.  And we haven't been very
> effective at that since the 'net grew up...

Well, let's be fair here: We can lead the horse to water...

I'm probably a bit biased since I wrote a bunch of the text, but I think we did
a reasonable job of documenting the risks of active content back when MIME was
first specified. And yet active content in the form of email viruses is one of
the major operational problems, if not THE operational problem, we face in
email today. We did our bit and nobody listened.

It's quite a conundrum. Even when we do our job things still go wrong. Some
folks argue that things capable of abuse should not be standardized. But there
are substantive and possibly even greater risks associated with the lack of
standardization as well.

> > > So the argument that a facility that requires direct connection over port 25
> > > (from the origin to the MX server, not necessarily to the final MTA)
> > > in order to work properly (i.e. in order to let the sender control the
> > > conversion) will function both locally and globally seems a bit dubious.

> > I have pointed out previously that such direct connection isn't required by the
> > present proposal. So this is nothing but another strawman.

> ah, but either the client has a direct connection to the MX or it has to
> relay the message to some MTA which may or may not be under control of the
> client.  so the relaying case introduces the same problem that OPES had
> when it was first proposed - that of having an intermediary make transformations
> to the content that aren't authorized by either party of the conversation.

The case I was thinking of was the one where caching of conneg information by
intermediaries eventually brings the information back to the server adjacent to
the originating client. So the transformation is still done on the originating
client or on a server very close to it.

I also discussed the situation where conneg transformation is done by a central
intermediary in a previous message and concluded that there seemed to be
no business case for it.

I should also note that the class of tranformations envisioned by OPES
is considerably broader than the class of transformations envisioned
here.

				Ned

From: Keith Moore <[email-masked]>
To: John C Klensin <[email-masked]>
cc: Keith Moore <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: RCAP mini-spec (was: Re: Keywords for "SMTP Service Extension for Content Negotiation" ) 
Date: Sat, 13 Jul 2002 17:29:58 -0400
Sender: [email-masked]

> > 4. the RCAP command is acceptable at any time that MAIL or RCPT
> > would be acceptable.  clients MAY issue RCAP before or after
> > MAIL, and they MAY issue RCAP and RCPT commands in any order.
> > (though generally it makes more sense to issue RCAP for a
> > recipient before sending RCPT for that recipient)
> > The RCAP response MUST be the same regardless of whether the
> > command is issued before or after MAIL, or before or after RCPT.
> 
> Keith, I don't assume this would be heavily used, but, as with
> SIZE, I can imagine some sites being willing to accept more
> complex formats from some senders than from others.  Of course,
> such mechanisms are easily thwarted in the absence of SMTP
> authentication or trusted transport (and would not be available
> with some of the other proposals). But, unless you have a
> substantive reason for permitting RCAP before MAIL, it would seem
> desirable to force it to come after MAIL, thereby preserving this
> implementation option at substantially no cost.

my assumption was that the return address is essentially useless for 
determining the kind of content that would be acceptable to the 
recipient - partially since they are easily forged, and partially
since there are various reasons to use a different return address than
one's normal email address.  SMTP authentication credentials might 
be useful for determining the conneg string returned, but those 
would presuambly be presented before MAIL.

my reason for allowing RCAP before MAIL was to allow query-only
sessions without having to specify a return address that won't
get used anyway and just serve to slow the query down.

still I don't claim to have nailed down every detail - this is just
a "strawman" proposal so that people have something to compare
with the proposal that uses RCPT.

Keith

Date: Sat, 13 Jul 2002 17:11:13 -0400
From: John C Klensin <[email-masked]>
To: Dave Crocker <[email-masked]>, John C Klensin <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Sender: [email-masked]

Dave,

You are, I think, equating packet fragmentation and reassembly to
email messages.  IMO, that is bogus (and, as you point out, this
argument is decades old and I believe it has been bogus for at
least that long), unless you constrain those packets to reliable
delivery and permit them to be very large.  I.e., the only
reasonable comparison is to a collection of packets in a a
connection-mode (or other serialized and reliable) context.  It
was in that context that I used the term "message" -- typically a
multipacket entity in the TCP and IP layers just as an email
message consists of a series of SMTP turnarounds over TCP.  In
that context, I stand by my assertion.   And the reassembly of
packets (as distinct from data streams/ messages) in the ARPANet
is, I'm sure, fascinating to the few people on this distribution
who didn't know about it and care.

      john

--On Saturday, 13 July, 2002 17:43 +0900 Dave Crocker
<[email-masked]> wrote:

> 
> At 03:11 AM 7/13/2002 -0400, John C Klensin wrote:
>> --On Saturday, 13 July, 2002 12:04 +0900 Dave Crocker
>> > Internet mail is store-and-forward.  ... This is like packet 
>> switching.  Each intermediate system is "relaying" the email.
>> 
>> It is like packet switching except, of course, in the packet
>> case, each  packet is typically a fragment or component of a
>> message.
> 
> "Typically", perhaps, but it is not true for all packet
> switching systems and it is not part of the formal definition
> of packet-switching.
> 
> Packet switching is about independent addressing (and usually
> independent routing) and handling of discrete chunks of data.
> It is not about the relationships among that data.  To that
> extent, Internet mail is quite similar to lower-level packet
> switching.
> 
> (I say it is exactly the same, but therein lies a debate that
> some of us have enjoyed for a few years -- errr, decades -- and
> it is not pertinent to the specification under review.)
> 
> 
>>   The message, or data stream, is sent one packet at a time;
>>   it is not  completely assembled at each intermediate host
>> and then forwarded after  it is complete.
> 
> On the contrary, Arpanet IMP packet switching did fragmentation
> and reassembly in the net, not in the hosts.  (Any further
> debate on this issue might be interesting, but it, too, is not
> relevant to the current topic.)
> 
> 
>> > The primary difference between relaying versus direct is that
>> > relaying might take much longer.  Therefore the sender does
>> > not know when it will receive a response.
>> 
>> I would disagree.  The primary difference I see between
>> relaying and direct is that, in a direct arrangement
>> (characteristic of e.g., HTTP) the originating sending system
>> can ask the ultimate receiving one a question by, well, asking.
> 
> The UA-based CONNEG mechanism does query/response between two
> UAs that are connected only by the store-and-forward email
> service.  There is nothing that dictates that application-level
> query/response exchanges be conducted over connection-oriented,
> direct or real-time associations.
> 
> Just to underscore how fuzzy all this gets:  HTTP relies on the
> underlying store and forward of IP.  Email UAs rely on the
> underlying store and forward of SMTP.  And, of course, SMTP
> also relies on the store and forward of IP.
> 
> More importantly, these concerns are essential matters for
> successful implementation, deployment and use.  But they have
> nothing to do with core architectural principals.
> 
> With respect to direct vs. relay, the architectural difference
> between a UA-UA exchange and an HTTP exchange is remarkably
> small, especially given that HTTP usage pretends it has no
> context.  These differences get even smaller with the
> introduction of caching servers, and the like, because they
> introduce a very high degree of indirection between consumer
> and provider.
> 
> 
>> A footnote on the "find out from a database" issue.  The more I
>> think about it, the more I conclude that the database lookup
>> (LDAP or otherwise) is the only rational way of implementing
>> the desired functionality here in a case that might involve
>> relaying.
> 
> Yes, it is an appealing model, except that LDAP has yet to
> achieve Internet-scale deployment and use.  Consequently it is
> not wise to make a new service dependent on it.
> 
> It is also worth noting that such a model requires real-time
> connectivity.  An email-based mechanism can operate with
> asynchronous access.
> 
> It makes far more sense to postulate a range of choices --
> including directly asking the own of the information via a UA
> mechanism, as well as direct intermediaries via ESMTP/CONNEG --
> and incorporate LDAP when it starts being a serious global
> service.
> 
> 
>> To the extent that speedy and confirmed delivery are often
>> cited as important values in fax, the "maybe it will get
>> delivered as requested, ... is hard or
>> impossible for the originating (or early relay) MTA to detect"
>> alternatives are unreasonable.
> 
> To the extent that a higher degrees of certainty is required,
> then the operational components should be configured to provide
> it.  Using LDAP does not provide any guarantees of proper or
> speedy performance, any more than using UA- or SMTP-based
> content negotiations guarantees the lack.
> 
> d/
> 
> ----------
> Dave Crocker  <mailto:[email-masked]>
> Brandenburg InternetWorking  <http://www.brandenburg.com>
> tel +1.408.246.8253;  fax +1.408.850.1850
> 
> 
>  

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 17:11:24 -0400
Sender: [email-masked]

> > > > None of this experience seems particularly applicable to the question at hand,
> > > > where we are comparing two different mechanisms for exporting only a small
> > > > amount of information - SMTP, or some other protocol (of which LDAP is only a
> > > > token example).  If an enterprise is reluctant to export "information about
> > > > their users" in general, then they will be about as reluctant to do so via SMTP
> > > > as via some other protocol.
> > >
> > > I'm afraid this is another piece of theoretical considerations that fly in the
> > > face of actual experience. The obvious counterexample is HTTP.
> 
> > The relevance of HTTP is even less obvious to me because HTTP and directory
> > services (LDAP or otherwise) tend to be used in very different ways.  I don't
> > see many sites exporting their company directories via either HTTP or LDAP.
> 
> Then you haven't looked very hard. The former is quite popular; the latter
> practically nonexistant.  And the fact that the same information is exposed
> either way yet one approach is preferred is directly relevant to the matter
> at hand.

what can I say - most of the company directories I've seen are not publically
accessible.  I don't doubt that many companies use HTTP to provide internal 
access; whether they make them available to the general public (which is what
I meant by "export") is a different question.  

> > > I agree that predictability is nice, but that's not something any approach can
> > > ever guarantee. Indeed, I can argue that a directory approach will lead to less
> > > predictability, not more, if for no other reason than directory access will
> > > tend to work locally but not remotely. SMTP access with appropriate caching, on
> > > the other hand, has the potential to work both locally and globally.
> 
> > The last two statements seem fairly odd.  Maybe the problem is the use of the
> > word "directory" which isn't my favorite word for this anyway.  So let's not
> > call this a directory - let's call it a capability discovery protocol.  Now if
> > the capability discovery protocol is defined in such a way that in order for
> > your recipients to benefit from content negotiation you have to make it accessible
> > (just like if you want to be able to accept mail then you have to make port 25
> > accessible) then it seems fairly likely that sites that want to enable content
> > negotation for their recipients will do so.
> 
> My point was they may make it available internally but not externally. And
> using anything that remotely smells like a directory protocols makes this
> outcome far more likely.

ah.  okay, I think we're in agreement about that (and that was very similar
to my point about LDAP) - though whether this capabliity discovery protocol
smells like a directory protocol may to some degree be a matter of social 
engineering, aka marketing.

> > On the other hand, port 25 is one
> > of the few ports that is commonly blocked/filtered for outgoing traffic -
> > perhaps even more often (and with more drastic filtering) than port 80.
> 
> Keith, my point was and is that the minute you start talking about using a
> directory protocol -- and I insist on using the term because it's the directory
> aspect that's important -- it makes the whole thing problematic in a way that
> piggybacking the information on top of other type of protocols does not.

well, if *we* insist on calling it a directory protocol then of course people 
will confuse it with other things they call directories.  On the other
hand sites seem happy to consider exporting limited information via DNS,
and extensive information via HTTP, SOAP, .NET, or whatever.  So if the
marketing lingo has to be "recipient capability web service" or some such,
so be it.  (as long as we don't actually *use* SOAP or .NET... :)

as to whether rescap would be branded a directory - the number of people who 
kept insisting that it should be LDAP was fairly significant, indicating
that many people couldn't readily distinguish the two problems.  this
even though one of the major reasons for making rescap a separate protocol was 
to distinguish it from directories and specifically to have a mechanism 
tailor-made for exporting public information.  but frankly part of that problem
was that some LDAP people were seeing rescap as a competitor/threat to LDAP.

as you say, it's not a sane world.  URNs have an even worse mis-perception 
problem, they always have, and it's not clear what to do about it.

> You can talk till you're blue in the face about how this is capabilities
> discovery, how that's intrinsicly different from a general directory
> protocol, and so on and so forth. It doesn't change the perception that
> a protocol intended solely for looking up stuff, no matter how innocuous
> that stuff is, no matter how divorced from the cporpoate directory or
> whatever it can be made, is somehow more problematic than piggybacking
> the exact same information on top of other protocols.

okay, I'll concede the point that there's this dark cloud over things
that (in the minds of network administrators) resemble directories.  
I don't think that should by itself stop working groups from using things 
that resemble directories if that's the right technical solution,
but it's useful to be aware of it.  it does I think illustrate that good
technical design is not always sufficient - unless we somehow convey
to users an understanding of new protocols, how they're meant to be used,
the risks associated with using them, etc.  And we haven't been very
effective at that since the 'net grew up...

> > So the argument that a facility that requires direct connection over port 25
> > (from the origin to the MX server, not necessarily to the final MTA)
> > in order to work properly (i.e. in order to let the sender control the
> > conversion) will function both locally and globally seems a bit dubious.
> 
> I have pointed out previously that such direct connection isn't required by the
> present proposal. So this is nothing but another strawman.

ah, but either the client has a direct connection to the MX or it has to
relay the message to some MTA which may or may not be under control of the
client.  so the relaying case introduces the same problem that OPES had
when it was first proposed - that of having an intermediary make transformations
to the content that aren't authorized by either party of the conversation.

Keith

Date: Sat, 13 Jul 2002 16:42:21 -0400
From: John C Klensin <[email-masked]>
To: Keith Moore <[email-masked]>, Dave Crocker <[email-masked]>
cc: Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: RCAP mini-spec (was: Re: Keywords for "SMTP Service Extension for Content Negotiation" )
Sender: [email-masked]

--On Saturday, 13 July, 2002 09:51 -0400 Keith Moore
<[email-masked]> wrote:

>...
> 4. the RCAP command is acceptable at any time that MAIL or RCPT 
> would be acceptable.  clients MAY issue RCAP before or after
> MAIL, and they MAY issue RCAP and RCPT commands in any order.
> (though generally it makes more sense to issue RCAP for a 
> recipient before sending RCPT for that recipient)
> The RCAP response MUST be the same regardless of whether the
> command is issued before or after MAIL, or before or after RCPT.

Keith, I don't assume this would be heavily used, but, as with
SIZE, I can imagine some sites being willing to accept more
complex formats from some senders than from others.  Of course,
such mechanisms are easily thwarted in the absence of SMTP
authentication or trusted transport (and would not be available
with some of the other proposals). But, unless you have a
substantive reason for permitting RCAP before MAIL, it would seem
desirable to force it to come after MAIL, thereby preserving this
implementation option at substantially no cost.

      john

Date: Sat, 13 Jul 2002 16:33:03 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Sender: [email-masked]

Ned, Keith, and others,

I'm going to try to respond to several messages in one, please
bear with me.  Also, please note that I've had more than my share
of terrible ideas and it is quite possible that the notion of a
MUA getting information out of a directory about recipient
capabilities is another one.

--On Saturday, 13 July, 2002 08:55 -0700
[email-masked] wrote:

>> A footnote on the "find out from a database" issue.  The more I
>> think about it, the more I conclude that the database lookup
>> (LDAP or otherwise) is the only rational way of implementing
>> the desired functionality here in a case that might involve
>> relaying. To the extent that speedy and confirmed delivery are
>>...
>> But, if database lookup is the only plausible implementation
>> that is consistent with the underlying design requirements,
>> then we should not be looking at capabilities negotiation in
>> the SMTP stream.  Instead, we should be looking at a piece of
>> protocol by which the sending system (probably the MUA, rather
>>...
> I think this is a fine piece of theoretical analysis which
> unfortunately has no applicability to the real world. In the
> real world directory/database systems that let sites provide
> Internet-wide access to information about their users do not
> deploy.

Well, it depends on how important they are.  And, while it may be
tautological, the observation that these things have not deployed
is an indication that they haven't been important enough, given
deployment and maintenance costs and security risks.  Of course,
we do have one wildly-successful counterexample and, while I hate
to even mention it as a possibility, if this capability is
important enough, and it were the right answer (I have serious
doubts about both), nothing prevents us from thinking about a
mail-capabilities RR.  It would be a bad idea if only because the
capabilities are really an MUA issue and the DNS identifies MTAs
(today), but...

>...

> The IETF has tried to tackle this problem several times, and
> indeed continues to do so now in the RESCAP WG. But I don't
> think anyone seriously believes that even if RESCAP produces
> some specifications it is going to succeed at its intended
> scale. Frankly, the silence is pretty deafening over there.

I certainly agree.

> Lacking direct capabilities to find capabilties information
> necessarily leads to various piggybacking options, that is,
> deploy this facility on top of or inside other protocols. This
> has some of the same characteristics as the
> everything-over-HTTP approach, but with several important
> differences: There is a close relationship between the
> piggybacked data and the intended purpose of the underlying
> protocols (a relationship that is often lacking with stuff
> travelling over HTTP) and the operational case for doing this
> is far more compelling (getting the OK to drill a hole in a
> firewall for a new protocol is triviality itself compared to
> the difficulty in setting up even restricted access to
> directory information).

Yes, but as Keith points out (more politely), using the general
organizational directory for this purpose would be madness.  I
hope you didn't assume I was proposing it.   But the "piggyback
everything, rather than think about new protocols" approach
(which I don't think you believe in either, at least in the
limiting case) leads to a different sort of madness, one which
often manifests itself as "give up on all other application
protocols and just use HTTP" (or, more recently, SIP).

> I think the availabilty of capabilities information has the
> potential to be as important a step as MIME was. As such, I
> don't begrudge healthy debate about how and where it is done.
> But let's not forget that the development of MIME was informed
> by the catastrophic failure of various other schemes for doing
> multimedia messaging -- some of them more elegant than MIME --
> leading to compromises in the design intended to insure
> deployability. I think the discussion of where and how to do
> content negotiation needs to be similarly informed by the
> catastrophic failure of Internet-wide directory services to
> deploy.

That should clearly inform many of the things that we do, but we
should learn the right lessons.   Just as "firewalls get in the
way of anything new, therefore we should run all new protocols
and services over HTTP" is typically a wrong answer, developing a
paranoid fear of directory-like services. Prior attempts have
been over-ambitious for what they promised to deliver relative to
what people needed.  Or they have failed for other reasons.   But
it may be that the lesson is "more focused and less ambitious and
general purpose approaches", rather than discarding the
bathwater, the baby, and anything that vaguely resembles either
because, e.g., it is wet.

So, given that I'm already generating plausibly-bad ideas, think
about a strawman.  Suppose we said "if you want to use email
capabilities, you need to be able to establish a connection to
the final recipient" (whatever that means).  This is, of course,
a variation on the "no relay" scheme.   But then suppose that,
instead of piggybacking on top of SMTP, we required that the
destination host have an A RR, or a very specific and slightly
peculiar SRV RR, or something similar, such that the client MUA
could open a connection to it, ask "what sort of fax capabilities
are you willing to provide to a message coming from foo.bar.baz
to [email-masked]", and get back a quick answer.  At the
price of a new RR type, it could even be done over UDP, on a "get
an answer or give it up and send the minimum" basis.  Or, at the
price of committing an obscenity, it could be done pretty easily
over HTTP or SIP.  The UDP approach, and probably the other two,
would almost certainly be faster than getting anywhere near an
LDAP server.   Those approaches are consistent with what I
intended to imply by "...LDAP (or something else)".  

It would require some configuration, but so does piggypbacking on
SMTP.  Is it important enough to deploy?  I don't know.  But, if
it isn't, is this capability really worth cluttering up SMTP?
Remember that quote from RFC 1425 and its successors... your name
is on that document too.

You also wrote:

> Second, I believe the notion of using a directory for this sort
> of thing has been discussed in the FAX WG on several occasions.
> And while it is axiomatic that nothing precludes the WG's
> having chosen to use a directory, nothing requires that a WG
> pursue it either. I take the rather stunning lack of interest
> in this overall approach as yet another sign that it is seen as
> nonviable.

Since I have been paying only intermittent attention to the Fax
WG, I missed those discussions.  But most of the discussions
(about other topics) I remember from that WG were focused,
appropriately, on fax.   One of the reasons we do IETF last
calls, IMO, is to be sure that applicable considerations outside
the scope of the WG are exposed and aired.  So, unless you are
convinced that the WG carefully and accurately balanced the
general (1425/1869, etc) argument against SMTP extensions on
behalf of the overall community, I'd be interested in their
reasons for rejecting alternate approaches but might not find
them compelling.
 
To conclude, Keith wrote:

> The two approaches are not mutually exclusive, but if we wanted
> to support both query methods then it would be good idea to 
> understand how they interact sooner rather than later.

And it is a good idea to weigh the strengths and weakness of
each, in an overall Internet and email context, not just a fax
one (unless this capability is to be narrowed to fax
applications).  That is why, regardless of the outcome of this
discussion, I think it is a good use of time and appreciate your
willingness to engage in it without dismissing the issues as
spurious.

     john

Date: Sat, 13 Jul 2002 13:04:52 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: Keith Moore <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > > None of this experience seems particularly applicable to the question at hand,
> > > where we are comparing two different mechanisms for exporting only a small
> > > amount of information - SMTP, or some other protocol (of which LDAP is only a
> > > token example).  If an enterprise is reluctant to export "information about
> > > their users" in general, then they will be about as reluctant to do so via SMTP
> > > as via some other protocol.
> >
> > I'm afraid this is another piece of theoretical considerations that fly in the
> > face of actual experience. The obvious counterexample is HTTP.

> The relevance of HTTP is even less obvious to me because HTTP and directory
> services (LDAP or otherwise) tend to be used in very different ways.  I don't
> see many sites exporting their company directories via either HTTP or LDAP.

Then you haven't looked very hard. The former is quite popular; the latter
practically nonexistant.  And the fact that the same information is exposed
either way yet one approach is preferred is directly relevant to the matter
at hand.

> Nor do I see how any of this is relevant  to the problem of exporting recpient
> capability information for content negotiation.

In a rational world it would not be relevant. We don't live in one of those.

> > I agree that predictability is nice, but that's not something any approach can
> > ever guarantee. Indeed, I can argue that a directory approach will lead to less
> > predictability, not more, if for no other reason than directory access will
> > tend to work locally but not remotely. SMTP access with appropriate caching, on
> > the other hand, has the potential to work both locally and globally.

> The last two statements seem fairly odd.  Maybe the problem is the use of the
> word "directory" which isn't my favorite word for this anyway.  So let's not
> call this a directory - let's call it a capability discovery protocol.  Now if
> the capability discovery protocol is defined in such a way that in order for
> your recipients to benefit from content negotiation you have to make it accessible
> (just like if you want to be able to accept mail then you have to make port 25
> accessible) then it seems fairly likely that sites that want to enable content
> negotation for their recipients will do so.

My point was they may make it available internally but not externally. And
using anything that remotely smells like a directory protocols makes this
outcome far more likely.

> On the other hand, port 25 is one
> of the few ports that is commonly blocked/filtered for outgoing traffic -
> perhaps even more often (and with more drastic filtering) than port 80.

Keith, my point was and is that the minute you start talking about using a
directory protocol -- and I insist on using the term because it's the directory
aspect that's important -- it makes the whole thing problematic in a way that
piggybacking the information on top of other type of protocols does not.

You can talk till you're blue in the face about how this is capabilities
discovery, how that's intrinsicly different from a general directory
protocol, and so on and so forth. It doesn't change the perception that
a protocol intended solely for looking up stuff, no matter how innocuous
that stuff is, no matter how divorced from the cporpoate directory or
whatever it can be made, is somehow more problematic than piggybacking
the exact same information on top of other protocols.

> So the argument that a facility that requires direct connection over port 25
> (from the origin to the MX server, not necessarily to the final MTA)
> in order to work properly (i.e. in order to let the sender control the
> conversion) will function both locally and globally seems a bit dubious.

I have pointed out previously that such direct connection isn't required by the
present proposal. So this is nothing but another strawman.

				Ned

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 15:55:56 -0400
Sender: [email-masked]

> > None of this experience seems particularly applicable to the question at hand,
> > where we are comparing two different mechanisms for exporting only a small
> > amount of information - SMTP, or some other protocol (of which LDAP is only a
> > token example).  If an enterprise is reluctant to export "information about
> > their users" in general, then they will be about as reluctant to do so via SMTP
> > as via some other protocol.
> 
> I'm afraid this is another piece of theoretical considerations that fly in the
> face of actual experience. The obvious counterexample is HTTP.

The relevance of HTTP is even less obvious to me because HTTP and directory 
services (LDAP or otherwise) tend to be used in very different ways.  I don't 
see many sites exporting their company directories via either HTTP or LDAP.  
Nor do I see how any of this is relevant  to the problem of exporting recpient 
capability information for content negotiation.

> > I do think it's reasonable to assume that any other protocol server is likely
> > to be viewed as a separate maintenance burden that needs to be justified, whereas
> > a wart on the side of SMTP will probably be viewed as just some incremental pain
> > that must be endured.  But if large sites end up needing a separate database server
> > anyway to support CONNEG because the MX servers don't have direct knowledge of
> > recipient capabilities, then that separate maintenance burden is there anyway,
> > at least for those sites.
> 
> While I obviously cannot speak for all large sites, I can tell you that
> database usage is sufficiently pervasive inside the sites I'm familiar with
> that this is basically a nonissue.

Well, I guess it hinges on what is meant by "large".  Are there a significant
number of sites "large" enough to not have their MX servers deliver directly
to recipient mailboxes, but not "large" enough to be already using a database
to store recipient information?  I suspect the answer is yes because there
are other reasons besides scale that a site would not have its MX servers
deliver directly to recipient mailboxes.  Inbound firewalls are fairly common,
and some of them operate as relays.

> First, this is not a quick dismissal, it is a conclusion based on decades of
> experience, experiencce in which LDAP only plays a small and relatively recent
> part.

not to dismiss your experience at all - but your stated justification didn't
seem (to me) to be particularly relevant.

> Second, I believe the notion of using a directory for this sort of thing has
> been discussed in the FAX WG on several occasions. And while it is axiomatic
> that nothing precludes the WG's having chosen to use a directory, nothing
> requires that a WG pursue it either. I take the rather stunning lack of
> interest in this overall approach as yet another sign that it is seen as
> nonviable.

again, is this a fax-specific extension or is it supposed to be generally
applicable?   I'll readily admit that fax machines don't need a directory.
but should the level of interest within the fax group be used as a measure
of the merit of the idea for other uses of email? 

> > I think there are benefits that favor of a rescap-like approach also.
> 
> > One is that it allows the conversion to be done by the sender's
> > MUA - or at any rate as early as possible - whereas with SMTP
> > this is not always possible because a number of sites either
> > block access to port 25 on nonlocal servers, or impose interception
> > proxies that pretend to be the remote server even if it's really a
> > relay or firewall.  I'm very much in favor of having negotiation
> > and transformation work predictably and with accountability to the
> > sender rather than having some random MTA in the chain decide that
> > a conversion is needed.
> 
> I agree that predictability is nice, but that's not something any approach can
> ever guarantee. Indeed, I can argue that a directory approach will lead to less
> predictability, not more, if for no other reason than directory access will
> tend to work locally but not remotely. SMTP access with appropriate caching, on
> the other hand, has the potential to work both locally and globally.

The last two statements seem fairly odd.  Maybe the problem is the use of the
word "directory" which isn't my favorite word for this anyway.  So let's not
call this a directory - let's call it a capability discovery protocol.  Now if
the capability discovery protocol is defined in such a way that in order for
your recipients to benefit from content negotiation you have to make it accessible 
(just like if you want to be able to accept mail then you have to make port 25 
accessible) then it seems fairly likely that sites that want to enable content 
negotation for their recipients will do so.  On the other hand, port 25 is one 
of the few ports that is commonly blocked/filtered for outgoing traffic - 
perhaps even more often (and with more drastic filtering) than port 80.

So the argument that a facility that requires direct connection over port 25
(from the origin to the MX server, not necessarily to the final MTA)
in order to work properly (i.e. in order to let the sender control the 
conversion) will function both locally and globally seems a bit dubious.

Keith

Date: Sat, 13 Jul 2002 12:08:07 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: Keith Moore <[email-masked]>
Cc: [email-masked], John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > I think this is a fine piece of theoretical analysis which unfortunately has no
> > applicability to the real world. In the real world directory/database systems
> > that let sites provide Internet-wide access to information about their users do
> > not deploy.

> Widespread deployment of LDAP as a public directory service was hampered
> by several factors - despite the designers' intention that it be used
> for publically accessible information it was widely hyped as a generic
> database access protocol, so an organizations' LDAP servers were
> typically filled with information that they wouldn't dream of making publically
> available.  Also, there wasn't an obvious global namespace or infrastructure
> that was easy for LDAP users to tie into, nor any widespread understanding of
> why it would be a good idea to do so.

LDAP is but one example, and in some ways not a good one. There are plenty of
others.

> None of this experience seems particularly applicable to the question at hand,
> where we are comparing two different mechanisms for exporting only a small
> amount of information - SMTP, or some other protocol (of which LDAP is only a
> token example).  If an enterprise is reluctant to export "information about
> their users" in general, then they will be about as reluctant to do so via SMTP
> as via some other protocol.

I'm afraid this is another piece of theoretical considerations that fly in the
face of actual experience. The obvious counterexample is HTTP.

> I do think it's reasonable to assume that any other protocol server is likely
> to be viewed as a separate maintenance burden that needs to be justified, whereas
> a wart on the side of SMTP will probably be viewed as just some incremental pain
> that must be endured.  But if large sites end up needing a separate database server
> anyway to support CONNEG because the MX servers don't have direct knowledge of
> recipient capabilities, then that separate maintenance burden is there anyway,
> at least for those sites.

While I obviously cannot speak for all large sites, I can tell you that
database usage is sufficiently pervasive inside the sites I'm familiar with
that this is basically a nonissue.

> I also believe that trying to use LDAP for this purpose would be a marketing
> disaster - because there is already such a mindset about what LDAP is used for,
> and what kind of data goes into it, that any proposal to expose this specific
> piece of information using the LDAP protocol will immediately get attacked
> by people who are concerned that other informaion often stored in LDAP will be
> inappropriately leaked.

Nowhere did I mention using LDAP to expose this information. So this appears to
be a complete strawman argument to me.

> > The IETF has tried to tackle this problem several times, and indeed continues
> > to do so now in the RESCAP WG. But I don't think anyone seriously believes that
> > even if RESCAP produces some specifications it is going to succeed at its
> > intended scale. Frankly, the silence is pretty deafening over there.

> I suspect the lack of significant interest in rescap is because it lacks a
> killer app - there are many uses for rescap but no one application for
> it sees a significant enough win to drive the people who work on that
> application to contribute to rescap.   That and each of those uses involves
> some deployment pain - and the first one involves significant pain, so nobody
> wants to be first.  Another way of saying this is that (at least in the apps
> area) we tend to divide our work around product lines - we tend to work on
> email, fax, voice mail, web replication, instant messaging and so on - and
> rescap cuts across all of these.

> (on the other hand, any technology that is initially driven by a narrow set
> of interests tends to get warped to serve that set of interests at the
> expense of others, even if the technology was generally useful.)

> I'm not sure whether this is good or bad, nor whether it is inherent or an
> artifact of our organizational structure.   Mostly I view this as a given.
> There are exceptions, but they're rare, at least in the apps area.

> But there's nothing stopping fax people, or email people, or voice mail
> people from considering a standard protocol for the lookup of recipient
> capability information and (separately) from considering whether that
> protocol should be accessed directly or through SMTP.  And while I suspect,
> intuitively, that accessing that information via SMTP is actually a
> better way to go overall, I'd like to see some more analysis on it rather
> than a quick dismissal based on experience with LDAP.

First, this is not a quick dismissal, it is a conclusion based on decades of
experience, experiencce in which LDAP only plays a small and relatively recent
part.

Second, I believe the notion of using a directory for this sort of thing has
been discussed in the FAX WG on several occasions. And while it is axiomatic
that nothing precludes the WG's having chosen to use a directory, nothing
requires that a WG pursue it either. I take the rather stunning lack of
interest in this overall approach as yet another sign that it is seen as
nonviable.

> for instance:

> One attractive thing about using SMTP is that SMTP servers for small
> sites can implement CONNEG without a separate server, so the cost is
> kept low for small sites.  Another one is that if there is a separate
> server then it's probably easier for the mail system administrator to
> configure his/her MTAs to talk to that server (and to fix bugs in
> that configuration) than for the mail system administrator to configure
> DNS so that external queries go to that server.

> If queries are made via SMTP then the interface to the database is not
> standardized, so MTA vendors end up supporting multiple interfaces.
> Some might see this as an advantage, since it allows use of a variety
> of existing back-ends.  Actually enough MTAs support mail forwarding
> via LDAP (or other database) that getting recipient capabilities from
> the same source is probably a straightforward extension.

There's no "probably" about it, although it is also true that more generic
callouts aren't exactly rocket science to implement.

> And assuming that the SMTP client-SMTP server connection has a
> significantly longer delay path than the SMTP server-database connection
> (which seems reasonable) then it could take less time overall to deliver
> the message via SMTP alone than via SMTP + a database query, provided
> the queries can be made in the same SMTP session as when the message
> is delivered.

> I think there are benefits that favor of a rescap-like approach also.

> One is that it allows the conversion to be done by the sender's
> MUA - or at any rate as early as possible - whereas with SMTP
> this is not always possible because a number of sites either
> block access to port 25 on nonlocal servers, or impose interception
> proxies that pretend to be the remote server even if it's really a
> relay or firewall.  I'm very much in favor of having negotiation
> and transformation work predictably and with accountability to the
> sender rather than having some random MTA in the chain decide that
> a conversion is needed.

I agree that predictability is nice, but that's not something any approach can
ever guarantee. Indeed, I can argue that a directory approach will lead to less
predictability, not more, if for no other reason than directory access will
tend to work locally but not remotely. SMTP access with appropriate caching, on
the other hand, has the potential to work both locally and globally.

				Ned

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: John C Klensin <[email-masked]>, Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 14:17:58 -0400
Sender: [email-masked]

> I think this is a fine piece of theoretical analysis which unfortunately has no
> applicability to the real world. In the real world directory/database systems
> that let sites provide Internet-wide access to information about their users do
> not deploy.

Widespread deployment of LDAP as a public directory service was hampered
by several factors - despite the designers' intention that it be used
for publically accessible information it was widely hyped as a generic 
database access protocol, so an organizations' LDAP servers were 
typically filled with information that they wouldn't dream of making publically 
available.  Also, there wasn't an obvious global namespace or infrastructure
that was easy for LDAP users to tie into, nor any widespread understanding of 
why it would be a good idea to do so.

None of this experience seems particularly applicable to the question at hand, 
where we are comparing two different mechanisms for exporting only a small 
amount of information - SMTP, or some other protocol (of which LDAP is only a 
token example).  If an enterprise is reluctant to export "information about 
their users" in general, then they will be about as reluctant to do so via SMTP
as via some other protocol.

I do think it's reasonable to assume that any other protocol server is likely 
to be viewed as a separate maintenance burden that needs to be justified, whereas 
a wart on the side of SMTP will probably be viewed as just some incremental pain 
that must be endured.  But if large sites end up needing a separate database server 
anyway to support CONNEG because the MX servers don't have direct knowledge of 
recipient capabilities, then that separate maintenance burden is there anyway, 
at least for those sites.

I also believe that trying to use LDAP for this purpose would be a marketing 
disaster - because there is already such a mindset about what LDAP is used for,
and what kind of data goes into it, that any proposal to expose this specific
piece of information using the LDAP protocol will immediately get attacked
by people who are concerned that other informaion often stored in LDAP will be
inappropriately leaked.  

> The IETF has tried to tackle this problem several times, and indeed continues
> to do so now in the RESCAP WG. But I don't think anyone seriously believes that
> even if RESCAP produces some specifications it is going to succeed at its
> intended scale. Frankly, the silence is pretty deafening over there.

I suspect the lack of significant interest in rescap is because it lacks a
killer app - there are many uses for rescap but no one application for
it sees a significant enough win to drive the people who work on that
application to contribute to rescap.   That and each of those uses involves 
some deployment pain - and the first one involves significant pain, so nobody 
wants to be first.  Another way of saying this is that (at least in the apps 
area) we tend to divide our work around product lines - we tend to work on 
email, fax, voice mail, web replication, instant messaging and so on - and 
rescap cuts across all of these.

(on the other hand, any technology that is initially driven by a narrow set 
of interests tends to get warped to serve that set of interests at the 
expense of others, even if the technology was generally useful.)

I'm not sure whether this is good or bad, nor whether it is inherent or an
artifact of our organizational structure.   Mostly I view this as a given.
There are exceptions, but they're rare, at least in the apps area.

But there's nothing stopping fax people, or email people, or voice mail
people from considering a standard protocol for the lookup of recipient
capability information and (separately) from considering whether that
protocol should be accessed directly or through SMTP.  And while I suspect,
intuitively, that accessing that information via SMTP is actually a 
better way to go overall, I'd like to see some more analysis on it rather
than a quick dismissal based on experience with LDAP.

for instance:

One attractive thing about using SMTP is that SMTP servers for small 
sites can implement CONNEG without a separate server, so the cost is 
kept low for small sites.  Another one is that if there is a separate 
server then it's probably easier for the mail system administrator to
configure his/her MTAs to talk to that server (and to fix bugs in
that configuration) than for the mail system administrator to configure
DNS so that external queries go to that server.

If queries are made via SMTP then the interface to the database is not 
standardized, so MTA vendors end up supporting multiple interfaces.
Some might see this as an advantage, since it allows use of a variety
of existing back-ends.  Actually enough MTAs support mail forwarding
via LDAP (or other database) that getting recipient capabilities from 
the same source is probably a straightforward extension.

And assuming that the SMTP client-SMTP server connection has a 
significantly longer delay path than the SMTP server-database connection
(which seems reasonable) then it could take less time overall to deliver
the message via SMTP alone than via SMTP + a database query, provided
the queries can be made in the same SMTP session as when the message
is delivered.

I think there are benefits that favor of a rescap-like approach also.

One is that it allows the conversion to be done by the sender's
MUA - or at any rate as early as possible - whereas with SMTP
this is not always possible because a number of sites either 
block access to port 25 on nonlocal servers, or impose interception
proxies that pretend to be the remote server even if it's really a 
relay or firewall.  I'm very much in favor of having negotiation 
and transformation work predictably and with accountability to the 
sender rather than having some random MTA in the chain decide that 
a conversion is needed.

Another is that the mechanism could be used for other things besides
just recipient capabilities - say to distribute a recipient's 
public key, to opt out of spam, etc.  These could also be provided 
via SMTP but some of them absolutely require access by the sender's
UA to be useful.

Keith

Date: Sat, 13 Jul 2002 08:55:41 -0700 (PDT)
From: [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
To: John C Klensin <[email-masked]>
Cc: Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> A footnote on the "find out from a database" issue.  The more I
> think about it, the more I conclude that the database lookup
> (LDAP or otherwise) is the only rational way of implementing the
> desired functionality here in a case that might involve relaying.
> To the extent that speedy and confirmed delivery are often cited
> as important values in fax, the "maybe it will get delivered as
> requested, maybe it will be downgraded, maybe the format will be
> influenced by the format requirements of other recipients, and
> maybe the message will be bounced in a form that is hard or
> impossible for the originating (or early relay) MTA to detect"
> alternatives are unreasonable.

> But, if database lookup is the only plausible implementation that
> is consistent with the underlying design requirements, then we
> should not be looking at capabilities negotiation in the SMTP
> stream.  Instead, we should be looking at a piece of protocol by
> which the sending system (probably the MUA, rather than the MTA)
> can figure out the capabilities of the recipient(s) and then
> prepare the message, make address grouping decisions, etc.,
> cleanly.   It seems to me that would be more clear, would put the
> responsibility back on the MUA or local system which has maximum
> information about sender intent, would avoid layering violations
> and other complications, and generally be a cleaner and more
> efficient model.

I think this is a fine piece of theoretical analysis which unfortunately has no
applicability to the real world. In the real world directory/database systems
that let sites provide Internet-wide access to information about their users do
not deploy.

I'm not sure why this is the case, and indeed I'm fairly sure that there is no
good reason for it, but a vast tract of experience tells me it for-sure is the
case. (Now is not the time to get into whys in more detail.)

The IETF has tried to tackle this problem several times, and indeed continues
to do so now in the RESCAP WG. But I don't think anyone seriously believes that
even if RESCAP produces some specifications it is going to succeed at its
intended scale. Frankly, the silence is pretty deafening over there.

Lacking direct capabilities to find capabilties information necessarily leads
to various piggybacking options, that is, deploy this facility on top of or
inside other protocols. This has some of the same characteristics as the
everything-over-HTTP approach, but with several important differences: There is
a close relationship between the piggybacked data and the intended purpose of
the underlying protocols (a relationship that is often lacking with stuff
travelling over HTTP) and the operational case for doing this is far more
compelling (getting the OK to drill a hole in a firewall for a new protocol is
triviality itself compared to the difficulty in setting up even restricted
access to  directory information).

I think the availabilty of capabilities information has the potential to be as
important a step as MIME was. As such, I don't begrudge healthy debate about
how and where it is done. But let's not forget that the development of MIME was
informed by the catastrophic failure of various other schemes for doing
multimedia messaging -- some of them more elegant than MIME -- leading to
compromises in the design intended to insure deployability. I think the
discussion of where and how to do content negotiation needs to be similarly
informed by the catastrophic failure of Internet-wide directory services to
deploy.

				Ned

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: John C Klensin <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 10:16:15 -0400
Sender: [email-masked]

> The UA-based CONNEG mechanism does query/response between two UAs that are
> connected only by the store-and-forward email service.  There is nothing
> that dictates that application-level query/response exchanges be conducted
> over connection-oriented, direct or real-time associations.

perhaps not, but once a message is relayed, the conversion (and the decision
of whether to deliver the message) is no longer under control of the sender - 
actually the most likely scenario when using the conneg proposal with the 
current infrastructure is that the message will be delivered, without 
conversion, regardless of recipient capabilities.

I thought that this is what the fax people wanted to avoid?

From: Keith Moore <[email-masked]>
To: John C Klensin <[email-masked]>
cc: Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 10:12:21 -0400
Sender: [email-masked]

> A footnote on the "find out from a database" issue.  The more I
> think about it, the more I conclude that the database lookup
> (LDAP or otherwise) is the only rational way of implementing the
> desired functionality here in a case that might involve relaying.

I'm reaching the same conclusion.

> But, if database lookup is the only plausible implementation that
> is consistent with the underlying design requirements, then we
> should not be looking at capabilities negotiation in the SMTP
> stream.  Instead, we should be looking at a piece of protocol by
> which the sending system (probably the MUA, rather than the MTA)
> can figure out the capabilities of the recipient(s) and then
> prepare the message, make address grouping decisions, etc.,
> cleanly.   It seems to me that would be more clear, would put the
> responsibility back on the MUA or local system which has maximum
> information about sender intent, would avoid layering violations
> and other complications, and generally be a cleaner and more
> efficient model.

I generally agree, or at least I think it's worth considering this
alternative.    Which makes more sense:

- explicitly configuring every general-purpose SMTP server that acts 
  as a mail exchanger (and that supports conneg) to talk to a database 
  for each of the recipient domains that it serves?

  or

- defining a standard recipient capability query mechanism
  (say, something like rescap) and having a standard way to
  advertise via DNS which servers to use to find capabilities 
  for recipients at a particular mail domain (say, something
  like SRV records)

Problem is, you get different answers depending on whether:

- you build appliances that receive fax over the internet (in 
  which case it's simpler to just implement one server process 
  and you'd rather avoid imposing the burden of extra DNS setup 
  on your customers), or

- you build or operate general-purpose SMTP servers (in which 
  case you might rather avoid having to explicitly configure
  those servers, and also avoid having your SMTP servers wait
  on more database queries)

The two approaches are not mutually exclusive, but if we wanted
to support both query methods then it would be good idea to 
understand how they interact sooner rather than later.

Keith

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Hiroshi Tamura <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation" 
Date: Sat, 13 Jul 2002 09:51:03 -0400
Sender: [email-masked]

> >- new command RCAP??
> 
> The current ESMTP/CONNEG specification uses the RCPT command for finding 
> each recipient's capabilities.
> 
> Moore-san prefers to have the capabilities query be in a separate 
> command.  For convenience, he has given this command a name, RCAP.  So, the 
> name refers to an idea that Moore-san has.  There is no written 
> specification for that command.  In my opinion, the current specification 
> is satisfactory.

this is a little misleading.

if a message is being sent to more than one recipient then there is a 
possibility that different recipients will need different versions of the 
content, or that some recipients will be unable to accept the content at all.

if the message being transmitted is a fax and all recipients are capable of fax, 
then perhaps it is acceptable to downgrade the message to a lower quality fax 
format which is acceptable to all.   however this is not always possible for
recipients in general - a SMTP server is allowed to support CONNEG if some, 
none, or all of its recipients support fax reception.

the use of the RCPT response to tell the client of the recipient's capabilities
is odd because it asserts that the client intends to deliver a message to
each of several recipients before the client actually knows whether it can 
deliver the same version of the message to each recipient, or whether it can
deliver any version of that message to the recipient.  

so as the proposal is written, any client which attempts to deliver a message
to multiple recipients in a single transaction must be prepared to abort the
current SMTP transaction (presumably by using RSET) after sending multiple RCPTs, 
to divide up the RCPTs according to which version of the message each is capable 
of receiving, and to start some number of new transactions for that purpose.  

if nothing else, that seems a bit confusing and it introduces the potential
for implementation errors.  

it's been claimed  that using RCPT in this way is faster that using a separate 
command, though I haven't actually seen an analysis of this.  I wonder if
such analysis is based on an assumption that it will usually be possible to 
downgrade the message to a common format that is acceptable to all recipients,
and I wonder if that will actually be true in practice.    but even if performance 
considerations justify using RCPT instead of a separate command, it would still 
be appropriate to document the need for clients to abort the current transaction 
if not all recipients can accept the same format.  the current proposal fails
to mention this at all.  this would be easy to fix.

another problem with RCPT is that it expects the server to return status codes
for error conditions that can only be distinguished on the client.

RCAP is just one of several ideas that have been suggested to do content
negotiation in a way that is simpler, more robust, and less confusing to 
implementors than the current proposal.  Dave is correct that there is no 
formal proposal for RCAP - though since the entire reason for proposing RCAP 
is to have something simpler, a detailed proposal hardly seems necessary to 
describe it.  Anyone with a good understanding of SMTP should be able to 
evaluate the relative merits of the current proposal, RCAP, and the other 
suggestions without having a detailed proposal for each one.

The following description for RCAP should be enough:

1. the command looks like this:   RCAP <recipient-address>

2. the server responds with one of the following  

2xx-recipient capabilities are as follows:
2xx-<conneg string goes here>
2xx

2yy recipient has no specified capabilities

2zz no information available about recipient capabilities

4xx recipient capabilities are temporarily unavailable

5xx error (for example, syntax error)

specific values for 2xx, 2yy, 2zz, 4xx, 5xx are to be determined.

3. if the server supports the ENHANCEDSTATUS codes extension then
the most appropriate enhanced status code should follow the 2xx,
2yy, 2zz, 4xx, or 5xx response code in the manner specified in RFC 2034.

4. the RCAP command is acceptable at any time that MAIL or RCPT 
would be acceptable.  clients MAY issue RCAP before or after MAIL,
and they MAY issue RCAP and RCPT commands in any order.
(though generally it makes more sense to issue RCAP for a 
recipient before sending RCPT for that recipient)
The RCAP response MUST be the same regardless of whether the
command is issued before or after MAIL, or before or after RCPT.

5. the RCAP command has no effect other than to query the server
about a recipient's capabilities.

6. the RCAP command does not indicate whether a recipient address
is valid or whether delivery of a message to the recipient will 
succeed.  any response - 2xx, 2yy, 2zz, 4xx, 5xx - is potentially
valid even for a nonexistent recipient or one which is not allowed
to receive mail.  for example, an SMTP server for a webmail 
environment which provided the same capabilities for all recipients
could return 2xx and the same conneg string for any recipient
address, regardless of whether that recipient were valid.

Keith

Date: Sat, 13 Jul 2002 17:43:03 +0900
To: John C Klensin <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 03:11 AM 7/13/2002 -0400, John C Klensin wrote:
>--On Saturday, 13 July, 2002 12:04 +0900 Dave Crocker
> > Internet mail is store-and-forward.  ... This is like packet 
> switching.  Each intermediate system is "relaying" the email.
>
>It is like packet switching except, of course, in the packet case, each 
>packet is typically a fragment or component of a message.

"Typically", perhaps, but it is not true for all packet switching systems 
and it is not part of the formal definition of packet-switching.

Packet switching is about independent addressing (and usually independent 
routing) and handling of discrete chunks of data.  It is not about the 
relationships among that data.  To that extent, Internet mail is quite 
similar to lower-level packet switching.

(I say it is exactly the same, but therein lies a debate that some of us 
have enjoyed for a few years -- errr, decades -- and it is not pertinent to 
the specification under review.)

>   The message, or data stream, is sent one packet at a time; it is not 
> completely assembled at each intermediate host and then forwarded after 
> it is complete.

On the contrary, Arpanet IMP packet switching did fragmentation and 
reassembly in the net, not in the hosts.  (Any further debate on this issue 
might be interesting, but it, too, is not relevant to the current topic.)

> > The primary difference between relaying versus direct is that
> > relaying might take much longer.  Therefore the sender does not
> > know when it will receive a response.
>
>I would disagree.  The primary difference I see between relaying
>and direct is that, in a direct arrangement (characteristic of
>e.g., HTTP) the originating sending system can ask the ultimate
>receiving one a question by, well, asking.

The UA-based CONNEG mechanism does query/response between two UAs that are 
connected only by the store-and-forward email service.  There is nothing 
that dictates that application-level query/response exchanges be conducted 
over connection-oriented, direct or real-time associations.

Just to underscore how fuzzy all this gets:  HTTP relies on the underlying 
store and forward of IP.  Email UAs rely on the underlying store and 
forward of SMTP.  And, of course, SMTP also relies on the store and forward 
of IP.

More importantly, these concerns are essential matters for successful 
implementation, deployment and use.  But they have nothing to do with core 
architectural principals.

With respect to direct vs. relay, the architectural difference between a 
UA-UA exchange and an HTTP exchange is remarkably small, especially given 
that HTTP usage pretends it has no context.  These differences get even 
smaller with the introduction of caching servers, and the like, because 
they introduce a very high degree of indirection between consumer and provider.

>A footnote on the "find out from a database" issue.  The more I
>think about it, the more I conclude that the database lookup
>(LDAP or otherwise) is the only rational way of implementing the
>desired functionality here in a case that might involve relaying.

Yes, it is an appealing model, except that LDAP has yet to achieve 
Internet-scale deployment and use.  Consequently it is not wise to make a 
new service dependent on it.

It is also worth noting that such a model requires real-time 
connectivity.  An email-based mechanism can operate with asynchronous access.

It makes far more sense to postulate a range of choices -- including 
directly asking the own of the information via a UA mechanism, as well as 
direct intermediaries via ESMTP/CONNEG -- and incorporate LDAP when it 
starts being a serious global service.

>To the extent that speedy and confirmed delivery are often cited
>as important values in fax, the "maybe it will get delivered as
>requested, ... is hard or
>impossible for the originating (or early relay) MTA to detect"
>alternatives are unreasonable.

To the extent that a higher degrees of certainty is required, then the 
operational components should be configured to provide it.  Using LDAP does 
not provide any guarantees of proper or speedy performance, any more than 
using UA- or SMTP-based content negotiations guarantees the lack.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

Date: Sat, 13 Jul 2002 03:11:13 -0400
From: John C Klensin <[email-masked]>
To: Dave Crocker <[email-masked]>, Hiroshi Tamura <[email-masked]>
cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Sender: [email-masked]

--On Saturday, 13 July, 2002 12:04 +0900 Dave Crocker
<[email-masked]> wrote:

>> - consideration of relay
> 
> Facsimile is direct, point-to-point.  The sending device and
> the receiving device are connected to each other at the same
> time.  They talk with each other directly.
> 
> Internet mail is store-and-forward.  The sender and the
> receiver are not required to be connected to the Internet at
> the same time.  The email may go through intermediate email
> services, when moving from the sender to the receiver.  (For
> example, it may go from a department's SMTP mail system to the
> enterprise system; the enterprise system may send it to the
> receiving enterprise's system; the receiving system may then
> send it to the receiver's department email server.) This is
> like packet switching.  Each intermediate system is "relaying"
> the email.

It is like packet switching except, of course, in the packet
case, each packet is typically a fragment or component of a
message.  The message, or data stream, is sent one packet at a
time; it is not completely assembled at each intermediate host
and then forwarded after it is complete.  With email, each relay
host obtains, assembles, and stores the entire message, only then
initiating a new transaction to forward it.  That difference is
significant for a few purposes, insignificant for most, but we
should be careful to not push analogies to packet-switching too
far.

> The primary difference between relaying versus direct is that
> relaying might take much longer.  Therefore the sender does not
> know when it will receive a response.

I would disagree.  The primary difference I see between relaying
and direct is that, in a direct arrangement (characteristic of
e.g., HTTP) the originating sending system can ask the ultimate
receiving one a question by, well, asking.  In the relay case,
such questions can only be asked of an intermediate system, which
must respond based on indirect information (which it finds in a
database, see below) or by acting as proxy for a remote system,
using information it has not obtained yet.   Some illustrations:

Q: Can mail be delivered to this address?

	Direct answer: "yes" or "no"
	
	Relay answer: "I have no idea, but I will accept
	responsibility for the mail and promise to forward it to
	someone else who will accept responsibility.  For each of
	us, we understand that "accepting responsibility" means
	that we will either deliver the mail to the final
	recipient, deliver it to someone else who agrees to
	accept responsibility, or return an error message which
	will, of necessity, come out-of-band relative to the
	original connection".
	
	Note: contrast this with TCP, where the packet-switching
	model and the nature of the protocol permits getting the
	possibly-analogous "yes, the destination host will accept
	traffic on that port" answer back before any data packets
	are sent.

Q: Does the recipient support this data format?

	Direct answer: "Yes" or "How about one of the following?"
	
	Relay answer 1: "The database says you should try..."
	(see below)
	
	Relay answer 2: "I don't have a clue, but I will accept
	the message on the assumption that it does, and pass it
	on, either to that destination or to some other site
	willing to take responsibility.  If it turns out the
	recipient cannot accept the format, I (or one of the
	systems to which responsibility has been passed) will
	(preferentially) convert the format to something that can
	be accepted or (less preferred) bounce it with an
	out-of-band message."

Those seem fairly different to me, much more different than
"slower".  In fact, I think that difference is part of what this
discussion is about: if the only difference were "might take
longer", I don't think we would be having this (part of) the
discussion at all.

     john

A footnote on the "find out from a database" issue.  The more I
think about it, the more I conclude that the database lookup
(LDAP or otherwise) is the only rational way of implementing the
desired functionality here in a case that might involve relaying.
To the extent that speedy and confirmed delivery are often cited
as important values in fax, the "maybe it will get delivered as
requested, maybe it will be downgraded, maybe the format will be
influenced by the format requirements of other recipients, and
maybe the message will be bounced in a form that is hard or
impossible for the originating (or early relay) MTA to detect"
alternatives are unreasonable.  

But, if database lookup is the only plausible implementation that
is consistent with the underlying design requirements, then we
should not be looking at capabilities negotiation in the SMTP
stream.  Instead, we should be looking at a piece of protocol by
which the sending system (probably the MUA, rather than the MTA)
can figure out the capabilities of the recipient(s) and then
prepare the message, make address grouping decisions, etc.,
cleanly.   It seems to me that would be more clear, would put the
responsibility back on the MUA or local system which has maximum
information about sender intent, would avoid layering violations
and other complications, and generally be a cleaner and more
efficient model.

Date: Sat, 13 Jul 2002 12:04:48 +0900
To: Hiroshi Tamura <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Keywords for "SMTP Service Extension for Content Negotiation"
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 07:11 PM 7/12/2002 +0900, Hiroshi Tamura wrote:
>There will be lots of discussion at FAX WG meeting in Yokohama.
>For your convenience, I summarize the keywords at very random,
>although they may not be covered.

Tamura-san, thank you very much for listing these terms.  It is easy to 
forget that these terms are not familiar to everyone.  Also, the discussion 
has sometimes been confusing  I encourage people to list additional terms 
that should be explained.

Here are my own, personal comments and explanations.  I think some are 
simple, but others might be controversial.  Also there might be more than 
one explanation.  I encourage others to offer their own versions:

>- fax-specific or not

It is strange,  but I think this is the most difficult term.  When we talk 
about T.30, it is easy to know what it means to say "fax-specific".  When 
we talk about FFPIM, perhaps it is not so easy.

For the IETF Fax working group, I believe "fax-specific" refers to features 
and functions that emulate T.30.  That is, we are trying to make Internet 
mail provide the same service as T.30.  So, "fax-specific" refers to email 
enhancements that permit email to work like fax.

>- multi-recipient

Internet mail permits a message to be addressed to more than one 
person.  For example, see the To and CC fields of this message.  T.30 
permits addressing a single receiving device.  Some fax store-and-forward 
services permit the sender to specify more than one receiver.  This is 
multi-recipient.

>- LCD

Least Common Denominator.  So, we are borrowing a term from math, to apply 
to our discussion of ESMTP/CONNEG, it is possible to receive CONNEG 
parameters for multiple recipients.  If the recipients have different 
capabilities, what capabilities should the sender use?  One possibility is 
to end the current SMTP session and then send each recipient the best 
content they can receive.  Another possibility is to find the 
"intersection" of capabilities for all the recipients and send this "least 
common denominator" or "most common format" to all of the recipients at once.

>- new command RCAP??

The current ESMTP/CONNEG specification uses the RCPT command for finding 
each recipient's capabilities.

Moore-san prefers to have the capabilities query be in a separate 
command.  For convenience, he has given this command a name, RCAP.  So, the 
name refers to an idea that Moore-san has.  There is no written 
specification for that command.  In my opinion, the current specification 
is satisfactory.

>- RSET
>- VRFY

These are SMTP commands.  RSET ends the current SMTP session.  The command 
means "reset".  VRFY lets a client SMTP ask a server SMTP to "verify" the 
validity of an address, before using it.  This command is no longer very 
popular, because it provides an easy way for spam senders to test for new 
addresses.

>- consideration of relay

Facsimile is direct, point-to-point.  The sending device and the receiving 
device are connected to each other at the same time.  They talk with each 
other directly.

Internet mail is store-and-forward.  The sender and the receiver are not 
required to be connected to the Internet at the same time.  The email may 
go through intermediate email services, when moving from the sender to the 
receiver.  (For example, it may go from a department's SMTP mail system to 
the enterprise system; the enterprise system may send it to the receiving 
enterprise's system; the receiving system may then send it to the 
receiver's department email server.) This is like packet switching.  Each 
intermediate system is "relaying" the email.

The primary difference between relaying versus direct is that relaying 
might take much longer.  Therefore the sender does not know when it will 
receive a response.

On the Internet, a direct connection usually has a response time of a few 
minutes, maximum.  Normally it is much shorter, of course.  For email 
relaying, response time can be days, not minutes.  This difference in 
latency requires different design choices.

>- content transforamation

Converting content from TIF to JPEG, and JPEG to GIF are examples of 
content transformation.  Converting from UTF-8 to UTF-16 is also an example 
of content transformation.

>- only CONNEG or CONNEG=REQUIRED/OPTIONAL

I do not know what "only CONNEG" means, here.

>- interoprability with existing environment

When we add a new feature, how does it affect the old operation?  Sometimes 
adding a new features makes the old system stop working.  For MIME, the 
goal was to add a new feature that made no change to email transport and 
also would not break an old email user agent.

The goal with ESMTP options is to make old operations behave the old way, 
but permit participants who support a new option to use them with each other.

>- how to handle with response/status code

I believe this refers to a concern that ESMTP/CONNEG uses extended ESMTP 
response/status codes in an usual way.  The concern is not widely held.

>- real-time

I think that we use this term very informally in the Fax working group 
discussions.  Probably it simply means "quickly".  Perhaps we can simply 
use it to mean "within the limits of T.30 timeouts" -- 2 seconds?

For the design of human interaction, real-time usually mean 1/2 second, or 
less.  For the design or high performance system, it can refer to 
milliseconds or microseconds.

>- bandwidth

The amount of data that can be send within a period of time.  The bandwidth 
of a T1 (or E1) line is very much greater than a dial-up connection.

>- multipart/alternative

The MIME standard for Internet mail specifies a means of attaching one or 
more segments (body-parts) of content.

Multipart/mixed is used for specifying multiple attachments that are not 
related to each other.

Multipart/alternate is use to specify multiple attachments that are 
different versions of the same content.  For example, one might be ASCII; 
another might be HTML, and another might be PDF.

When the sender does not know the capabilities of the recipient, the sender 
can use multipart/alternative to send more than one version of the 
content.  The sender is guessing that the receiver can process at least one 
of the forms.

>- security

Gomen nasai.  This is a very big topic, of course.

I think the discussion of security for ESMTP/CONNEG only has the concern 
about violating privacy.   By using this option, the receiving server is 
divulging features of the recipient.  It is possible that such information 
is private, or that distribution of such information must be restricted.

>- Experimental or Proposed Standard

The IETF standards process has 3 classes of categorization:

And Experimental document is NOT on the standards track.  The document is 
published for the benefit of the community and the author is requesting 
feedback, especially comments about experiments that use the specification.

A Best Current Practises categorization refers to documents that describes 
operations and administration procedures.  They do not define protocols, 
they define the USE of protocols and procedures.

The IETF standards track has 3 levels:  Proposed, Draft and (full) 
Standard.  Proposed is the first level.  A document must be complete, for 
the task it defines and must have no known errors.  However the 
specification usually does not have to be implemented and 
tested.  Implementation and testing are required to reach Draft standard.

In the discussion for ESMTP/CONNEG, some participants are concerned that 
the option may cause bad effects on normal SMTP usage.  They believe that 
the specification should be classed as Experimental until it is tested.  My 
own comments about this are that all new protocols carry some risk.  The 
pressure to select Experimental (non-standards) status usually is reserved 
for specifications that are complicated and poorly understand, or for 
specifications that have an inherently high degree of risk.

In my opinion, ESMTP/CONNEG is a simple option with little risk.  I believe 
that we DO need to learn to use it.  It DOES add some complexity and it 
extends use of the ESMTP option feature in some way.  However I believe 
that the risks involve efficiency, not core functionality.  That is, I 
believe that implementors must learn to make the option work well, but that 
the option should be relatively safe.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

To: [email-masked], [email-masked], [email-masked]
Cc: [email-masked], [email-masked]
Subject: Keywords for "SMTP Service Extension for Content Negotiation"
From: Hiroshi Tamura <[email-masked]>
Date: Fri, 12 Jul 2002 19:11:42 +0900 (JST)
Lines: 32
Sender: [email-masked]

Folks,

There will be lots of discussion at FAX WG meeting in Yokohama.
For your convenience, I summarize the keywords at very random,
although they may not be covered.

- fax-specific or not
- multi-recipient
- LCD
- new command RCAP??
- RSET
- VRFY
- consideration of relay
- content transforamation
- only CONNEG or CONNEG=REQUIRED/OPTIONAL
- interoprability with existing environment
- how to handle with response/status code
- real-time
- bandwidth
- multipart/alternative
- security
- Experimental or Proposed Standard
- ...
- ...

Toyoda-san and Dave:
Could you please consider some of them at your presentation?

Regards,
--
Hiroshi Tamura, Co-chair of IETF-FAX WG
[email-masked]

Date: Tue, 09 Jul 2002 02:48:04 -0700 (PDT)
From: [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> [email-masked]
> > > Negotiation can mean that A and B receive different messages, both stating
> > > (in to/cc) that the other received the one they didn't. Sounds very fishy
> > > to my ears.
> >
> > This is nothing new; situations arise routinely where the header recipient
> > lists don't tell the full story about who did or did not get a message

> Agreed. It's nothing new -- just more of a bad thing.

> I suppose it's okay, though. If to/cc never were truly reliable, making
> them even less reliable is no big deal. Even if it's the first time a
> Proposed Standard does it.

Hardly the first time. MIME itself has the notion of downgrading from 8bit to
7bit or from binary to 8bit.

> > or a
> > particular version of a message.

> Hm? I know that some mailing lists do rather a lot of header field
> rewriting and/or append a new signature.  Are there other examples?

Sure. There's plenty of content conversion/charset
conversion/reformatting/virus removal, etc. etc. etc. going on out there
already.

				Ned

Date: Tue, 9 Jul 2002 11:47:33 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Cc: Keith Moore <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Sender: [email-masked]

[email-masked]
> > Negotiation can mean that A and B receive different messages, both stating
> > (in to/cc) that the other received the one they didn't. Sounds very fishy
> > to my ears.
> 
> This is nothing new; situations arise routinely where the header recipient
> lists don't tell the full story about who did or did not get a message

Agreed. It's nothing new -- just more of a bad thing.

I suppose it's okay, though. If to/cc never were truly reliable, making
them even less reliable is no big deal. Even if it's the first time a
Proposed Standard does it.

> or a
> particular version of a message.

Hm? I know that some mailing lists do rather a lot of header field
rewriting and/or append a new signature.  Are there other examples?

> > The thing sounds as if it's 100% incompatible with S/MIME and other
> > signature mechansisms. Shouldn't that be mentioned in the draft's security
> > considerations section?
...
> As for mentioning any of this in the security considerations section, I
> have no problem with that.

Good. That's what I want.

--Arnt

From: Keith Moore <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Keith Moore <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Mon, 08 Jul 2002 18:18:14 -0400
Sender: [email-masked]

> Negotiation can mean that A and B receive different messages, both stating
> (in to/cc) that the other received the one they didn't. Sounds very fishy
> to my ears.

That's an interesting point.  However it's always been possible for 
recipients listed in the to/cc fields to receive different versions
of the message, for some of those listed to not receive the message
at all, or for additional recipients to receive the message.  

Keith

To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
From: [email-masked]
Date: Mon, 08 Jul 2002 14:08:04 -0400
Sender: [email-masked]

--==_Exmh_-2078677764P
Content-Type: text/plain; charset=us-ascii

On Mon, 08 Jul 2002 10:20:21 PDT, [email-masked] said:

> On the contrary, far from being incompatible, this facility may actually
> facilitate the ability to use signatures in situations where it presently isn't
> possible to use them. Specifically, if a client is aware of a recipient's needs
> it can tailor a message for that recipient and then sign it after the 
> tailoring is complete. Without this knowledge what tends to happen is that the
> tailoring is done after the signature is applied, which of course renders the
> signature invalid.

Unfortunately, the draft as written doesn't provide a mechanism to push that
information back as far as the sending MUA - you could only get this to work
if the MUA included its own sender-side MTA functionality (which would be at
odds with the push to use port 587 and the practice of many ISP's of blocking
outbound to port 25 except from their mail servers).
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_-2078677764P
Content-Type: application/pgp-signature

--==_Exmh_-2078677764P--

Date: Mon, 08 Jul 2002 10:20:21 -0700 (PDT)
From: [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
To: Arnt Gulbrandsen <[email-masked]>
Cc: Keith Moore <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> Keith Moore <[email-masked]>
> > > IMO, the ultimate aim (even if this is not practical) is to deliver
> > > to each recipient the same version of the Fax as they would have
> > > received IF they were the only addressee
> >
> > I think this is a reasonable and practical goal.  I don't think that
> > recipient A should experience a difference in message quality
> > (or worse, fail to receive the message at all) based on whether or
> > not you cc'ed recipient B.

> Negotiation can mean that A and B receive different messages, both stating
> (in to/cc) that the other received the one they didn't. Sounds very fishy
> to my ears.

This is nothing new; situations arise routinely where the header recipient
lists don't tell the full story about who did or did not get a message or a
particular version of a message.

> The thing sounds as if it's 100% incompatible with S/MIME and other
> signature mechansisms. Shouldn't that be mentioned in the draft's security
> considerations section?

On the contrary, far from being incompatible, this facility may actually
facilitate the ability to use signatures in situations where it presently isn't
possible to use them. Specifically, if a client is aware of a recipient's needs
it can tailor a message for that recipient and then sign it after the 
tailoring is complete. Without this knowledge what tends to happen is that the
tailoring is done after the signature is applied, which of course renders the
signature invalid.

Now you may argue that such transformations are a bad idea. And on a
theoretical level I'd agree with you. But on a practical level the need for
them is there, and I'd much rather see then done by the client than by some
intermediary.

The fact that multipart/signed is designed so that signatures can be attached
to content on the fly is also useful in this context.

As for mentioning any of this in the security considerations section, I
have no problem with that.

				Ned

Date: Mon, 8 Jul 2002 19:05:04 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Sender: [email-masked]

Keith Moore <[email-masked]>
> > IMO, the ultimate aim (even if this is not practical) is to deliver
> > to each recipient the same version of the Fax as they would have
> > received IF they were the only addressee 
> 
> I think this is a reasonable and practical goal.  I don't think that 
> recipient A should experience a difference in message quality 
> (or worse, fail to receive the message at all) based on whether or 
> not you cc'ed recipient B.

Negotiation can mean that A and B receive different messages, both stating
(in to/cc) that the other received the one they didn't. Sounds very fishy
to my ears.

The thing sounds as if it's 100% incompatible with S/MIME and other
signature mechansisms. Shouldn't that be mentioned in the draft's security
considerations section?

--Arnt

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Fri, 05 Jul 2002 21:02:11 -0400
Sender: [email-masked]

> > > >however the current proposal doesn't really give the sender a way
> > > >to specify that threshhold - unless, perhaps, the negotiation/conversion
> > >
> > >
> > > The protocol is MTA-MTA.
> >
> >indeed, but you said that the choice of whether to downgrade to LCD
> >is the sender's choice,
> 
> sending MTA.

ah, that's a slightly different thing.

but I really don't think it's a good idea to have MTA making value judgements
about the suitability of converting (whether or not using an LCD strategy)
in the absence of explicit direction from either sender or recipient.

Keith 

Date: Fri, 05 Jul 2002 17:53:31 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 08:47 PM 7/5/2002 -0400, Keith Moore wrote:

> > >however the current proposal doesn't really give the sender a way
> > >to specify that threshhold - unless, perhaps, the negotiation/conversion
> >
> >
> > The protocol is MTA-MTA.
>
>indeed, but you said that the choice of whether to downgrade to LCD
>is the sender's choice,

sending MTA.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, Dave Crocker <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Fri, 05 Jul 2002 20:47:02 -0400
Sender: [email-masked]

> >however the current proposal doesn't really give the sender a way
> >to specify that threshhold - unless, perhaps, the negotiation/conversion
> 
> 
> The protocol is MTA-MTA.

indeed, but you said that the choice of whether to downgrade to LCD
is the sender's choice, which is kind of an odd statement when 
in the vast majority of cases the sender doesn't have any control
over the MTA's decision-making process, and you haven't suggested
a mechanism to do this.

> You are discussing UA-MTA.  That's a different protocol.  Feel free to
> draft a version.

I'm not discussing a different protocol, I'm discussing a characteristic 
of your proposal with respect to a claim that you made about your proposal.

Ketih

Date: Fri, 05 Jul 2002 17:04:42 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Cc: Dave Crocker <[email-masked]>, "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 08:00 PM 7/5/2002 -0400, Keith Moore wrote:
>however the current proposal doesn't really give the sender a way
>to specify that threshhold - unless, perhaps, the negotiation/conversion

The protocol is MTA-MTA.

You are discussing UA-MTA.  That's a different protocol.  Feel free to 
draft a version.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Fri, 05 Jul 2002 20:00:48 -0400
Sender: [email-masked]

> >IMO, the ultimate aim (even if this is not practical) is to deliver to
> >each recipient the same version of the Fax as they would have received IF
> >they were the only addressee while at the same time generating only the
> >minimum number of copies to allow multi-
> 
> That is a worthy and reasonable use for this capability.  And, of course
> yes, it motivated creating this option.
> 
> However that level of "perfection" is not required.  There is nothing wrong
> with a service model that is satisfied with sending the least common
> denominator.
> 
> The choice is the sender's.

as long as the sender gets to specify the conditions under which the
conversion is done, that seems just fine - it is no different than
allowing the the sender to personally look at a set of potential
recipients' capabilities and decide what the minimum requirements
for reception (and therefore the LCD format) should be.

however the current proposal doesn't really give the sender a way
to specify that threshhold - unless, perhaps, the negotiation/conversion 
is performed by an MTA that is under control of the sender (in which
case the threshold could be specified by explicit configuration)
and that MTA has the abliity to speak SMTP directly with MTAs that 
know the recipients' capabilities.

in other words, enabling the sender to make the choice is not 
compatible with the vast majority of the deployed email 
infrastructure (even assuming that those MTAs are extended to 
support this extension) 

Keith

Date: Fri, 05 Jul 2002 16:13:53 -0700
To: "Robert A. Rosenberg" <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation  to Proposed Standard
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 01:58 PM 7/4/2002 -0400, Robert A. Rosenberg wrote:
>IMO, the ultimate aim (even if this is not practical) is to deliver to 
>each recipient the same version of the Fax as they would have received IF 
>they were the only addressee while at the same time generating only the 
>minimum number of copies to allow multi-

That is a worthy and reasonable use for this capability.  And, of course 
yes, it motivated creating this option.

However that level of "perfection" is not required.  There is nothing wrong 
with a service model that is satisfied with sending the least common 
denominator.

The choice is the sender's.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Fri, 05 Jul 2002 14:08:28 -0400
Sender: [email-masked]

> >here are a couple of trivial examples of the effect of a "downgrade to
> >LCD format" implementation strategy. 
> >
> >assume:
> >
> >recipient A accepts only HTML and plain text
> >recipient B accepts only PDF and plain text
> >
> >an HTML message sent only to A gets delivered as HTML, but if B is cc'ed
> >(or bcc'ed) on the message then both A and B get plain text (if the
> >client knows how to convert HTML to plain text).  A is accustomed
> >to receiving HTML from B and is surprised that there's an information
> >loss.
> 
> No - This is a standard M-P/A situation and should be sent to both as 
> such (just as if the original message was generated by the MUA and 
> packaged as M-P/A [which is the normal action of most of the MUAs I 
> know unless the user has overridden the send both option).

I don't follow your reference to multipart/alternative.  The source
message in the example is HTML only.  And I don't think anyone has 
proposed that the SMTP client performing a conversion should use 
multipart/alternative to include both the old and the versions -
if nothing else, the fax stuff that is driving this has had the
ability to use multipart/alternative for 10+ years and they obviously
don't consider it sufficient for their purposes.

Also, B might not want HTML, not even in a multipart/alternative - 
and it seems quite reasonable to me that B would specify this 
(since I don't want it either).

The fact that many MUAs generate multipart/alternative with text+html
by default should not be cited as if it were a desirable practice.
 
> >more seriously, a PDF message sent only to B gets delivered as PDF, but
> >if A is cc'ed on the message then the message is bounced for both A and
> >B because there's no good way to downgrade PDF to a format acceptable
> >to B (or the client doesn't know how).
> 
> This is a more complex case. It would/could be solvable by a Smart 
> MUA that acted the same as the HTML/Plain case by sending/creating as 
> M-P/A with Text/Plain and application/PDF versions. The "Best" 
> solution is if the MUA could also deliver a HTML copy (which would be 
> packaged between the Text and the PDF in the stacking order).

senders' MUAs are irrelevant to this discussion, unless they also perform
conversion based on SMTP CONNEG responses (which would reqiure them
to connect directly to recipients' ultimate MTAs - not likely)

the scenario is this - some MTA gets a message, it looks at the recipient
capabilities, and decides that it needs to convert the message to make
it acceptable to the recipients.  the point of my examples is to demonstrate
that there will be cases where there is no single message format that
is acceptable to all recipients, which implies that any MTA which attempts 
to deliver to more than one recipient at a time (and which uses CONNEG)
must be prepared to handle the case where different behavior is applied 
to different recipients - and this further requires RSET and/or 
close-connection-and-retry logic.
(even with a close RSET should be strongly recommended)

Keith

Date: Fri, 5 Jul 2002 12:45:34 -0400
To: [email-masked], [email-masked], [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Sender: [email-masked]

At 15:15 -0400 on 07/04/2002, Keith Moore wrote about Re: Last Call: 
SMTP Service Extension for Content Negot:

>>  IMO, the ultimate aim (even if this is not practical) is to deliver
>>  to each recipient the same version of the Fax as they would have
>>  received IF they were the only addressee
>
>I think this is a reasonable and practical goal.  I don't think that
>recipient A should experience a difference in message quality
>(or worse, fail to receive the message at all) based on whether or
>not you cc'ed recipient B.
>
>>  while at the same time
>>  generating only the minimum number of copies to allow
>>  multi-addressing.
>
>I think this is an implementation decision.  It should be *possible*
>to do this (and it is), but it shouldn't be required of implementations.

Bad phrasing on my part (I had intended the minimal message count to 
be a target not a cast-in-stone requirement). If you must degrade 
into one message per address mode - so be it - BUT I think that if 
you are going to do so, you are cheaping out by not trying to add 
some minimal logic to run the chain of pending copies to see if any 
can be satisfied by the current version. I'm not asking for a full 
matrix analysis (which is the "Best"/"Optimum" Method) - just take 
them in order of the addresses and run the pending chain. This will 
probably not create the minimal count (except by luck) but can still 
reduce the count somewhat.

>here are a couple of trivial examples of the effect of a "downgrade to
>LCD format" implementation strategy. 
>
>assume:
>
>recipient A accepts only HTML and plain text
>recipient B accepts only PDF and plain text
>
>an HTML message sent only to A gets delivered as HTML, but if B is cc'ed
>(or bcc'ed) on the message then both A and B get plain text (if the
>client knows how to convert HTML to plain text).  A is accustomed
>to receiving HTML from B and is surprised that there's an information
>loss.

No - This is a standard M-P/A situation and should be sent to both as 
such (just as if the original message was generated by the MUA and 
packaged as M-P/A [which is the normal action of most of the MUAs I 
know unless the user has overridden the send both option).

>
>more seriously, a PDF message sent only to B gets delivered as PDF, but
>if A is cc'ed on the message then the message is bounced for both A and
>B because there's no good way to downgrade PDF to a format acceptable
>to B (or the client doesn't know how).

This is a more complex case. It would/could be solvable by a Smart 
MUA that acted the same as the HTML/Plain case by sending/creating as 
M-P/A with Text/Plain and application/PDF versions. The "Best" 
solution is if the MUA could also deliver a HTML copy (which would be 
packaged between the Text and the PDF in the stacking order).

Note: I am assuming a 2 stage MUA where the messages are created and 
queued for delivery and upon connect, the MUA can drop out (if it 
wants) unneeded alternatives (instead of doing the transforms in real 
time). I see no reason that the M-P/A creation (with optional 
deletion prior to transmit) could not be a viable alternative for 
real-time transforms (at least in the case where the number of 
different variations is limited/minimal such as in the above case or 
in the case of a Fax where you must convert Color to B&W and/or 
support 2 or 3 resolutions). I am working in a Blue Sky mode with 
these comments and I'm just tossing out a best functionality case 
which must/will be modified by implementation and real-world 
constraints.

>
>So far I have not been able to identify conditions for which downgrading
>to LCD is a viable implementation strategy.  My conclusion is that
>implementations that support CONNEG and that attempt to deliver to
>mutliple recipients in one transaction MUST be prepared to do RSET and
>split up the delivery if it becomes evident that delivering the same
>version of the message to all recipients is not appropriate.
>
>Keith

Note: I am offering all of the above ideas/processing methods as 
suggestions for possible listings of ways to reach the goal of 
delivering the message to the recipient as if in a one-to-one mode 
while trying to minimize the overhead that is associated with 
one-to-one mode by leveraging the processing overhead of the message 
generation/uploading for additional addressees. As someone who has 
not been associated with the evolution of this effort I think my "new 
perspectives" can, at times, point out methodologies that might not 
be obvious those who have participated in its design. OTOH, I may be 
totally of base and my ideas may have already been raised and 
considered <g>.

From: Keith Moore <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Thu, 04 Jul 2002 15:15:10 -0400
Sender: [email-masked]

> IMO, the ultimate aim (even if this is not practical) is to deliver
> to each recipient the same version of the Fax as they would have
> received IF they were the only addressee 

I think this is a reasonable and practical goal.  I don't think that 
recipient A should experience a difference in message quality 
(or worse, fail to receive the message at all) based on whether or 
not you cc'ed recipient B.

> while at the same time
> generating only the minimum number of copies to allow
> multi-addressing.

I think this is an implementation decision.  It should be *possible*
to do this (and it is), but it shouldn't be required of implementations.

here are a couple of trivial examples of the effect of a "downgrade to 
LCD format" implementation strategy.  

assume:

recipient A accepts only HTML and plain text
recipient B accepts only PDF and plain text

an HTML message sent only to A gets delivered as HTML, but if B is cc'ed 
(or bcc'ed) on the message then both A and B get plain text (if the 
client knows how to convert HTML to plain text).  A is accustomed
to receiving HTML from B and is surprised that there's an information
loss.

more seriously, a PDF message sent only to B gets delivered as PDF, but 
if A is cc'ed on the message then the message is bounced for both A and 
B because there's no good way to downgrade PDF to a format acceptable 
to B (or the client doesn't know how).  

So far I have not been able to identify conditions for which downgrading 
to LCD is a viable implementation strategy.  My conclusion is that
implementations that support CONNEG and that attempt to deliver to
mutliple recipients in one transaction MUST be prepared to do RSET and 
split up the delivery if it becomes evident that delivering the same 
version of the message to all recipients is not appropriate.

Keith

Date: Thu, 4 Jul 2002 13:58:59 -0400
To: [email-masked], [email-masked], [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation   to Proposed Standard
Sender: [email-masked]

At 14:27 -0700 on 07/03/2002, Dave Crocker wrote about Re: Last Call: 
SMTP Service Extension for Content Negot:

>At 04:40 PM 7/3/2002 -0400, Robert A. Rosenberg wrote:
>  >CONNEG appears at its root to be an attempt to avoid the "wasted"
>  >bandwidth and processing time/effort to create a M-P/A MIME message most
>>of whose content the Recipient is going to ignore after receipt. It also
>>removes the requirement that the Recipient is even capable of parsing a
>>M-P/A MIME message to locate the "correct" version of FAX which it can
>>understand/process.
>
>Hmmm.  As I recall, the specification was not developed with this goal in
>mind.  However I think that you are pointing out a trade-off in the use of
>ESMTP/CONNECT and Multipart/Alternative that is largely correct.
>
>(I say largely rather than completely because there are some feature
>combinations that define so many combinatorials, they could not be
>considered for M/A but are straightforward for CONNEG.)
>
>Still, you do raise an interesting perspective.
>
>d/
>

Thank you. I am looking at the issue from a overview perspective 
(IOW: How does CONNEG work vs. How would M-P/A work). Since I was not 
involved in the development of CONNEG, I do not know the intent of 
the developers - All I can comment on is the effect of its operation 
in comparison with other methods and state what the EFFECT of the 
choices made are (while taking the effect as the intent even though 
this linkage is probably spurious).

As to the multiple combinations, that was covered by my " 'wasted' 
... processing time/effort". Beyond a certain point, the generation 
of multiple versions (just to be complete) when only one (or a 
limited number) are going to ultimately be used is an exercise in 
futility.

The advantage of CONNEG is that you get the information to generate a 
full list of the different combinations you must produce. The issue 
is what you do then. Do you generate the requisite number of 
combinations and send them out as the minimal number of message 
copies (lumping together those addresses that can share a copy - even 
if that copy is not an exact match to the full capabilities of the 
receiver as in the case of sending a B&W Fax to a Color Capable 
Recipient).

IMO, the ultimate aim (even if this is not practical) is to deliver 
to each recipient the same version of the Fax as they would have 
received IF they were the only addressee while at the same time 
generating only the minimum number of copies to allow 
multi-addressing. IOW: A one copy (/limited number of copies) LCD 
"solution" is not the "Best" solution since it will produce a Fax 
that is not the same as would have been delivered if there were no 
other addresses.

Date: Wed, 03 Jul 2002 22:51:45 -0700
To: John C Klensin <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 01:41 AM 7/4/2002 -0400, John C Klensin wrote:
>   With a
>sufficiently complex (non-binary and non-orthogonal) capability
>set, I suspect the computation might even turn out to be
>NP-hard.

With enough thrust, a pig can fly, but that is not a relevant concern for 
building an airline.

The mechanism here provides a means of communicating arbitrary capabilities 
data.  It is one of multiple ways to communicate that data.

If you have a concern about the nature and interactions of the data, then 
that concern needs to be pursued separately from the communication mechanism.

It might also be worth noting how thoroughly abstract your concern is.  In 
fact it is so abstract that it does not qualify to call it unfounded.

On the average, such generic concerns fail to facilitate forward progress 
or protocol improvement.  Quite the contrary.

>A few more constraints, or some explicit guidance that a client
>that gets sufficiently confused is quite justified in just
>giving up, would go a long way here.

Or more thrust.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

Date: Wed, 03 Jul 2002 19:12:12 -0700
To: John C Klensin <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Cc: "Robert A. Rosenberg" <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 09:31 PM 7/3/2002 -0400, John C Klensin wrote:
>   Would you
>expect the client to work its way through those combinatorics in
>real time,

yes.

no matter what the details are, computing the intersection is not that 
difficult and the computation of such issues does not take that long.  No 
matter what the computations are.

As to the question of real-time generation of the chosen transform(s), that 
bridge was crossed as soon as the client went down the path of CONNEG.  In 
other words, if the client wants to do ESMTP/CONNEG, it has better be 
programmed to do the necessary real-time transforms.

>Alternately, the client could notice that the different
>recipients had different capabilities (_that_ ought to be fairly
>easy in real time, especially if each of them announced its
>capabilities in the same order and using the same syntax --
>which the spec doesn't seem to require)

doesn't use the same syntax?

as to order:  were we doing data processing of millions of records, fixed 
position formats might make sense.  We don't do it for message or content 
headers, why should we do it for these attributes?

>  but the potentially-long SMTP connection
>times (in the RSET, figure out the formats and build them while
>the SMTP connection is still open, and then send them
>separately) or abandoning the connection and starting over

and RSET is markedly inefficient, yes.  that's clear.

my personal expectation is that a client will either be designed to send 
the least common denominator of the address list response capabilities, or 
it will send the addresses one per DATA and tailor each.

the alternative that has been the subject of such delightful speculation, 
here, with the RSET required, strikes me as worth very little of any 
implementor's time.

>Just trying to understand the expected situation...

The spec is pretty simple, and it is mostly based on a set of previous work 
involving CONNEG.

The situations for the spec's use are also pretty simple.

The actual uses will no doubt have some surprises, because new features 
always have surprises.

So what *I* would like to understand is where this is going.

There are some small writing errors in the specification that have been 
noted and will be changed.  There are some suggestions for clarification 
and/or guidance that seem pretty reasonable, to.

As to the rest of the discussions about the thread, I have so far failed to 
detect any analysis that discloses a failure in the specification, of the 
type: "it won't work".

So, as I say, what *I* would like to understand is where this is going.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

Date: Wed, 03 Jul 2002 14:27:58 -0700
To: "Robert A. Rosenberg" <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 04:40 PM 7/3/2002 -0400, Robert A. Rosenberg wrote:
>CONNEG appears at its root to be an attempt to avoid the "wasted" 
>bandwidth and processing time/effort to create a M-P/A MIME message most 
>of whose content the Recipient is going to ignore after receipt. It also 
>removes the requirement that the Recipient is even capable of parsing a 
>M-P/A MIME message to locate the "correct" version of FAX which it can 
>understand/process.

Hmmm.  As I recall, the specification was not developed with this goal in 
mind.  However I think that you are pointing out a trade-off in the use of 
ESMTP/CONNECT and Multipart/Alternative that is largely correct.

(I say largely rather than completely because there are some feature 
combinations that define so many combinatorials, they could not be 
considered for M/A but are straightforward for CONNEG.)

Still, you do raise an interesting perspective.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

Date: Wed, 3 Jul 2002 16:40:36 -0400
To: [email-masked], [email-masked], [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard
Sender: [email-masked]

At 18:45 -0700 on 07/02/2002, Dave Crocker wrote about Re: Last Call: 
SMTP Service Extension for Content Negot:

>  >  the client
>>is being asked to make a convert-or-fail decision without
>>any reasonable (sender-supplied or standardized) criteria as
>>to what kinds of conversions are acceptable.
>>
>>This seems like another major omission in the current spec.
>
>You again seem to be missing the fact that that "omission" predates CONNEG
>and has been part of Internet mail for 8 years.

Fixing/Avoiding that "Omission" it seems to me to be the 
job-of/use-for Multi-Part/Alternate. The Sender creates all the 
Permutations/Combinations they want and sends the total package. It 
is then the job of the Recipient to wade through the mess and select 
the versions that it wants to accept for display/processing.

CONNEG appears at its root to be an attempt to avoid the "wasted" 
bandwidth and processing time/effort to create a M-P/A MIME message 
most of whose content the Recipient is going to ignore after receipt. 
It also removes the requirement that the Recipient is even capable of 
parsing a M-P/A MIME message to locate the "correct" version of FAX 
which it can understand/process.

I apologize if I am stating the obvious but sometimes it is needed to 
do so to refocus the discussion (the "Forest for the Trees" 
Phenomenon).

To: Dave Crocker <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
From: [email-masked]
Date: Tue, 02 Jul 2002 23:55:12 -0400
Sender: [email-masked]

--==_Exmh_-233041992P
Content-Type: text/plain; charset=us-ascii

On Tue, 02 Jul 2002 18:45:32 PDT, Dave Crocker said:

> >            however if pursuing this path it would still be necessary to
> >            leave room for VOICENEG or MIMENEG or whatever,
> 
> If you would supply some sort of functional description for these, it might 
> be possible to understand how the current specifications fails to satisfy 
> them.  I have no idea what you are intending, and therefore cannot even 
> guess at the deficiencies you are implying.

I think that's Keith's POINT - that this has gotten to "Last Call" and there's
no understanding on the part of the supporters what the "voice" or "mime"
communities want, or whether this can/should be extended to address their
concerns as well.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_-233041992P
Content-Type: application/pgp-signature

--==_Exmh_-233041992P--

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Tue, 02 Jul 2002 22:06:10 -0400
Sender: [email-masked]

> Keith,
> 
> At 09:14 PM 7/2/2002 -0400, Keith Moore wrote:
> >         1. do the necessary protocol fixes to make it generally
> >            applicable for email
> 
> Given that we DID make this generally applicable for email, someone still 
> needs to list the way(s) the current specifications is deficient in that 
> regard.  What capabilities or scenarios need to be support that cannot be 
> with the current specification?

go back and reread my earlier messages.

> >         2. make clear that it has only limited applicability
> >            for smtp-to-fax gateways,
> 
> Perhaps you missed the portion of my response that pointed out that it is 
> not limited to that use and, in fact, that is not even its primary intended 
> use.  If there is something in the specification that indicates otherwise, 
> please point it out.

I've done so.  go back and reread my earlier mssages.

> >            same capabilities, and where relaying and forwarding
> >            were specifically disallowed.
> 
> There is a difference between being optimized for single-user and/or 
> single-hop scenarios, versus prohibiting multi-user and/or multi-hop 
> scenarios.  As nearly as I can tell, you are missing that difference.

the current document leaves relaying and forwarding undefined.
this needs to be fixed.  it might be a simple fix, but it's still an
omission.

> >            however if pursuing this path it would still be necessary to
> >            leave room for VOICENEG or MIMENEG or whatever,
> 
> If you would supply some sort of functional description for these, it might 
> be possible to understand how the current specifications fails to satisfy 
> them.  I have no idea what you are intending, and therefore cannot even 
> guess at the deficiencies you are implying.

I'm not going to waste the time to try to specify something which I'm
clearly convinced is a Bad Idea.  

> >- Also, unless the client is acting directly on behalf of the
> >recipient (which is not required by the draft)
> 
> Please specify what you mean by "acting directly on behalf of the 
> recipient" and how the current specification fails to provide the necessary 
> technical specification.

I've already done so, by saying that the recipient needs a standard way
to set the conneg string that is returned.  

> >  the client
> >is being asked to make a convert-or-fail decision without
> >any reasonable (sender-supplied or standardized) criteria as
> >to what kinds of conversions are acceptable.
> >
> >This seems like another major omission in the current spec.
> 
> You again seem to be missing the fact that that "omission" predates CONNEG 
> and has been part of Internet mail for 8 years.

I'm not aware of a content-specific convert-or-fail decision that has to be 
made as part of Internet mail, much less one that has been around for 8 years.

Keith  

Date: Tue, 02 Jul 2002 18:45:32 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

Keith,

At 09:14 PM 7/2/2002 -0400, Keith Moore wrote:
>         1. do the necessary protocol fixes to make it generally
>            applicable for email

Given that we DID make this generally applicable for email, someone still 
needs to list the way(s) the current specifications is deficient in that 
regard.  What capabilities or scenarios need to be support that cannot be 
with the current specification?

>         2. make clear that it has only limited applicability
>            for smtp-to-fax gateways,

Perhaps you missed the portion of my response that pointed out that it is 
not limited to that use and, in fact, that is not even its primary intended 
use.  If there is something in the specification that indicates otherwise, 
please point it out.

>            same capabilities, and where relaying and forwarding
>            were specifically disallowed.

There is a difference between being optimized for single-user and/or 
single-hop scenarios, versus prohibiting multi-user and/or multi-hop 
scenarios.  As nearly as I can tell, you are missing that difference.

>            however if pursuing this path it would still be necessary to
>            leave room for VOICENEG or MIMENEG or whatever,

If you would supply some sort of functional description for these, it might 
be possible to understand how the current specifications fails to satisfy 
them.  I have no idea what you are intending, and therefore cannot even 
guess at the deficiencies you are implying.

>- Also, unless the client is acting directly on behalf of the
>recipient (which is not required by the draft)

Please specify what you mean by "acting directly on behalf of the 
recipient" and how the current specification fails to provide the necessary 
technical specification.

If the specification contained whatever language about "acting directly on 
behalf" you are seeking, how would it change the normative technical 
specification?

>  the client
>is being asked to make a convert-or-fail decision without
>any reasonable (sender-supplied or standardized) criteria as
>to what kinds of conversions are acceptable.
>
>This seems like another major omission in the current spec.

You again seem to be missing the fact that that "omission" predates CONNEG 
and has been part of Internet mail for 8 years.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked], Keith Moore <[email-masked]>, Dave Crocker <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Tue, 02 Jul 2002 21:14:56 -0400
Sender: [email-masked]

a few brief comments on John's comments, not all of which have to do with
the conneg-over-smtp draft:

- I see two possible paths to "fixing" this proposal

	1. do the necessary protocol fixes to make it generally 
           applicable for email

	2. make clear that it has only limited applicability
	   for smtp-to-fax gateways, either supporting a
	   single recipient or where all recipients have the
	   same capabilities, and where relaying and forwarding
	   were specifically disallowed.

	   part of making this clear would be to clearly label the extension -
	   not "CONNEG" or "Content_Negotiation" but FAXNEG or "Fax_Negotiation"

	   however if pursuing this path it would still be necessary to 
	   leave room for VOICENEG or MIMENEG or whatever, and this still
	   probably means NOT overloading RCPT.
	
I strongly prefer alternative #1, because I believe it leads to a less
complex and less costly end-state.  In other words I'd far rather have
a single SMTP extension that makes sense in an integrated messaging world 
that can also be used by specialized services, than to have separate 
extensions for (for lack of a better term) "workstation mail", internet 
fax, voice mail,  etc.

I do however recognize that there is probably pressure to do this 
soon rather than to wait until there can be consensus from 
all of the various parties.  However, for this same reason I doubt 
that the #2 approach should result in a standards-track document.

- I also feel compelled to say that waiting until IETF Last Call to get 
public review of any SMTP extension from the wider SMTP community 
is a very questionable strategy.  This statement could be generalized 
by replacing "SMTP" with any other protocol name that has a significant
number of implementations and/or installed base.

- Regarding the idea for a NORELAY extension: I think we would do well
to encourage more direct delivery for all kinds of email.  The email 
system seems to be becoming more fragile all the time, thanks in large
part to spam and viruses and (often poorly chosen or misconfigured) 
countermeasures for these.    The more hops that a message has to 
travel, the lower the probability that it will be delivered.

However it's not immediately clear to me how a NORELAY extension helps -
will senders (fax or otherwise) really be willing to request it?

- In a similar vein, I think we should reconsider the traditional 
encouragement for SMTP receivers to accept RCPT commands without
verification.  The reason is that the mail system  is now considerably
burdened by spam that is often sent to invalid recipient addresses,
and which often has an invalid return address.  Such messages
(and the resulting bounces) clutter up the mail queues and postmaster 
mailboxes where they're generally discarded because most postmasters
don't have time to read them any more.  Early detection of invalid
recipient addresses allows the message to be immediately rejected
rather than bounced, reducing the load on the receiving MTA.

What we really should do is encourage MTA implementors to provide
the necessary hooks to make possible early detection of bad recipient
addresses.

- regarding failures due to requesting CONNEG

> (i) So far, it has been a basic assumption of the ESMTP
> environment that a server advertising a particular capability
> does not (I'd have to go back and look, but I think perhaps MUST
> NOT) reject an attempt to request the desired capability.   I
> understand the reasons here, given the general model, but
> returning a 504 to "RCPT TO:<baz@bar> CONNEG" (implying
> "REQUIRED") would seem to violate that principle.

perhaps more to the point, I think the current semantics are broken.

if the server cannot supply a conneg string for a particular
recipient (because it does not have one or because the sender
lacks permission or whatever) the only reasonable behavior is
for the server to report that fact (which should NOT be a 
server-reported error)

at this point the client can either fail to deliver the message
(because it's not willing to downgrade that far) or it can 
assume that the recipient has only minimal capabliities (e.g. 
simple mode) and do the required downgrade to that mode.

but in the unwilling-to-downgrade case it's the CLIENT's job 
to assert the error, not the server's job.  the client may have
to assert the error in any cases, since it may not be possible
to reasonably downgrade the message to suit the recipient's
capabliities even if the recipient does support CONNEG.
having the server assert the error just makes things more complex
without adding any real functionality.

- Also, unless the client is acting directly on behalf of the
recipient (which is not required by the draft) the client
is being asked to make a convert-or-fail decision without
any reasonable (sender-supplied or standardized) criteria as 
to what kinds of conversions are acceptable.  

This seems like another major omission in the current spec.

Keith

Date: Tue, 02 Jul 2002 19:45:19 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Keith Moore <[email-masked]>, Dave Crocker <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Sender: [email-masked]

IESG,

I want to partially agree with Keith -- this proposal makes me
very nervous and I believe the IETF should share that
nervousness.  Since Keith and Dave are more than capable of
discussing the various issues that have been identifed to the
point of clarity, I want to take a slightly different cut at the
problem and tentatively suggest a different approach.

For me, the root problem here is that the proposal seems to
simultaneously say:

	* The MTA here knows a good deal about the MUAs
	associated with each of the recipients.
	
	* The MTA is able to support a relay function.

It appears to me that, even ignoring layering violations, having
both of those things be true requires either a good deal of
negotiation among a set of relays and the final target host
--going beyond the "next relay will take it and accept
responsibility" model that some SMTP extensions use, e.g., to
support non-7bit-transport-- or there has to be some magic.  I
don't see either in a quick reading of the document.

More generally, I think we should hesitate about SMTP extensions
whose purpose is to support a particular application, without
understanding all of the implications of generalizing them.  I
think this is quite similar to one of Keith's points.

But I believe it has been a characteristic of much of the fax
work so far that relaying is not anticipated (this draft, in
repeatedly commenting about "direct" connections, reinforces
that belief).  Indeed, we have text floating around that
suggests that Internet Fax setups be configured to avoid relays,
just as we have had similar suggestions about EDI-over-email
applications.  Sending a virtual fax to a fax broadcast device
is another matter, of course: to the degree to which any action
takes place in the message content, we've traditionally thought
about that as a UA function, with the broadcast device acting as
a receiving SMTP, delivering the message to the relevant UA (no
matter how automated), which then initiates a completely new
SMTP connection (or set of connections).  Or we have modelled
the broadcast device as a fax-capable SMTP gateway, with some
different mail transport on the far side, which completely
changes the rules.

This is all leading to a constructive suggestion:  Unless there
is a clear need for an SMTP relay function that follows SMTP
rules --e.g., the relay MTA doesn't look at message content at
all-- it may be time for a new SMTP extension/ capability
announcement to deal with these specialized applications that
use SMTP as a message transport but that really aren't email in
the normal sense.  

One version of such an extension would be "NORELAY".  An SMTP
server advertising it would guarantee, if the client requested
that option (presumably on MAIL or a separate command -- this
makes no sense on a pre-recipient basis) to either deliver a
message or bounce it, but, under no circumstances to relay it
off to another SMTP server.  Of course, a server advertising
that extension (or not) could refuse relaying even if the client
didn't request the option: we never require servers to relay.

In the presence of such an option, content negotiation that
would require that the server know about its local MUAs becomes,
IMO, somewhat more plausible.  We have always permitted
originating and delivery MTAs more flexibility than intermediate
ones.

In this particular case, if there was no relaying and the
capabilities of the receiver were really server capabilities,
rather than MUA ones (as would be the case for a dedicated
Internet fax device or a distribution system that simulated
one), the whole picture could be considerably simplified by
treating capability negotiation on a per-SMTP-connection basis,
rather than a per-recipient one.

Is it possible that something of that nature would make sense?

-------------

A few other issues that Keith didn't seem to pick up on (to my
surprise):

(i) So far, it has been a basic assumption of the ESMTP
environment that a server advertising a particular capability
does not (I'd have to go back and look, but I think perhaps MUST
NOT) reject an attempt to request the desired capability.   I
understand the reasons here, given the general model, but
returning a 504 to "RCPT TO:<baz@bar> CONNEG" (implying
"REQUIRED") would seem to violate that principle.

(ii) In order to facilitate efficient relaying and processing of
messages with large numbers of recipients, we permit, indeed
almost encourage, SMTP receivers to accept RCPT commands without
verification and then to prepare bounce messages if needed.  I
strongly suspect that there are conforming MTAs that can behave
no other way.  It seems to me from this specification that a
server that advertises CONNEG must not perform this types of
deferred operation, at least with regard to that parameter, but
must handle errors within the SMTP transaction.   If that
requirement actually exists, I believe it is very important that
the document point it out.

Also, if relaying is permitted, then either the first relay in
the chain has to have complete knowledge about what will be
permitted downstream (and the document doesn't give any
indication as to how it would find that out), or emailed
rejection messages become an absolute requirement (at least
unless no response is issued to any RCPT TO until the final
delivery MTA has generated a 250 code and returned that along
the line -- I suppose that would be possible, but it would
require yet other SMTP extensions to specify the behavior and it
would probably be a performance disaster).

(iii) I can't reconcile the last statement of section 4.2:

	If the server returns an EHLO 250 code without CONNEG
	capabilities, the client MUST work as "a Simple Mode of
	Facsimile Using Internet Mail" [RFC2305].

With the assertion that this is a general facility, not limited
to fax.   If it is a duck, let's call it a duck and be done with
it.  And, if fax is sufficiently special that we need things
like this, perhaps it is time to add something like NORELAY, or
a FAX extension, to make it clear that what is happening is not
normal SMTP.

(iv) Like Gregory Neil Shapiro, I'm not sure what a "target" is
and I'm not sure the term is used consistently.  E.g., in 4.2,
it appears as if the "target" is not the receiving SMTP server,
but is rather the mailbox (and, presumably, MUA) listed in the
RCPT command.  But 4.3 says "... the target MUST reject..."
which strongly implies an SMTP server action.

(v) Just as an editorial matter, but, in order to avoid
unnecessary confusion, it would probably be helpful if the
document adopted terminology consistent with RFC 2821 (and 821).
E.g., there is no such command as "RCPT-TO", the name of the
relevant command is "RCPT".

--------

To paraphrase advice Dave has given on many other occasions,
let's divide this set of issues up and solve the problems that
are well-defined and that we know how to solve and leave the
rest until there is a demonstrated need.  The problem here, as I
understand it --both generally and after reading the
introduction to this draft-- is fax and fax-like
start-of-session 
capabilities negotiation in a direct connection environment.
Let's think about using SMTP extension mechanisms, if necessary,
to specify/require that case and just avoid generalizations that
require us to deal with all of the poorly-understood (or at
least unspecified in this document)	relaying, intermediary
server, and other issues. 

     john

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Tue, 02 Jul 2002 10:33:32 -0400
Sender: [email-masked]

> At 01:53 AM 7/2/2002 -0400, Keith Moore wrote:
> >executive summary:  I recommend  against approval of this document
> >withut extensive changes.  It's not well thought-out
> 
> What additional thinking out are you looking for?

see below.
 
> >  (except for the rather narrow application of SMTP to a one-recipient fax 
> > gateway)
> 
> In fact the motivation for it is not necessarily a gateway, but rather any 
> recipient environment that conforms to this profile, including usage that 
> is wholly retained among Internet-capable participants.
> 
> For that matter, the mechanism applies on a per-recipient basis, so what 
> prompts the claim that it works only for a single recipient?

if different recipients have different capabilities, the sender has
to either send a single message with a format that is suitable for
all recipients, or it has to do a RSET and resend the RCPT commands
separately for each version of the message that it sends.  nowhere
in the document do I see any mention of this, and furthermore it
complicates error handling.  so I assume that it wasn't adequately 
considered.

> >and it seems likely to degrade SMTP interoperabliity if widely adopted.
> 
> How will that happen?  How is it likely?

because the proposal complicates the SMTP state machine (especially when 
you consider the multiple recipient case), it adds additional
failure modes, and the error handling is not well described.
 
> >1. layering problems
> >
> >this functionality is a poor fit for SMTP because SMTP is the
> >mail *transport* protocol, and as such should not be dealing
> >with content at all, which is a user agent function.
> 
> The construct of using SMTP to verify support of a higher level feature is 
> not all that different from DSN. 

now that's a bizarre statement.  DSN is a per-mta feature.  it is either
supported or not supported by an MTA.  it doesn't modify content.  
DSN doesn't reveal anything about a recipient's user agent. 

> >but what's to stop this mechanism from getting out-of-sync also?
> 
> The end-to-end asynchrony and latency for this mechanism certainly do 
> provide an opportunity for a recipient's capabilities to change, after it 
> has advertised a particular set.  However the key point to note is that 
> there is no mediating storage/directory service, so the advertisement has a 
> degree of directness not possible through any other mechanism.

Dave, the document itself recognizes the possibility of a directory
service, even listing a specific error code for this case, so your statement 
is facially false.  Furthermore it must be recognized that the vast majority
of MTAs do not interact directly with user agents at all, so the 
"degree of directness" you claim is completely bogus.

> For recipients concerned about such volatility, all they have to do is 
> refrain from advertising CONNEG support.  

in other words, what you are saying is that this extension has very
narrow applicabliity - it is not  usable on the vast majority of
message transfer agents, but only on those that are tightly coupled
with an MUA or gateway that applies to all recipients.  

so why are we complicating SMTP with an extension that only applies
to a very narrow case?  is support for fax really so important 
that we have to extend SMTP in such a way that it's only usable for
fax, and in such a way that fax-capable MTAs are a completely different
class than other MTAs?

> For senders concerned about such 
> volatility, they should use CONNEG=Required.

the volatility isn't really the issue I'm concerned about - the issue 
is the lack of a standard mechanism for communication of capability
sets from the UA (or the user) to the inbound MTA; and more generally, 
the difficulty of using the CONNEG mechanism to support users who
use multiple UAs with different capabilities.  at least, the latter
scenario doesn't seem to have been considered.

> >it seems like this extension has very limited applicability - it will
> >only work reliably when the SMTP server and user agent or gateway are
> >very closely tied and when the recipient always uses the same UA
> >or UAs of very similar capabilities.  (certainly not true of me!)
> 
> 1.  The specification already highlights that it is tailored for rather 
> 'direct' scenarios.

true, but it also says that indirect ones are possible.

> 2.  Each user can decide what to advertise.  

HOW?  there's no mechanism defined that allows them to do so.

> If they vary their UA they can 
> choose to advertise a safe and lesser set of capabilities.  In any event, 
> it is their choice.  The mechanism does not impose any problems that are 
> not inherent in use of multiple UAs.  However it DOES permit relatively 
> direct advertising of capabilities, which is currently not feasible.
> 
> 
> >but if we're going to extend SMTP to perform functions
> >on behalf of the user agent, it needs to be made very explicit that this
> >negotiation is being done on behalf of the user agent - and in particular
> >that the negotiation being requested represents the preferences of the
> >*user*
> 
> In what way does that affect the technical specification?

because if third parties make unauthorized modifications to messages
then the end-to-end integrity of the message is destroyed.

is that technical enough for you?

> That is, how is the current specification deficient, with respect to your 
> concern?
> 
> 
> >in particular there seems to be no
> >ability to apply content-transformation on a per-recipient basis,
> >to handle the case where different transformations are acceptable to
> >differnet recipients.
> 
> We must be reading different specifications.  Choice of enforcement of this 
> option is per-recipient.  Each recipient may generate a capabilities 
> response if they wish a different form of the content.
> 
> What, exactly, are you looking for that is missing?

reread my earlier message, and see above.

> >of course it's not easy to get SMTP to handle different client-side
> >transformations for different recipients, since SMTP is designed to
> >transfer the exact same message body (modulo received fields) to each
> >of a set of recipients.
> 
> It sounds as if you are assuming that later transmission of content that is 
> revised according to capabilities information is required (or expected) to 
> be sent using that same address list.

well, by the time the sender knows what the recipients' capabilities are,
the receiver already thinks it has the recipient list, and it is expecting
a single messgae to be sent to all of those recipients.

maybe you need to read the draft you claim to have co-written?

> 
> That is like requiring that all BCC copies are sent along with the TO and 
> CC copies.
> 
> In other words, there is not such requirement.  It is OK to do it and it is 
> OK not to.
> 
> Yes, the current CONNECT option is intended to send the same content to all 
> recipients, since that is the style of SMTP.  There is nothing preventing 
> later sending an alternative content form to selected recipients.

so you're suggesting that multiple copies of the message be sent to those
recipients?  

> Interestingly, this would not actually create any additional round-trips or 
> additional content transfers.

that's about the most bizarre thing I've ever seen you write...

> 
> >  this is a different model than used by
> >phone-based facsimile protocols which expect there to be only one
> >recipient of any given transmission,
> 
> You might want to take a look at fax broadcasting services.  The last-hop 
> telephone portion behaves as you describe, but only after a 'generic' 
> posting process.

perhaps, and I'm not familiar with these.  but my guess is that they don't
do per-recipient negotiation on earlier hops.

> >  so it's not surprising that the fax-style negotiation doesn't fit well 
> > into SMTP.
> 
> Actually, the proposed scheme permits content transmission tailoring that 
> is considerably more flexible than is typical in the real fax world.
> 
> 
> >in order to make client-side negotiation a clean fit into SMTP, it is
> >necessary for the client to first query the server for the capabilities
> >of each intended recipient, then to deduce how many total transformations
> >are necessary, then to send one or more transformed versions of the message.
> 
> Ahh.  I see.  You want to insist on an interaction scenario that is known 
> to be unacceptable, since it is guaranteed to introduce too much delay 
> before there is any utility at all.

PIPELINING would make the delay negligible.  and it's far preferable to 
overloading the RCPT command.
 
> >   overloading the RCPT
> >command to return capabilities breaks the RCPT command and forces
> >the client to do RSET in order to send different versions to
> >different recipients.
> 
> If that is what the client chooses to do, then yes.  However there are 
> other courses of action available.  At the least, note that the content is 
> not sent until the client has CONNEG responses about all of the recipients.

perhaps you'd like to describe those "other courses" which allow the
client to meet the requirement that it deliver the highest quality 
content to each recipient?  or for that matter, that allow the client
to satisfy mutually conflicting needs of recipients?  (e.g. one recipient
will only take gif while another will only take jpeg)

> >putting content negotiation in the transport protocol is likely
> >to result in a significant increase in anomolous behaviors
> >from the mail system which are difficult to diagnose and fix -
> >and the proposal is being introduced at a time when the mail system
> >is already suffering from serious reliabliity problems (often due
> >to ill-conceived measures for blocking spam).
> 
> How does use of the CONNEG mechanism affect reliability or accountability?

it complicates error handling.

> >   at this time,
> >*any* proposal for adding new functionality to SMTP should be viewed
> >with strong reservations, unless it somehow offers an improvement
> >in reliability.
> 
> I was not aware that we had any problems being attributed to SMTP 
> options.  Please provide some citations, so we can compare their details to 
> the current proposal.

the problems are with SMTP in general.  if you want citations, go read
delivery failure notices from a few large mailing lists.

> 
> >4. naming and applicability
> >
> >the protocol claims to be a general-purpose content-negotiation scheme
> >but actually has "fax" assumptions embedded very deeply.
> 
> The fax world is the basis for the idea, yes.  However the specification 
> has been done so as to be independent of fax, thereby using an approximate 
> model that is known to be useful from the fax world and then applying it 
> for general enhancement of email.

It's evident that not very much thought has been invested in making
the non-fax case work.

> >first of all, it assumes that there is a linear ordering relationship
> >for document suitability to the recipient.
> 
> Ok.  I give up.  I have no idea what "linear ordering relationship for 
> document suitability" means.

it means that the proposal assumes there is a notion of "higher quality" 
vs "lower quality" versions of a message that can be determined for all 
messgaes and that is obvious to an implementor.

> >a.  I strongly recommend that you redesign this extension so that, instead
> >of overloading the RCPT command, you define a separate command to
> >request a recipient's capabilities.  e.g.
> 
> In spite of the proffered benefits of a separate command, I fail to see how 
> it produces any real benefits.
> 
> It certainly does not make anything simpler, and it certainly vastly 
> increases relay delay, due to nearly doubling the round-trip exchanges 
> during an SMTP session that uses the option.

it gets you out of the "oops I don't really want to send this message to
that recipient because that recipient needs a different format of the
message" trap.

as far as round trips, RCAP RCAP RCAP RCPT DATA RCPT DATA RCPT DATA
is slightly better than RCPT RCPT RCPT RSET RCPT DATA RCPT DATA RCPT DATA

but you really ought to pipeline these things anyway.  

> >one nice side-benefit is that it doesn't change the error handling
> >model,
> 
> How does the current proposal change the error handling "model"?

by introducing new error conditions that require different handling
than existing errors.

> >b. the idea that the sender should send the "best" or "highest
> >quality" version needs to be relaxed or generalized,
> 
> It already has been relaxed.  Note the SHOULD rather than MUST.

it's still a conditional requirement of the specification that
is so vaguely defined as to be unimplementable except perhaps
for the narrow fax case.  different implementors could easily interpret
the SHOULD in different ways even absent any conditions that would allow
them to disregard the SHOULD.  in other words, it doesn't produce 
predictable behavior.

> >  since there's currently no clear notion of what this means.
> 
> Yet we have lived with that notion in multipart/alternative for some years.

in retrospect we might consider mp/alternative a bug.
I don't think citing it adds weight to your argument.

> 
> >c. this extension needs to be made less fax-specific.
> 
> What, specifically, makes it fax specific, and what changes are you 
> specifically seeking that will be less fax-specific?
> 
> 
> >it needs to be examined with some realistic non-fax-specific
> >test cases, and it needs non-fax conneg examples also.
> 
> Test cases are always a good thing.  Feel free to offer some that satisfy 
> your non-faxiness goal.

hey, you're the one making the proposal, seems like it's your job to
make sure it really has the applicability that you claim it does.  

I don't really think the proposal provides enough benefit to make it
worthwhile even if it's done right. but rather than rejecting it out 
of hand (and recognizing that the fax community wants this badly)
I'm trying to offer some constructive criticism that would make it 
less objectionable.

> >d. there needs to be a good way for the recipient to specify
> >capabilities to the SMTP server, at least for any SMTP server
> >that can serve arbitrary UA recipients.  probably this means another
> >extension along with SMTP AUTH.
> 
> You want to require that that mechanism be in place before the current 
> specification can be advanced?  I think the phrase "chicken and egg" works 
> here.  You create a deadly embrace for reaching any utility.

I think it's called 'having a complete specification'.   I didn't
say anything about having the mechanism deployed, but I do think it
needs to be defined.

> >still, this is a big can of worms, because there's never been
> >any ability of a UA to specify something to its inbound SMTP
> >servers before - there's presently not even a good way for
> >a UA to know which server(s?) should be given this information...
> 
> Think hard.  What is likely to drive creation of such a 
> mechanism?  Requiring that it be built in the abstract or looking for it to 
> be built in reaction to actual demand?

no argument there - but my point is that this is breaking new ground, 
and as such there are many issues that need to be considered that we
haven't thought of before.

> 
> >e. make it absolutely clear that such transformations are only to be
> >made with explicit authorization by the sender and/or recipient.
> 
> This sort of parental direction in IETF specifications probably does no 
> harm, but it is difficult to imagine that it does much benefit.

tell that to the OPES people who were trying to push for web 
proxies that would modify content without either the provider's
or consumer's consent.     the last thing I want is an ISP 
advertising content capabilities on my behalf without my authorization -
and I can think of some large ones that would do exactly that.

I used to think we could rely on each layer to maintain the appropriate
degree of transparency for that layer without our demanding that it be so.
history has proven me wrong.  it needs to be specified.
 
> >f. the document needs to consider the effect of message forwarding
> >on the returned string - if a recipient has his mail forwarded
> >elsewhere it's not clear what conneg string should be returned.
> 
> There are all sorts of implications in forwarding.  In general, forwarding 
> that is not simply SMTP relaying is outside of current IETF 
> specification.  Imposing a requirement about it here, without demonstrating 
> actual problems, is a tad extreme.

apparently you'd rather just deploy a new protocol extension without 
even bothering to think about how it affects the infrastructure.

let's just say I disagree.

> >g. the document alludes to the ability to work through intermediaries
> >but this isn't explained anywhere, and it needs to be explained.
> 
> All it says is that it works with SMTP relaying, just as one would expect 
> and SMTP extension tow work.

how does it work with SMTP relaying if there's no defined way to 
relay the capabilities?  either you leave it to the last hop,
which (while fine with me) makes the statement kind of trivial,
or you do something really ugly...  again, a bit of clarification
of this statement wouldn't hurt.

> > > 4.2 Client Action:
> >...
> >this incorrectly assumes that (a) there is only one target when
> >the client can easily specify multiple RCPT commands, and
> >(b) there is a clear notion of "highest level of capability"
> >for the target.
> 
> It does not make the assumption you suggest.  In fact, I can't imagine how 
> you came to the assessment that it makes that assumption.

well, I can't imagine how you came to a number of the assessments you
are claiming.  perhaps we live in different universes.

> 
> > > 4.3 Server Action:
> > >
> > >      If the client specifies "CONNEG = REQUIRED" in the RCPT-TO,
> > >      but the target does not support the CONNEG parameter, the target
> > >      MUST reject the RCPT-TO command with a 504 reply.
> >...
> >(it may be that other extensions have similar wording, in which case
> >they're similarly flawed)
> 
> It is probably a good idea that we not try to fix SMTP (yet again, given 
> the protracted drums effort.)  And it certainly is not a good idea to 
> require that the current specification fix something this vague when no 
> SMTP-related specification is required to.

so we should always write new specifications as poorly as existing ones?

this wasn't so much of a problem when we didn't have so many extensions -
but the more extensions we have the more problems like this we introduce .

> > >      If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the
> > >      target MUST process the address and message as if the requested
> > >      CONNEG capabilities had not been specified.
> >
> >this doesn't seem to make sense as written.  is "CONNEG = OPTIONAL"
> >really supposed to be interpreted the same as omission of the
> >CONNEG parameter?
> 
> I think that the text that is in the first paragraph of that section ("but 
> the target does not support the CONNEG parameter") got deleted during 
> revision.  In general, those first 3 paragraphs are dealing with lack of 
> support, whereas the following paragraphs are dealing with presence of support.
> 
> 
> >also, I don't think there should be spaces around the = sign -
> 
> ack.
> 
> 
> >I wonder here - assuming that this can of worms does not get closed
> >back up, and there might later be demand for other mechanisms
> >to request recipient capabilities (e.g. since this one is not very general),
> >might there be multiple kinds of strings emitted in the RCPT response?
> 
> Please indicate what capabilities are missing from this capabilities 
> mechanism that would make it more general and for which there is a 
> demonstrated need.

actually this one is the least of my concerns about the document.
if the other problems can be fixed this one is easy...
but the fix for this depends to some degree on how you fix the other
problems, so I'll defer making suggestions for now.

Keith

Date: Tue, 02 Jul 2002 00:23:43 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 01:53 AM 7/2/2002 -0400, Keith Moore wrote:
>executive summary:  I recommend  against approval of this document
>withut extensive changes.  It's not well thought-out

What additional thinking out are you looking for?

>  (except for the rather narrow application of SMTP to a one-recipient fax 
> gateway)

In fact the motivation for it is not necessarily a gateway, but rather any 
recipient environment that conforms to this profile, including usage that 
is wholly retained among Internet-capable participants.

For that matter, the mechanism applies on a per-recipient basis, so what 
prompts the claim that it works only for a single recipient?

>and it seems likely to degrade SMTP interoperabliity if widely adopted.

How will that happen?  How is it likely?

>1. layering problems
>
>this functionality is a poor fit for SMTP because SMTP is the
>mail *transport* protocol, and as such should not be dealing
>with content at all, which is a user agent function.

The construct of using SMTP to verify support of a higher level feature is 
not all that different from DSN.  What is the practical problem with 
applying it here?

>but what's to stop this mechanism from getting out-of-sync also?

The end-to-end asynchrony and latency for this mechanism certainly do 
provide an opportunity for a recipient's capabilities to change, after it 
has advertised a particular set.  However the key point to note is that 
there is no mediating storage/directory service, so the advertisement has a 
degree of directness not possible through any other mechanism.

For recipients concerned about such volatility, all they have to do is 
refrain from advertising CONNEG support.  For senders concerned about such 
volatility, they should use CONNEG=Required.

>it seems like this extension has very limited applicability - it will
>only work reliably when the SMTP server and user agent or gateway are
>very closely tied and when the recipient always uses the same UA
>or UAs of very similar capabilities.  (certainly not true of me!)

1.  The specification already highlights that it is tailored for rather 
'direct' scenarios.

2.  Each user can decide what to advertise.  If they vary their UA they can 
choose to advertise a safe and lesser set of capabilities.  In any event, 
it is their choice.  The mechanism does not impose any problems that are 
not inherent in use of multiple UAs.  However it DOES permit relatively 
direct advertising of capabilities, which is currently not feasible.

>but if we're going to extend SMTP to perform functions
>on behalf of the user agent, it needs to be made very explicit that this
>negotiation is being done on behalf of the user agent - and in particular
>that the negotiation being requested represents the preferences of the
>*user*

In what way does that affect the technical specification?

That is, how is the current specification deficient, with respect to your 
concern?

>in particular there seems to be no
>ability to apply content-transformation on a per-recipient basis,
>to handle the case where different transformations are acceptable to
>differnet recipients.

We must be reading different specifications.  Choice of enforcement of this 
option is per-recipient.  Each recipient may generate a capabilities 
response if they wish a different form of the content.

What, exactly, are you looking for that is missing?

>of course it's not easy to get SMTP to handle different client-side
>transformations for different recipients, since SMTP is designed to
>transfer the exact same message body (modulo received fields) to each
>of a set of recipients.

It sounds as if you are assuming that later transmission of content that is 
revised according to capabilities information is required (or expected) to 
be sent using that same address list.

That is like requiring that all BCC copies are sent along with the TO and 
CC copies.

In other words, there is not such requirement.  It is OK to do it and it is 
OK not to.

Yes, the current CONNECT option is intended to send the same content to all 
recipients, since that is the style of SMTP.  There is nothing preventing 
later sending an alternative content form to selected recipients.

Interestingly, this would not actually create any additional round-trips or 
additional content transfers.

>  this is a different model than used by
>phone-based facsimile protocols which expect there to be only one
>recipient of any given transmission,

You might want to take a look at fax broadcasting services.  The last-hop 
telephone portion behaves as you describe, but only after a 'generic' 
posting process.

>  so it's not surprising that the fax-style negotiation doesn't fit well 
> into SMTP.

Actually, the proposed scheme permits content transmission tailoring that 
is considerably more flexible than is typical in the real fax world.

>in order to make client-side negotiation a clean fit into SMTP, it is
>necessary for the client to first query the server for the capabilities
>of each intended recipient, then to deduce how many total transformations
>are necessary, then to send one or more transformed versions of the message.

Ahh.  I see.  You want to insist on an interaction scenario that is known 
to be unacceptable, since it is guaranteed to introduce too much delay 
before there is any utility at all.

>   overloading the RCPT
>command to return capabilities breaks the RCPT command and forces
>the client to do RSET in order to send different versions to
>different recipients.

If that is what the client chooses to do, then yes.  However there are 
other courses of action available.  At the least, note that the content is 
not sent until the client has CONNEG responses about all of the recipients.

>putting content negotiation in the transport protocol is likely
>to result in a significant increase in anomolous behaviors
>from the mail system which are difficult to diagnose and fix -
>and the proposal is being introduced at a time when the mail system
>is already suffering from serious reliabliity problems (often due
>to ill-conceived measures for blocking spam).

How does use of the CONNEG mechanism affect reliability or accountability?

>   at this time,
>*any* proposal for adding new functionality to SMTP should be viewed
>with strong reservations, unless it somehow offers an improvement
>in reliability.

I was not aware that we had any problems being attributed to SMTP 
options.  Please provide some citations, so we can compare their details to 
the current proposal.

>4. naming and applicability
>
>the protocol claims to be a general-purpose content-negotiation scheme
>but actually has "fax" assumptions embedded very deeply.

The fax world is the basis for the idea, yes.  However the specification 
has been done so as to be independent of fax, thereby using an approximate 
model that is known to be useful from the fax world and then applying it 
for general enhancement of email.

>first of all, it assumes that there is a linear ordering relationship
>for document suitability to the recipient.

Ok.  I give up.  I have no idea what "linear ordering relationship for 
document suitability" means.

>a.  I strongly recommend that you redesign this extension so that, instead
>of overloading the RCPT command, you define a separate command to
>request a recipient's capabilities.  e.g.

In spite of the proffered benefits of a separate command, I fail to see how 
it produces any real benefits.

It certainly does not make anything simpler, and it certainly vastly 
increases relay delay, due to nearly doubling the round-trip exchanges 
during an SMTP session that uses the option.

>one nice side-benefit is that it doesn't change the error handling
>model,

How does the current proposal change the error handling "model"?

>b. the idea that the sender should send the "best" or "highest
>quality" version needs to be relaxed or generalized,

It already has been relaxed.  Note the SHOULD rather than MUST.

>  since there's currently no clear notion of what this means.

Yet we have lived with that notion in multipart/alternative for some years.

>c. this extension needs to be made less fax-specific.

What, specifically, makes it fax specific, and what changes are you 
specifically seeking that will be less fax-specific?

>it needs to be examined with some realistic non-fax-specific
>test cases, and it needs non-fax conneg examples also.

Test cases are always a good thing.  Feel free to offer some that satisfy 
your non-faxiness goal.

>d. there needs to be a good way for the recipient to specify
>capabilities to the SMTP server, at least for any SMTP server
>that can serve arbitrary UA recipients.  probably this means another
>extension along with SMTP AUTH.

You want to require that that mechanism be in place before the current 
specification can be advanced?  I think the phrase "chicken and egg" works 
here.  You create a deadly embrace for reaching any utility.

>still, this is a big can of worms, because there's never been
>any ability of a UA to specify something to its inbound SMTP
>servers before - there's presently not even a good way for
>a UA to know which server(s?) should be given this information...

Think hard.  What is likely to drive creation of such a 
mechanism?  Requiring that it be built in the abstract or looking for it to 
be built in reaction to actual demand?

>e. make it absolutely clear that such transformations are only to be
>made with explicit authorization by the sender and/or recipient.

This sort of parental direction in IETF specifications probably does no 
harm, but it is difficult to imagine that it does much benefit.

>f. the document needs to consider the effect of message forwarding
>on the returned string - if a recipient has his mail forwarded
>elsewhere it's not clear what conneg string should be returned.

There are all sorts of implications in forwarding.  In general, forwarding 
that is not simply SMTP relaying is outside of current IETF 
specification.  Imposing a requirement about it here, without demonstrating 
actual problems, is a tad extreme.

>g. the document alludes to the ability to work through intermediaries
>but this isn't explained anywhere, and it needs to be explained.

All it says is that it works with SMTP relaying, just as one would expect 
and SMTP extension tow work.

> > 4.2 Client Action:
>...
>this incorrectly assumes that (a) there is only one target when
>the client can easily specify multiple RCPT commands, and
>(b) there is a clear notion of "highest level of capability"
>for the target.

It does not make the assumption you suggest.  In fact, I can't imagine how 
you came to the assessment that it makes that assumption.

> > 4.3 Server Action:
> >
> >      If the client specifies "CONNEG = REQUIRED" in the RCPT-TO,
> >      but the target does not support the CONNEG parameter, the target
> >      MUST reject the RCPT-TO command with a 504 reply.
>...
>(it may be that other extensions have similar wording, in which case
>they're similarly flawed)

It is probably a good idea that we not try to fix SMTP (yet again, given 
the protracted drums effort.)  And it certainly is not a good idea to 
require that the current specification fix something this vague when no 
SMTP-related specification is required to.

> >      If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the
> >      target MUST process the address and message as if the requested
> >      CONNEG capabilities had not been specified.
>
>this doesn't seem to make sense as written.  is "CONNEG = OPTIONAL"
>really supposed to be interpreted the same as omission of the
>CONNEG parameter?

I think that the text that is in the first paragraph of that section ("but 
the target does not support the CONNEG parameter") got deleted during 
revision.  In general, those first 3 paragraphs are dealing with lack of 
support, whereas the following paragraphs are dealing with presence of support.

>also, I don't think there should be spaces around the = sign -

ack.

>I wonder here - assuming that this can of worms does not get closed
>back up, and there might later be demand for other mechanisms
>to request recipient capabilities (e.g. since this one is not very general),
>might there be multiple kinds of strings emitted in the RCPT response?

Please indicate what capabilities are missing from this capabilities 
mechanism that would make it more general and for which there is a 
demonstrated need.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

From: Keith Moore <[email-masked]>
To: [email-masked]
cc: [email-masked], [email-masked]
Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard 
Date: Tue, 02 Jul 2002 01:53:12 -0400
Sender: [email-masked]

[referencing draft-ietf-fax-esmtp-conneg-02.txt]

executive summary:  I recommend  against approval of this document
withut extensive changes.  It's not well thought-out (except for the
rather narrow application of SMTP to a one-recipient fax gateway)
and it seems likely to degrade SMTP interoperabliity if widely
adopted.   I do suggest some changes which would make it less
harmful, but it still needs considerable work to be of general 
applicability.

1. layering problems

this functionality is a poor fit for SMTP because SMTP is the 
mail *transport* protocol, and as such should not be dealing 
with content at all, which is a user agent function.

I admit that (by design) there is no opportunity for e2e interaction
at the user agent level in the current protocols, and also that any 
mechanism that might be used to advertise user agent capabliities 
(e.g. rescap) introduces the opportunity for that mechanism to get 
out-of-sync with the actual capabilities/preferences of the recipient.
but what's to stop this mechanism from getting out-of-sync also?
it seems like this extension has very limited applicability - it will 
only work reliably when the SMTP server and user agent or gateway are
very closely tied and when the recipient always uses the same UA
or UAs of very similar capabilities.  (certainly not true of me!)

one can argue that extending SMTP to allow it to perform negotiation on 
behalf of the user agent is no worse than putting that negotiation in a 
separate protocol.  but if we're going to extend SMTP to perform functions 
on behalf of the user agent, it needs to be made very explicit that this 
negotiation is being done on behalf of the user agent - and in particular
that the negotiation being requested represents the preferences of the 
*user*  (i.e. the sender and/or recipient) rather than some third party.

IIRC, this is the same thing that was insisted on for the OPES stuff - 
the negotiation/transformation needs to have explicit consent from, and 
be under control of, at least one of the ultimate endpoints
(sender or receiver) and it seems to me that it's appropriate to apply
the same policy for email.

2. protocol design problem

for transformations that should be under control of specific recipients,
the protocol is poorly designed.  in particular there seems to be no
ability to apply content-transformation on a per-recipient basis,
to handle the case where different transformations are acceptable to
differnet recipients.  again, this drastically limits the applicability
of the protocol (it won't work well unless all recipients at that
SMTP server have the same capability set, which is very unlikely for
most conventional SMTP servers)

of course it's not easy to get SMTP to handle different client-side
transformations for different recipients, since SMTP is designed to 
transfer the exact same message body (modulo received fields) to each 
of a set of recipients.  this is a different model than used by
phone-based facsimile protocols which expect there to be only one
recipient of any given transmission, so it's not surprising that
the fax-style negotiation doesn't fit well into SMTP.

in order to make client-side negotiation a clean fit into SMTP, it is 
necessary for the client to first query the server for the capabilities 
of each intended recipient, then to deduce how many total transformations 
are necessary, then to send one or more transformed versions of the message.
for instance, if recipients A, B, E, and F could accept one transformation 
of the message and recipients B, C, and D could could accept a higher 
quality transformation of the message, the client might attempt to send the 
high-quality version to B, C, and D and then send the lower quality 
version to A, E, and F.

but as currently designed the proposed extension doesn't have the 
necessary protocol hooks to permit this - at least not without
a considerable amount of ugliness.  overloading the RCPT 
command to return capabilities breaks the RCPT command and forces
the client to do RSET in order to send different versions to 
different recipients.

3. timing

this proposal could not have come at a worse time.

putting content negotiation in the transport protocol is likely 
to result in a significant increase in anomolous behaviors
from the mail system which are difficult to diagnose and fix -
and the proposal is being introduced at a time when the mail system 
is already suffering from serious reliabliity problems (often due 
to ill-conceived measures for blocking spam).  at this time,  
*any* proposal for adding new functionality to SMTP should be viewed 
with strong reservations, unless it somehow offers an improvement
in reliability.  a proposal that significantly increases the complexity
of SMTP error handling seems unlikely to do that.

4. naming and applicability 

the protocol claims to be a general-purpose content-negotiation scheme
but actually has "fax" assumptions embedded very deeply. 

first of all, it assumes that there is a linear ordering relationship
for document suitability to the recipient.  while this might be arguable 
for facsimile images (even this is debatable given orthogonal attributes 
like resolution and color), it's certainly not the case for MIME
email in general.

5. recommendations

a.  I strongly recommend that you redesign this extension so that, instead
of overloading the RCPT command, you define a separate command to
request a recipient's capabilities.  e.g.

RCAP <recipient-addr>

would return the conneg capability associated with that recipient,
if available.  there would need to be separate indications
for permanent error (perhaps if the recipient addr were invalid),
temporary error (e.g. can't do the database lookup),
no capabliities available (perhaps not an error, just no information),
and a return of capabilities.

one nice side-benefit is that it doesn't change the error handling
model, because the proposal mentioned above doesn't affect 
the actual message handling commands at all.  (however it does
add a few round trip delays)

b. the idea that the sender should send the "best" or "highest
quality" version needs to be relaxed or generalized, since there's
currently no clear notion of what this means. (and thus no way to 
know whether an implementation conforms to these criteria).  

c. this extension needs to be made less fax-specific.
it needs to be examined with some realistic non-fax-specific
test cases, and it needs non-fax conneg examples also.

d. there needs to be a good way for the recipient to specify
capabilities to the SMTP server, at least for any SMTP server
that can serve arbitrary UA recipients.  probably this means another
extension along with SMTP AUTH.

still, this is a big can of worms, because there's never been
any ability of a UA to specify something to its inbound SMTP
servers before - there's presently not even a good way for
a UA to know which server(s?) should be given this information...

e. make it absolutely clear that such transformations are only to be
made with explicit authorization by the sender and/or recipient.

(a dedicated smtp-to-printer hardware device would be considered 
to have explicit authorization simply because it was purchased
for such a purpose, but a SMTP server acting on behalf of a user
using a general-purpose MUA would need such authorization) 

f. the document needs to consider the effect of message forwarding
on the returned string - if a recipient has his mail forwarded
elsewhere it's not clear what conneg string should be returned.

g. the document alludes to the ability to work through intermediaries
but this isn't explained anywhere, and it needs to be explained.

6. specific problems

> 4.2 Client Action:
>           
>      If the server issued a 250-CONNEG, as part of its
>      EHLO response for the current session, the client MAY
>      issue the CONNEG parameter with RCPT-TO.
>           
>      If the client issues the CONNEG parameter with
>      RCPT-TO, then it MUST honor the capabilities specified
>      in the CONNEG RCPT-TO reply, and transform data that is
>      sent, so that the target can accept the data. The
>      client SHOULD transform the data to the "highest" level
>      of capability of the target.

this incorrectly assumes that (a) there is only one target when
the client can easily specify multiple RCPT commands, and
(b) there is a clear notion of "highest level of capability"
for the target.

also why is the term "target" used rather than the more traditional
"recipient" or "mailbox"?

>      If the server rejects the RCPT-TO command with a 404 reply, the 
>      client may later reissue the RCPT-TO with the CONNEG parameter.

presumably in the context of a different SMTP session?
it's not clear...

> 4.3 Server Action:
>           
>      If the client specifies "CONNEG = REQUIRED" in the RCPT-TO, 
>      but the target does not support the CONNEG parameter, the target
>      MUST reject the RCPT-TO command with a 504 reply. 

here there is a potential conflict between various extensions.
a RCPT command may fail for any number of reasons.  if any of those
reasons is a permanent fatal error it should be sufficient for 
the server to report any one of those errors.  it's not reasonable
for this extension to pre-empt other error notificaitons.
(it may be that other extensions have similar wording, in which case
they're similarly flawed)

>      If the target support the CONNEG parameter, but it can not return 
>      the recepient's capability temporarily, the target MUST reject the 
>      RCPT-TO command with a 404 reply. For example, if the target get 
>      the capability information from a directory, but its connection is 
>      offline, the target MUST reject the RCPT-TO command with a 404 
>      reply. 

similarly: if any permanent failure is detected in the RCPT command, 
the appropriate 5xx response SHOULD probably be returned in preference
to any 4xx response.  for instance if the recipient's mailbox has
been administratively disabled then a 5xx response should be returned
rather than a 404 response which would mean "ask me later what kind
of media types  this recipient can accept"... whereupon the client
would do so only to (eventually) find out that it couldn't deliver 
the message to the recipient anyway.

>      If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the 
>      target MUST process the address and message as if the requested 
>      CONNEG capabilities had not been specified.

this doesn't seem to make sense as written.  is "CONNEG = OPTIONAL"
really supposed to be interpreted the same as omission of the 
CONNEG parameter?

also, I don't think there should be spaces around the = sign -
I don't think this is legal syntax for SMTP extensions.

>      Regardless of the value of the parameter, if the target does 
>      support the CONNEG parameter, then it MUST issue a 250 reply, 

again, there could be other reasons for permanent or temporary failure
of the RCPT command, and the assertion of a CONNEG parameter should 
not pre-empt the reporting of such failures.

>      followed by the capabilities of the target that is specified by 
>      the RCPT-TO address. Successful responses to CONNEG RCPT-TO 
>      requests will always be multiple SMTP lines.  The first line 
>      is the normal RCPT-TO response, and subsequent lines beginning 
>      with the exact string "250-CONNEG " and "250 CONNEG " are 
>      the CONNEG responses. The last line begins with "250 
>      CONNEG".  

I wonder here - assuming that this can of worms does not get closed
back up, and there might later be demand for other mechanisms
to request recipient capabilities (e.g. since this one is not very general),
might there be multiple kinds of strings emitted in the RCPT response?
e.g. CONNEG and CONNEG2 and FINAL_SOLUTION_CONNEG?
and if so should this document insist that CONNEG be last?

>      If the SMTP server supports ENHANCEDSTATUSCODES, the response 
>      strings for a success are "250-2.1.5 CONNEG" and "250 2.1.5 
>      CONNEG". The response strings for indicating a permanent 
>      failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
>      The response strings for a temporary failure are "404-4.3.3 
>      CONNEG" and "404 5.3.3 CONNEG".

are these codes already defined in 1893?  (I didn't look) 

I don't think it's appropriate to dictate extended status codes
since their entire purpose is to provide more precision than
the SMTP reply codes.  there are various sources of permanent
failure and various sources of temporary failure and the set
of status codes is extensible independently of SMTP reply codes.

>      All CONNEG-capable clients and CONNEG-capable servers MUST
>      be able to successfully process CONNEG lines that are up to 512
>      characters long, as required by RFC2821. If the length of CONNEG
>      lines is greater than 512 characters, the server MUST insert
>      line breaks and make next CONNEG line. 

CONNEG *what* lines?  command lines? response lines?  both?

>      The contents of the capability listing MUST conform to
>      [RFC2506], [RFC2533] and [RFC2534], and MAY be used to emulate
>      services, such as the facsimile start-of-session capabilities
>      negotiation as described in "Content Feature Schema for
>      Internet Fax". [RFC2879]
> 
> 
> 
> 
> 5.   SYNTAX
>      
>      Command with "CONNEG":
>       "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
>                    / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
>                  "OPTIOANL") CRLF

the space doesn't belong after CONNEG, and OPTIONAL is misspelled. 

> 
>      Reply:
>       ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
>         ("250 CONNEG" capability CRLF) )/
>       ( ("250-2.1.5" CRLF) 
>           *("250-2.1.5 CONNEG" capability CRLF)
>            ("250 2.1.5 CONNEG" capability CRLF) )/
>       ("504" CRLF) /
>       ("504 5.3.3" CRLF) /
>       ("404" CRLF) /
>       ("404 4.3.3" CRLF) /

it's not appropriate to put literal response codes in the grammar,
particularly the extended status codes which might actually need to
vary more than is indicated here.

much better to have something like:

reply: initial-reply *(conneg-reply/other-reply) final-reply

initial-reply: reply-code "-" [ status-code ] CRLF

conneg-reply: reply-code "-" [ status-code " " ] "CONNEG" capability CRLF

other-reply: reply-code "-" [ status-code ] *char CRLF

final-reply: reply-code " " [ status-code ] CRLF 

> 
>       capability = <<as per [RFC2506], [RFC2533], [RFC2534]
>                     and [RFC2879] >>
> 
> 
> 
> 6.   EXAMPLE
> 
>      An example of ESMTP sequence with successful RCPT-TO response
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 250-<[email-masked]> recipient ok
>      S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

this needs clarification - it looks like the sender is sending
multiple lines without a 250 or CONNEG on each line.
it would probably be better to break up this example into
several SMTP lines and also to explain that any of those lines 
can be longer (up to XXX bytes) than shown in the example.

>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per:
>           (  image-file-structure=TIFF-minimal
>              dpi=400
>              image-coding=JBIG
>              size-x=2150
>            )
>         >>

this is confusing - it makes it appear that TIFF-FX
is the only content-type acceptable.  

also TIFF-FX is not the name of the content-type,
presumably it's application/tiff-fx or some such.

>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
>      An example of successful RCPT-TO response when the length of 
>      capability is greater than 512 characters. 
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250-2.1.5 CONNEG (&(image-file-structure=TIFF-minimal) ...
>      S: 250-2.1.5 CONNEG   .....
>      S: 250 2.1.5 CONNEG   (color=Binary)
> 
> 
>      An example of succssful RCPT-TO response when CONNEG-capable 
>      server supports ENHANCEDSTATUSCODES.
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

again, this response is confusing because it's line wrapped

> 
> 
>      An example of ESMTP sequence with parmanent failure RCPT-TO 
>      response. 
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 504 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
>      An example of an ESMTP sequence with temporary failure RCPT-TO 
>      response when the value of parameter is "REQUIRED":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 404 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
>      .
>      .
>      .
>      retry according to implementation
> 
> 
> 
>      An example of an ESMTP sequence with temporary failure RCPT-TO 
>      response when the value of parameter is "OPTIONAL":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = OPTIONAL
>      
>      S: 250 <[email-masked]> recipient ok
>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per "A Simple Mode of Facsimile Using Internet Mail"
>           RFC2305
>         >>
>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
> 7.   IANA CONSIDERATIONS
>      
>      On publicatiom of this document by the RFC Editor, the IANA 
>      shall register the Content_Negotiation ESMTP extension defined 
>      in section 3.
> 
> 
> 8.   SECURITY CONSIDERATIONS
>      
>      This ESMTP option calls for a respondent to disclose
>      its capabilities.  Mechanisms for determining the
>      requestor's authenticated identity are outside the
>      scope of this specification.  It is intended that this
>      mechanism permit disclosure of public information;
>      hence there is no particular need for security
>      measures.
>      
>      However there is nothing to prevent disclosure of
>      sensitive information that should receive restricted
>      distribution.  It is, therefore, the responsibility of
>      the disclosing ESMTP server to determine whether
>      additional security measures should be applied to the
>      use of this ESMTP option.

seems like you should at least mention SMTP AUTH. 

>      A man-in-the-middle attack might change the capabilities reported 
>      for a given recipient. For example: Suppose the sender knows the 
>      recipient has the ability to view color documents so they mark 
>      some things in red in what is otherwise a black and white 
>      document. But someone interferes with the returned capabilities, 
>      indicating that the recipient only supports black and white. The 
>      document is duly downgraded, with the result that the recipient 
>      doesn't see what the sender marked. 

the STARTTLS command could be useful in providing integrity assurance.

>      An indirect exposure can occur when the report of a capability 
>      implies use of specific software.  If that software is known to 
>      have security weaknesses, the capabilities report effectively 
>      advertises the associated opportunity to exploit the security 
>      weakness.
>      
>      For target SMTP servers that require security mechanisms to be in 
>      force at the start of the session, the target SHOULD refrain from 
>      including the CONNEG parameter in an EHLO response until the 
>      requisite security mechanisms are in force.

To: [email-masked]
Subject: [fwd] proposed SMTP extension
Date: Tue, 02 Jul 2002 00:38:14 -0400
From: Keith Moore <[email-masked]>
Sender: [email-masked]

I noticed that this wasn't cc'ed to the ietf-smtp list, and it seems
like SMTP people who aren't in the fax WG might want to sniff at this.
(the idea of having SMTP act on the UA's behalf smells funny to me)

I'm already writing up my comments.

Keith

------- Forwarded Message

Message-Id: <[email-masked]>
To: IETF-Announce:;
Cc: [email-masked]
From: The IESG <[email-masked]>
SUBJECT: CORRECTION: Last Call: SMTP Service Extension for Content 
	   Negotiation to Proposed Standard
Reply-to: [email-masked]
Date: Mon, 01 Jul 2002 17:34:24 -0400
Sender: [email-masked]

[Please note the corrected four week last call date, due to the 
upcoming 54th IETF.  Sorry for any confusion.]

The IESG has received a request from the Internet Fax Working Group to 
consider SMTP Service Extension for Content Negotiation 
<draft-ietf-fax-esmtp-conneg-02.txt> as a Proposed Standard.  

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.  Please send any comments to the 
[email-masked] or [email-masked] mailing lists by July 29, 2002.

Files can be obtained via 
http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-02.txt

------- End of Forwarded Message