Subject: [EAI] ESMTP -- is bounce only (no downgrade) allowed

From: Kari Hurtta <[email-masked]>
Date: 11 Feb 2007 08:43:13 +0200

On 8BITMIME there is on operation mode where
        - SMTP server announces 8BITMIME
        - never parses or downgrades MIME structure

        Instead mail is always bounces if next hop
        does not support 8BITMIME

This is possible because on ESMTP there is parameter
BODY=8BITMIME

Is following wanted on UTF8SMTP
        - SMTP server announces UTF8SMTP
        - never parses or downgrades MIME structure
        - never downgrades mail header fields

        Instead mail is always bounces if next hop
        does not support UTF8SMTP

This requires that SMTP server knows that mail
is using  UTF8SMTP.  So if that opartion mode
is make posible, that requires that on ESMTP
there is parameter HDR=UTF8

Basically that is: Do we allow SMTP servers
which do not parse MIME structure ?

( SMTP servers which does downgrade do not need
  either HDR=UTF8 or BODY=8BITMIME.  They can
  just start downgrading if next hop does not
  support 8BITMIME or UTF8SMTP.  If there is 
  no anything that requires downgrading, output
  is original mail. That downgrade process is 
  also that pass when mail is parsed. No other
  parsing is required, if that output is stored
  to memory and after full succesfully downgrade 
  then is DATA commend given to next hop. )

/ Kari Hurtta

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 11 Feb 2007 16:41:16 +0100

>         does not support 8BITMIME

It better rejects it instead of "accept and bounce"
in this situation, otherwise it's in serious trouble.

> This is possible because on ESMTP there is parameter

Yes.  Are you using the confusing 2821-definition of
"bounce == reject" ?  Please don't, let's stick to
"bounce == send NDN", or don't use "bounce" at all.

>         does not support UTF8SMTP

The next hop could still support 8BITMIME, and for
that it's good enough to downgrade the mail header.

> there is parameter HDR=UTF8

With an UTF-8 address for the MAIL FROM that's not
necessary:  At some point in time it would cause
an UTF-8 Return-Path => message/utf-8.

With an UTF-8 address for the RCPT TO it could in
theory cause an UTF-8 "for" clause in the timestamp
line => message/utf-8 for these RCPT TOs, as far as
the relay uses "for".

So your scenario is apparently "no UTF-8 in the
envelope" (or only in some RCPT TOs), and then it
can determine "HDR=UTF8" by looking at the header.

Anything related to the body is already determined
by BODY=8BITMIME, it's only relevant for next hops
not supporting 8BITMIME.

> which do not parse MIME structure ?

Sure, not parsing MIME is fine as long as you stay
within an 8BITMIME environment.

>   SMTP servers which does downgrade do not need
>   support 8BITMIME or UTF8SMTP.

UTF8SMTP to 8BITMIME is a simple procedure.  But
8BITMIME to 7bit is tricky.  I think you have all
info you need (without HDR=UTF8).  We have to say
that "look at the header" is actually about the
header as created by an MDA (incl. Return-Path),
even if the MTA in question is no MDA, because
UTF-8 could be used _only_ in the envelope.

Frank

---

From: Kari Hurtta <[email-masked]>
Date: 11 Feb 2007 18:38:12 +0200

Frank Ellermann <[email-masked]> writes in gmane.ietf.ima.

> by BODY=8BITMIME, it's only relevant for next hops

No. It is not only relevent for next hops not supporting 
8BITMIME. 

So, I disagree with that answer.  It is already on many messages
discussed what all is required for UTF8SMTP downgrade. So
I will not repeat it on here.

/ Kari Hurtta

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 12 Feb 2007 02:22:36 +0100

> So, I disagree with that answer.  It is already on many
> messages discussed what all is required for UTF8SMTP 
> downgrade.

I propose to get a MIME expert (e.g., Keith, or Ned) to tell
us how 2049-MIME is supposed to work:  Is a message/utf-8 or
message/unknown part within a message/rfc822 an opaque object,
only subject to 8bit vs. 7bit considerations, or is its 
"internal" structure inherently known.

Likewise we need to know if message/utf-8 or message/unknown
parts within an (implicitly) message/utf-8 are opaque objects,
not subject to UTF8SMTP to 8BITMIME downgrading efforts.

Finally we should know whether introducing UTF-8 in MIME part
headers (Content-*) is somehow possible for MIME version 1.0,
or if that would require a new MIME version (as I think).

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 13 Feb 2007 21:42:15 -0000

> This is possible because on ESMTP there is parameter

Except that even where BODY=8BITMIME is absent, systems (e.g. sendmail)  
still do the full recursive descetn check, just to make sure.

> there is parameter HDR=UTF8

And there again, this WG seems to have decided that systems are expected  
to do the full recursive descent check regardless (that is why we got rid  
of Header-Type, which was a far better marker for the existence of  
UTF8SMTP in the message than HDR=UTF8 would be).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 13 Feb 2007 21:46:13 -0000

> UTF-8 could be used _only_ in the envelope.

No! It has been explained to you again and again that you need a full  
recursive-descent check in both cases, in order to find the places that  
need to be downgraded. The overall structure of the test is exactly the  
same in both cases (just that you are looking to downgrade bodies on one  
case, and to donwgrade headers (or maybe both) in the other.

---

From: Kari Hurtta <[email-masked]>
Date: 14 Feb 2007 10:35:55 +0200

"Charles Lindsey" <[email-masked]> writes in gmane.ietf.ima:

> Except that even where BODY=8BITMIME is absent, systems
> (e.g. sendmail)  still do the full recursive descetn check, just to
> make sure.
> 
> And there again, this WG seems to have decided that systems are
> expected  to do the full recursive descent check regardless (that is
> why we got rid  of Header-Type, which was a far better marker for the
> existence of  UTF8SMTP in the message than HDR=UTF8 would be).

On consensus call there was:

| Note that this is not a call on the proposal to have a similar marker
| in the SMTP protocol; that's a separate issue.

So I was analyzing on what situations HDR=UTF8  parameter is required.
I'm assuming that this parameter goes to MAIL command.

Seems that it is required, if WG want allow MTAs to announce UTF8SMTP
without full recursive mime structure parsers.  On that case these
MTAs which announce UTF8SMTP, and they do not have full recursive mime structure 
parser  then need bounce (*) message if next hop of any rcpt do not
announce  UTF8SMTP.

(*) Bounce means, either
    1) accept on SMTP level and send NDN later
or  2) reject RCPT TO on SMTP level
or  3) reject SMTP level after DATA

If there is not HDR=UTF8 paramater, but there is full recursive parser,
then possibilities are
    1) accept message, and downgrade it 
    2) accept on SMTP level and send NDN later
    3) reject SMTP level after DATA
(when next hop of any rcpt do not announce  UTF8SMTP.)

Without HDR=UTF8 paramater there is no possibility
    - reject RCPT TO on SMTP level

/ Kari Hurtta

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 15 Feb 2007 11:32:09 -0000

> I'm assuming that this parameter goes to MAIL command.

Oh! I had supposed it would go with the DATA command, since it gives  
information about the message that is to follow.
> without full recursive mime structure parsers.

Sure, but the WG seemed not to be wanting to do that. If you want to avoid  
a full recursive descent, and provide HDR=UTF8 in order to achieve that,  
then you might as well provide Header-Type as well. So in practice the  
decision not to allow Header-Type seems to force the other decision as  
well. But if the WG still wants to go that route, then I don't object -  
but OTOH I hear no clamour to do so.

The WG seems happy to accept recursive descent. I didn't like it (and  
still don't), but I now see that it is not as inefficient as I had thought  
it might be, though there are still uncertainties on how to handle message  
types