Subject: [Ltru] Re: [EAI] New draft for allowing UTF-8 in SMTP responses

From: Martin Duerst <[email-masked]>
Date: Fri, 18 Aug 2006 16:30:51 +0900

Hello Alexey,

I have looked at the draft. I'm not totally familiar with
all the details of email transmission, but will comment
mostly from an LTRU perspective. I have cc'ed the LTRU
WG, because I think your draft makes a good usage example
for the work of the LTRU WG, and some people may be able
to provide more feedback.

At 00:27 06/08/15, Alexey Melnikov wrote:

>       Title           : SMTP Language Extension
>       Author(s)       : M. Gahrns, A. Melnikov
This list of authors and the one in the draft seem to be
out of sync, probably a clerical error?

>       Filename        : draft-melnikov-smtp-lang-05.txt
>       Pages           : 0
This seems to be some administrative error, too, probably
a tool that counts drafts without explicit page breaks as
0 pages (I'd understand 1 page, 0 is really weird).

>       Date            : 2006-8-2

Section 3: "It also recommends that server recognizes languges that
   have multiple different tags (for example "ru" and "rus")."

   This is a bad idea. RFC 3066, and its successor RFC 3066bis
   (draft-ietf-ltru-registry-14.txt, approved by the IESG and in
    operation) both make it very clear that for languages that
   have both two-letter and three-letter codes, only the two-letter
   codes are used.

Please update the reference to RFC 3066 to RFC3066bis.

For language fallback, I suggest you have a look at
http://www.ietf.org/internet-drafts/draft-ietf-ltru-matching-15.txt
(also IESG-approved). This gives you the (basic) "language-range" ABNF
construct that includes the "*" wildcard.

Also, it seems that the matching going on on the server when the
client issues a LANG command (e.g. in Example 5) is very close to
and can (and should) be described in terms of Section 3.4, Lookup,
of the above draft. The only difference I see is that Section 3.4
requires a solution (maybe default) in all cases, whereas in your
case, the default if no matching language is found is not i-default,
but "no change" or in other words, "previously selected language".

I think the discussion of mul and und is overkill. There is no
interoperability problem if the client uses 'mul', it is just
a language that's not supported. Suggesting, as in Example 2,
that the server has specific code for that issue is a bad idea,
it will tend to make the server more complex than necessary.

Somewhere in example 1, probably in the comment that starts with
"< Once the client changes...", you should say that the example
here doesn't show accents because they cannot be represented
in this format, but that accented characters, encoded in UTF-8,
would be present in the actual protocol exchange.

Note 1: This is in part overkill, and in part incorrect.
If you do fallback on the server, then you have to leave
the decision to the server (or the server programmer or
administrator). There is no a-priori difference between
the special prefixes i- and x- and two-letter and three-
letter prefixes. It's very well possible that for i-foo-bar
and i-foo-baz, there is a mutually intellegible i-foo on
the server. The server will most probably only have i-foo
if the mutual intellegibility is a reasonal assumption.
On the other hand, for some people, zh-Hans (Chinese
written in simplifed script) and zh-Hant (Chinese written
in traditional script) are not mutually intellegible,
and so it may be a bad idea to make "zh" available on
the server. I think that if you refer to the abovementioned
two drafts, that should be enough.

Aside: You could get rid of server-side fallbacks if you
could assume that the server can always send the complete
list of available languages. However, I think this would
be a bad idea, because a) there may be dozenz or hundreds
of languages, and just sending the list may present somewhat
of a scalability problem, and b) it may be much easier
computationally for the server to check for the existence
of a requested language (or it's fallback) than to list
all languages available (which may require an extensive
search for files in a large number of directories).

For the LANG command, you only allow one language.
You could extend that to use a language priority list,
but this is not really necessary, the client can
simulate a language priority list by using successive
requests until one of them is successful.

On the other hand, for the LANG parameter for the MAIL
command should allow a language priority list. The reason
for this is that (if my understanding is correct), this
parameter is passed on along the relay chain of SMTP servers,
and is supposed to go back to the original sender, and
using a list increases the chance that there is something
at the relevant server that can be understood by the
originator.

I'm also not completely clear on / happy with the following
paragraph:

   If the message is relayed to another SMTP server that supports LANGUAGE
   ESMTP extension, the MTA acting as the client MUST check if the receiving
   MTA lists the language specified in lang-param ("requested language") in
   the list of supported language tags in LANGUAGE EHLO response.  If the
   receiving MTA either lists the requested language or doesn't list any
   language tag (i.e. the receiving MTA is unable to list languages it
   supports) the sender MUST issue LANG command for the requested language.
   After that, regardless of the result of LANG command, the client MTA MUST
   specify LANG parameter in MAIL command.

After repeated reading, I think this is okay, but some clarification
might help. In particular, it should be pointed out that trying to
use the LANG command is done so that potential error messages in
this relay step can be sent back to the original sender in the
appropriate language if possible.

Also, on a first reading, I was thinking that if the desired language
wasn't available, the LANG parameter would no longer be used.
I think the reason for this misunderstanding is that the last two
lines come very late. I think it would be easier to understand if
the paragraph started out as follows:

   If the message is relayed to another SMTP server that supports the
   LANGUAGE ESMTP extension, the MTA acting as the client MUST do
   the following two things, in the following order:
   1) Attempt to set the language of server responses to the requested
      language(s).
   2) Independently of whether 1) is successful, use the LANG
      parameter on the MAIL command to transmit the requested
      language(s) to the next relay.

If you think more explanations and details for 1) are needed, they
should come after this list.

Regards,     Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]     

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Tony Finch <[email-masked]>
Date: Mon, 21 Aug 2006 13:04:26 +0100

On Fri, 18 Aug 2006, Martin Duerst wrote:
>
> For the LANG command, you only allow one language. You could extend that
> to use a language priority list, but this is not really necessary, the
> client can simulate a language priority list by using successive

The client knows which languages that the server supports (unless the
server can't enumerate them), so it can simply do any priority matching
within itself then choose the best match with one LANG command. Perhaps
the draft should clarify this.

The idea of using multiple LANG commands to implement a priority list is
really bad for latency. If the LANG command could be pipelined then you
could just issue a load of them in order of increasing priority, so the
last one to succeed would be the best match. But this might not be very
efficient on some servers.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FISHER: WEST OR NORTHWEST 4 OR 5 BECOMING VARIABLE 3 OR 4. FAIR. MODERATE OR
GOOD.

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 21 Aug 2006 11:10:26 -0700

Hi -

as a technical contributor...

> From: "Tony Finch" <[email-masked]>
> To: "Martin Duerst" <[email-masked]>
> Cc: "Alexey Melnikov" <[email-masked]>; "LTRU Working Group" <[email-masked]>; <[email-masked]>
> Sent: Monday, August 21, 2006 5:04 AM
> Subject: [Ltru] Re: [EAI] New draft for allowing UTF-8 in SMTP responses
...
> server can't enumerate them),
...

Or (if I read the draft correctly) if their tags won't all fit on one line.

I also think the statement in the draft that "for the purpose of this
document it is safe to treat all languages, whose tags starts with
primary language described in ISO 639-1 and ISO 639-2 (i.e. all 2
or 3 letters primary languages) as hierarchical" is incorrect
for languages for which multiple scripts are in common use.

Randy

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 22 Aug 2006 13:36:25 +0100

>to provide more feedback.
Thank you for your feedback Martin.
Draft draft-melnikov-smtp-lang-05.txt is a revision of a draft that 
expired in 2001 (before LTRU was chartered) and I haven't been following 
LTRU.

>out of sync, probably a clerical error?
No. Mike Garhns written the original IMAP LANGUAGE extension, this 
document is based on it.

>0 pages (I'd understand 1 page, 0 is really weird).
I think you are right.

>   have both two-letter and three-letter codes, only the two-letter
Ok, I've removed the paragraph.

>Please update the reference to RFC 3066 to RFC3066bis.
Done.

>but "no change" or in other words, "previously selected language".
Ok, this change would require some editing work on my part. I've added 
to my todo list.

>a language that's not supported.
>
Ok, removed.

>Suggesting, as in Example 2,
>it will tend to make the server more complex than necessary._
>_
>
The server is actually [re-]using the same error code, but the error 
text might be different.
This is actually quite useful for debugging implementation errors. 
However SMTP implementations are not required to do that anyway.

>would be present in the actual protocol exchange.*
>*
>
Done.

>if the mutual intellegibility is a reasonable assumption.
Thanks. I removed the note.

>search for files in a large number of directories).*
>*
>
Agreed. The list of all available langues is not returned for the 
reasons you stated.

>requests until one of them is successful.*
>*
>
Actually, the LANG command already allows for priority list.

>at the relevant server that can be understood by the
Ok, this is a valid point. I will try to do this in a future revision.

>appropriate language if possible.
I've clarified this, thanks.

>wasn't available, the LANG parameter would no longer be used.*
>*
>
Yes, that was my original intent. But this would change, if the draft is 
changed to allow for priority list.

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Aug 2006 12:35:02 -0400

I've deferred comment on this draft to see what others think,
but now seems to be the time...

--On Monday, 21 August, 2006 11:10 -0700 Randy Presuhn
<[email-masked]> wrote:

>> Cc: "Alexey Melnikov" <[email-masked]>; "LTRU
>> Working Group" <[email-masked]>; <[email-masked]> Sent: Monday,
>> August 21, 2006 5:04 AM
>> The client knows which languages that the server supports
>> (unless the server can't enumerate them),
> ...
> 

that is ok, because the basic statement is false in the general
case -- in a relay situation, which the draft does not appear to
completely analyze, the originating client (which is the only
system that really knows the user's language preferences, if it
does) may have no practical way to know the server's language
capability.

> I also think the statement in the draft that "for the purpose
> of this document it is safe to treat all languages, whose tags
> starts with primary language described in ISO 639-1 and ISO
> 639-2 (i.e. all 2 or 3 letters primary languages) as
> hierarchical" is incorrect for languages for which multiple
> scripts are in common use.

Yes.

Another issue you didn't mention is that, in an environment in
which the language negotiated runs right to left, it not clear
where all the pieces go.   Presumably, if this specification
doesn't change the syntax specified in 2821 (the examples in
this document imply that, but don't state it), we have

        Reply-line = Reply-code [ SP text ] CRLF

But, if "text" can run right to left, do extended reply codes go
at the beginning of the string (i.e., to the right) or do they
imply a separate subfield... noting that either raises all of
the BIDI issues simply because the Reply-code and SP are
required to be expressed in Western digits and ASCII characters.

Even if we can make this model work, I suggest it is the wrong
model.  I further suggest that the difficulties with the small,
inconvenient, cases, such as those above, are symptomatic of why
it is the wrong model.  

The issue of making the text of SMTP responses
language-appropriate has been recognized since RFC 821 was
written.  As with many other Internet protocols, the most
effective model is to use a single form on the wire and then
perform translations (of character sets, terminal information,
formats, and, yes, languages) at the endpoints.   The "rely on
the codes" requirement of 821 was, and is, part of that concern.
When the codes proved inadequate in practice, we extended the
codes, which was entirely consistent with that general model.

If a user needs to look at an SMTP reply message directly, then
I suggest that either there is a debugging case in action or the
relevant MUA is broken.  Debugging cases are easier with more
uniformity and simplicity, not more options.  

So I suggest that:

	(1) This draft be dropped in favor of an effort to make
	SMTP reply text easier to translate to other languages
	or generate-able from the codes.
	
	(2) Any server implementer who is concerned about
	internationalization -- and I think all of us should be
	-- should be generating extended reply codes.  If one
	were to adopt a LANG extension, I would suggest that it
	would make extended codes mandatory anyway.
	
	(3) We should look at the text associated with extended
	reply codes to be sure that the portions of those lines
	that are not boilerplate explanations are easily
	identified and extracted by automatic processes.  If we
	discover cases where that goal is not met, we should fix
	them.

With the latter two conditions in place, I believe the right
(and fairly simple) way to get an SMTP reply tailored to local
language needs would become:

	(i) MUA that receives the reply, or MTA that is about to
	deliver the reply to such an MUA, extracts the extended
	reply code and non-boilerplace, message-dependent,
	material from the "text" and discards the rest.
	
	(ii) The extended reply code is used to generate
	response text in the appropriate language, substituting
	the non-boilerplate elements as needed.  

While this requires the efforts in (1) - (3) above, it does not
require SMTP changes.  And the recipient systems can do the
right thing as dictated by their environments (which they should
do even if the text is more friendly to their usual practices).

SMTP is already too complicated.  Let's avoid adding more
complexity unless it is strictly necessary.  This is not.

    john

---

From: Addison Phillips <[email-masked]>
Date: Tue, 22 Aug 2006 11:10:14 -0700

A few comments follow...

>>> (unless the server can't enumerate them),

Enumerating languages can be difficult. Most locale systems are quite 
extensive, with hundreds of potential values available. The server would 
have to attempt to load resources in each and enumerate whether the 
resources were extant in the locale or not in order to build an accurate 
list. In practice, few languages are actually available, but the code to 
figure it out is not available either.

> where all the pieces go.  

This is a common misperception. The logical order of the text is 
left-to-right until the [ SP text ] is reached. This text is rendered 
RTL, if appropriate to do so, up to the CRLF (which ends the RTL "run"). 
See http://www.w3.org/TR/CharMod and http://www.unicode.org/reports/tr9/

> inconvenient, cases, such as those above, are symptomatic of why

The examples given are not very convincing. However... I tend to agree 
that this is the wrong model. The design of SMTP and other protocols is 
such that user-agents can present "friendly" messages (including 
localized messages) to end users. Almost no one interacts with SMTP 
servers directly.
> 	or generate-able from the codes.

Allowing UTF-8 would still seem to be a Good Thing, since even 
boilerplate messages might need to include runtime data that is 
non-ASCII. Language negotiation is another issue.

> 	discover cases where that goal is not met, we should fix
If extended reply codes permit the server to emit its own internal error 
messages which are not entirely standardized (i.e. boilerplate), then 
the user-agent may have not choice other than to render the string on to 
  the user. This is unsatisfactory.

> 	material from the "text" and discards the rest.

This works best if the message dependent material were clearly separate 
and in a machine-friendly format. This:

   9106 Invalid user name; user=nobody, domain=example.org

Is easier to deal with than:

   "9106 Invalid user name 'nobody' for domain 'example.org'"

> complexity unless it is strictly necessary.  This is not.

+1

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Aug 2006 15:48:39 -0400

--On Tuesday, 22 August, 2006 11:10 -0700 Addison Phillips
<[email-masked]> wrote:

> Enumerating languages can be difficult. Most locale systems
> are quite extensive, with hundreds of potential values
> available. The server would have to attempt to load resources
> in each and enumerate whether the resources were extant in the
> locale or not in order to build an accurate list. In practice,
> few languages are actually available, but the code to figure
> it out is not available either.

That is consistent with what I had assumed, and reinforces the
impression that this proposal is not a good idea.
 
> This is a common misperception. The logical order of the text
> is left-to-right until the [ SP text ] is reached. This text
> is rendered RTL, if appropriate to do so, up to the CRLF
> (which ends the RTL "run"). See http://www.w3.org/TR/CharMod
> and http://www.unicode.org/reports/tr9/

Exactly (!).    If I said that badly, I apologize, but I read
821/ 2821 as requiring a three digit code and the SP (in
left-to-right order) followed by "text".  The extended status
code stuff is specific about the text, but wasn't written
anticipating internationalization and is not required by the
proposed specification anyway.   And, to the best of my
knowledge, there is no standard requiring the used of W3C
CharMod or Unicore TR9 in Internet mail replies.

I believe that could be fixed for this spec, and even that it
wouldn't be hard to do.  But it is, IMO, evidence that the spec
is not completely worked out even if it were a good idea.

>> model.  I further suggest that the difficulties with the
>> small, inconvenient, cases, such as those above, are
>> symptomatic of why it is the wrong model.  
> 
> The examples given are not very convincing. However... I tend
> to agree that this is the wrong model. The design of SMTP and
> other protocols is such that user-agents can present
> "friendly" messages (including localized messages) to end
> users. Almost no one interacts with SMTP servers directly.

Right

> boilerplate messages might need to include runtime data that
> is non-ASCII. Language negotiation is another issue.

Clearly, the EAI effort is going to need to deal, in some way,
with replies that contain non-ASCII addresses.  But, as you say,
that is a different issue from language negotiation.

> If extended reply codes permit the server to emit its own
> internal error messages which are not entirely standardized
> (i.e. boilerplate), then the user-agent may have not choice
> other than to render the string on to   the user. This is
> unsatisfactory.

I think that is what I said.   In the current situation, the
parts that are clearly non-boilerplate (going back to 821) are
situations in which the reply is expected to include an address
or part of one.  And I was suggesting only that those pieces
needed to be easily identified and parsed out.

> This works best if the message dependent material were clearly
> separate and in a machine-friendly format. This:
> 

Yes.  How much of this we can successfully change now, after two
or three decades of practice, is an open question.  But that is
one of the reasons for pushing toward the use of extended reply
codes (and formats) which, at the moment, the EAI work is
expected to require.

>> complexity unless it is strictly necessary.  This is not.

Thanks.

     john

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 23 Aug 2006 04:21:41 -0700

> uniformity and simplicity, not more options.  
just because I have some available, here are some examples of SMTP error 
codes...

| SMTP error in RCPT TO: 553 '<@>' <n5ial@n5ial> not matched: (ERR |
| Error in RCPT TO: 550 5.1.1 unknown or illegal alias: @ (MX) |
| Error in RCPT TO: 501 5.7.1 This system is not configured to rel |
| Connect(raw) failed: Connection refused |
| 5.1.1 (Unknown user): QMail; 550 <[email-masked]>... U |
| Error in RCPT TO: 550 Sorry, no mailbox here by that name. (#5.1 |
| Error in RCPT TO: 554 5.0.0 Mail to ace3 is not permitted (MX) |
| Error in RCPT TO: 550 5.1.1 <@>... email address lookup in domai |
| Error in RCPT TO: 553 5.3.0 <@>... Unknown user [email-masked] |
| 5.1.1 (Unknown user): QMail; 550 <[email-masked] |
| Error in RCPT TO: 550 5.7.1 Unable to relay for @ (MX) |
| Error in RCPT TO: 554 Mail for @ rejected for policy reasons. (M |
| RELAY problem: 550 5.7.1 <@>... Relaying denied (MX) |
| Error in RCPT TO: 553 sorry, that domain isn't in my list of all |
| Error in RCPT TO: 554 5.7.1 <@>... User unknown (MX) |

<@> is an abbreviation for "the original email address was here". These 
are truncated to 64 chars.

the important points are:

- a lot of servers return quite different errors with the same error code
- the servers try to return info that's useful for the sender, not for 
the relay agent

Query: Has anyone seen a production piece of software that tries to 
decode the extendedstatuscodes as text for display to the users?

Harald

---

From: Tony Finch <[email-masked]>
Date: Wed, 23 Aug 2006 13:35:53 +0100

On Wed, 23 Aug 2006, Harald Alvestrand wrote:
>
> Query: Has anyone seen a production piece of software that tries to decode the
> extendedstatuscodes as text for display to the users?

Yes, and the result is usually unhelpful in my experience. A significant
number of support queries that I have to deal with are solved by me
pulling an error message out of the logs that was hidden by some software
between my servers and the user. (Admittedly, this is more likely to
happen for us because we don't support enhanced status codes, but since
they carry so little extra information I doubt they would help.)

John says:
>
> As with many other Internet protocols, the most effective model is to
> use a single form on the wire and then perform translations (of
> character sets, terminal information, formats, and, yes, languages) at
> the endpoints.  The "rely on the codes" requirement of 821 was, and is,
> part of that concern.

That's not the way I read it. To me it says that the non-enhanced codes
are for the client state machine and the text is for a human; it does not
imply that a client should replace the server's text with its own idea of
what went wrong.

> If a user needs to look at an SMTP reply message directly, then I
> suggest that either there is a debugging case in action or the relevant
> MUA is broken.  Debugging cases are easier with more uniformity and
> simplicity, not more options.

If a user sees an SMTP response then something has gone wrong so it's a
debugging case by definition.

Addison says:
>
> Almost no one interacts with SMTP servers directly.

Of course they do, to the same extent that they interact with IMAP or POP
or HTTP servers - mediated by a user agent.

>From my point of view, and from the point of view of my colleagues on the
help desk, it would be much easier if MUAs did not try to impose their own
gloss on my servers' error messages (or do other stupid things like
truncate them). That way I can provide helpful text that explains what
went wrong, and/or a URL to help pages or a directory.

The current set of enhanced status codes is non-orthogonal and does not
cover the range of possibilities or provide enough detail within their
coverage. For example, there are codes for bad syntax and bad domain for
both sender and recipient addresses, but you can only specifically say the
local part is bad for destination addresses, and there are different OK
codes for senders and recipients. There's a code for "too many recipients"
but not "too few recipients" (e.g. RFC 4468 has a different gloss of 5.5.0
for this situation). In SMTP you can say that an address has changed (551)
but not with enhanced status codes. (We've had a number departments
changing name recently where it might have been useful to use the 551
feature, but since no-one implements it we just used an ad-hoc "try
newname.cam.ac.uk instead" error.) The range of codes for client policy
restrictions is inadequate: there's no way of expressing the distinction
between "relaying is not permitted", "you have been blacklisted", "you are
making too many concurrent connections", "you have been rate-limited",
"you have been greylisted", etc. There are no codes for saying that the
message content violates security restrictions, such as a virus infection
or a forbidden attachment type or a spam classification.

Even if you solve these problems, there is no provision for extending the
set of status codes except as a standards action - there is no IANA
registry. And in fact an IANA registry wouldn't solve the problem because
no existing software will have glosses for any new status codes you might
register, so the new codes would be useless.

The idea of a fixed set of status codes is fundamentally unable to cope
with evolving operational requirements.

Tony.

---

From: Kristin Hubner <[email-masked]>
Date: Wed, 23 Aug 2006 11:22:59 -0700

On Aug 23, 2006, at 4:21 AM, Harald Alvestrand wrote:

> just because I have some available, here are some examples of SMTP 
> error codes...
>
>
> <@> is an abbreviation for "the original email address was here". 
> These are truncated to 64 chars.
>

I'm not sure that this is the sort of example you had in mind, as this 
is at the MTA
level, not the MUA level...but we added at it the MTA level to 
partially make up for
the current dearth of such "localizing SMTP extendedstatuscode" support 
at the MUA level.

The Sun JES MS MTA has options that for DSNs generated by the MTA, 
sites can specify
their own choice (in their own choice of language and charset -- to 
correspond to
their choice of language and charset for the entire 
first/human-readable portion
of the DSN) an additional text string to be reported corresponding to 
each
extendedstatuscode.  That is, when the MTA is generating a DSN (based 
on, say, an
SMTP rejection from a remote SMTP server), the DSN can include an 
optional
site-specified "interpretation" of the extendedstatuscode in addition 
to the actual
SMTP rejection code and text. This will then be included in the first 
(human readable)
portion of the multipart/report.

That is, a site could set

5.1.1=<localized-string>
5.1.2=<another-localized-string>

etc., (actually, they can set a bunch of such options, one set for each 
language/charset
combination for which they wish such special support) and then when the 
MTA generates a
DSN reporting on encountering such an extended code, the MTA will 
include the site's localized
string, as well as the actual "original" text (if any) issued by the 
remote SMTP server in the
human-readable first portion of the DSN.

This option has been used by some sites, though probably not many.  As 
Harald Alvestrand
observed, since the current situation is that a single 
extendedstatuscode will be used
by different servers to report various (though "related" in being the 
same RFC 1893 category)
errors, the general "interpretation" you can make for a particular 
extendedstatuscode
currently tends towards the vague -- essentially a localized version of 
the category as
defined in RFC 1893.

Regards,

Kristin
[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Thu, 24 Aug 2006 07:50:58 +0900

>left-to-right order) followed by "text".

The protocol only defines logical order, i.e. on the wire, the
bytes for the three digit code are followed by (a SP and then)
the extended error code and then (a SP and then) the TEXT.
As Doug has said, how to display these is entirely up to the
client. For an Arabic or Hebrew client, something like
                              [(RTL) TEXT goes here] 5.1.1 550
is okay and may make sense.                                 

>The extended status

TR9 is an Unicode Standards Annex (UAX), which means it is
part of the Standard. In many ways, at the very moment you
say UTF-8, you include it. For the IETF,
http://www.rfc-editor.org/rfc/rfc3629.txt normatively references
Unicode 4.0, and this includes TR9 at
http://www.unicode.org/unicode/standard/versions/components-4.0.0.html
(and I'm sure that the actual book says the same).

>> users. Almost no one interacts with SMTP servers directly.

I think I agree to quite some extent with John and Addison, but
as most of us know and Harald has shown again, the situation
with SMTP error message consistency is much worse than with any
other, comparable protocol.

Also, while these error messages with their codes may not have
been intended to be show to users, they routinely turn up in
bounce messages, which go directly to the end user. Telling
the end user that s/he may have mistyped an address should be done
in the user's language, should not require support staff invention,
and should not be treated as a protocol debugging problem.

For EAI, I think it would be immensly helpful to have return
messages defined with enough precision so that an MUA
receiving a bounce can tell the user directly, or otherwise
take an appropriate action (e.g. flagging the relevant entry
in the address book as questionable) without necessarily having
to show the actual message to the user. If we don't manage to
get that for EAI, where we have a fresh slate, I don't think we
are in a position to try anything for the vast reminder of
error codes and error messages.

Regards,     Martin.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 24 Aug 2006 05:52:09 +0200

Martin Duerst wrote:

> say UTF-8, you include it.

You could take UTF-8 as "ASCII plus opaque garbage" in various
protocols, and then other parts of [Unicode] are not relevant.

Nobody's forced to use say Unicode collation or hyphenation
only because (s)he uses UTF-8 referencing STD 63,  Another
example would be 3066bis, it references ISO 3166.  Nobody
using language tags is now forced to use also 3166-3 codes,
they can still make up their own stuff, use CLDR or what else.

> with SMTP error message consistency is much worse than with
> any other, comparable protocol.

The RFC 4408 folks made sure that URIs work, indirectly that
arrives as whatever e.g. http offers.  A scenario for RFC 4408
is a user trying to send on a route not permitted by the sender
policy of his domain.

What really happens:  User submits mail to the "wrong" MSA, and 
this MSA doesnt't enforce submission rights (RFC 4409 6.1), it
tries to forward the mail towards the recipient.  The MX checks
the sender policy (RFC 4408) by querying a nameserver of the
domain in the Return-Path.

That policy says "FAIL" and offers an explanation in the form
of an URI (optional of course).  The MX "encodes" this as 5.7.1
error response (RFC 2034) - oops, the 2034-reference was still
correct in draft 00, odd, maybe the missing 2034 was my bad -
while talking with the "wrong" MSA (or its mailout).  Of course
the MX is not forced to add the explanation of this third party
to its 5.7.1, but it can, let's assume that it did.

The MSA had accepted the mail for forward or delivery, and that
didn't work, therefore it MUST (RFC 821/2821) generate a non-
delivery message (aka "bounce", not the definition in 2821).
It will probably copy the 5.7.1 error message verbatim into
its non-delivery message.  

It then puts the error message in the mailbox of the user of
this domain.  Or forwards it if the user arranged it this way.

Later the user finds the bounce, that quotes the 5.7.1 (maybe),
and the 5.7.1 quotes the FAIL-explanation (maybe), that is an
URI, and now the user can contact that URI (maybe), and if it
is HTTP depending on the server, the browser, LTRU, matching,
and some other details the user can read the explanation of
an admin in his domain in their preferred language, why this
mail didn't make it on the wrong route.

That's in a nutshell the reason why there are no explicit I18N
considerations in RFC 4408:  It would never do to tunnel some
non-ASCII text from DNS through a 5.7.1 and a bounce, besides
it would be pointless, the domain can have many users with many
different languages.

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 24 Aug 2006 15:07:55 +0100

> is okay and may make sense.

Actually, I don't see a problem. The "[(RTL) TEXT goes here] 5.1.1 550",  
if that is how it appears, should be fine for the human reader. Nothing  
breaks protocol-wise.

For an agent which needs to take some protocol action (e.g. whether to  
hold on to the message and try again later), all it needs to see is the  
"550", and for that all it has to do is to look as the first 3 characters  
as they come off the wire

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 27 Aug 2006 13:19:04 +0100

Tony Finch wrote:

>coverage.
Having recently reviewed Enhanced Status Codes and error messages 
emitted by Isode's SMTP server, I fully agree with this statement.

>For example, there are codes for bad syntax and bad domain for
>but not with enhanced status codes.
>
(slightly off topic)
Indeed. Some time ago I've tried to use 551 in order to implement Sieve 
redirect in LMTP server. I wanted to avoid adding SMTP client to Sieve 
engine, but after looking at Enhanced Status Codes I can only found:
      X.1.6   Destination mailbox has moved, No forwarding address
while I was expecting a status code saying "mailbox moved, send your 
message here <address>".

> (We've had a number departments
>newname.cam.ac.uk instead" error.)
>
Speaking of 551 and response boilerplates. Is there any interest in 
standardizing "mailbox moved, send your message here <address>" Enhanced 
Status Code with explicit boilerplate, so that the forwarding address is 
easily extractable?

> The range of codes for client policy restrictions is inadequate: there's no way of expressing the distinction
Very true. Software is unlikely to be written to automatically download 
updates to an IANA registry. There are many cases when this is not 
practical or explicitly prohibited by IT department.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 27 Aug 2006 13:19:35 +0100

>that is a different issue from language negotiation.
My draft is trying to address the following issues. Let's try to discuss 
them separately whenever possible, I will split the draft into multiple 
(or even drop some of them) based on feedback:
1). Allowing for UTF-8 in response text.
This is required for both UTF-8 email addresses and localized human 
readable text

2). Language negotiation.
If we agree that localized human readable text is a good idea, then 
there is a need for the sender to select a language. If the sender is 
Russian and the server emits responses in Chinese (or vice versa) by 
default, the response text would be of no much use for the sender. And 
as many others said, users end up seeing such text in non-delivery 
reports or in MUAs on message submission.

3). Allow for localized text in DSN messages. There are multiple ways of 
doing this:
a). localize the first text/plain part
b). extend format for message/delivery-status to allow for localized 
reason string, in addition to non-delivery reason in English.
c). extend multipart/report to allow for extra bodyparts, that would 
convey this information. (I am not sure how well this will interoperate 
in real world, but I am listing this choice here for completeness)

4). Allow for UTF-8 addresses in DSN messages. There are also several 
ways of achieving this:
a). Use ACE in message/delivery-status bodypart.
b). Extend message/delivery-status to allow for 8bit data. RFC 3464 seem 
to disallow this:

2.1 The message/delivery-status content-type

   The message/delivery-status content-type is defined as follows:

   MIME type name:             message
   MIME subtype name:          delivery-status
   Optional parameters:        none
   Encoding considerations:    "7bit" encoding is sufficient and
                               MUST be used to maintain readability
                               when viewed by non-MIME mail readers.

And RFC 2046 seem to advise against non-7bit encodings in message/*:

5.2.4. Other Message Subtypes

   MIME implementations must in general treat unrecognized subtypes of
   "message" as being equivalent to "application/octet-stream".

   Future subtypes of "message" intended for use with email should be
   restricted to "7bit" encoding. A type other than "message" should be
   used if restriction to "7bit" is not possible.

Alexey

P.S. However I still thing that encouraging servers to implement 
Enhanced Status Codes is a good idea.
I would also like to work on response boilerplates.