Subject: Re: [EAI] other thought about <eai@address <ascii@address>>

From: John C Klensin <[email-masked]>
Date: Thu, 01 Feb 2007 07:14:51 -0500

--On Thursday, February 01, 2007 11:41 AM +0900 Soobok Lee 
<[email-masked]> wrote:

>> Within an MUA, that certainly works.  Not to my taste, but UI
>> behavior is, after all, a matter of taste.
>
> I made manually a message file which contains unquoted
> eai@address as below.
> You can save the next rfc2822 message into a mbox file
> "/tmp/mbx1"   and open it using "mutt -f /tmp/mbx1" (or
> elm,pine etc) and then  try to make reply-all.  If your MUAs
> is "robust",
> you will see quotations marks are added around CC's name_parts
> below  in the reply form.

As with many other issues with email design, especially in MUAs, 
many violations of he specifications can be "robustness" to one 
person and "bugs" to another.  The bottom line on this, I 
believe, is:

	(1) We should not specify syntax that varies from the
	base 2821/2822 model unless that is specifically
	necessary to our task.  Permitting (i.e., requiring that
	systems accept) display names, without quotes, that
	contain special characters is such a change.  We not
	only should not make it, we should not even be
	considering it.
	
	(2) This is symptomatic of the many reasons why the IETF
	very rarely tries to specify, or even make strong
	recommendations about, MUA behavior. When we do,
	typically we either tie ourselves in knots or people
	ignore us.  The specifications we are working on are
	about the formats of messages as they are transmitted
	across the network.  The behavior you describe
	essentially demonstrates that, while these systems are
	permissive at the user interface, they fix things up and
	put the right things on the wire.  So, even if we
	somehow "like" the unquoted form, we should still be
	specifying it with quotes and without exceptions.

Gentlemen, with the understanding that this is a personal 
reaction and request only...

This WG has a very hard, and very important, task.  We are 
horribly behind our original schedule: had we managed to keep to 
it, all of the protocol documents would be in Last Call by now. 
While I know your intentions are good, distracting everyone with 
many messages about either what are essentially UI issues or 
that seem to reopen and re-hash issues already settled (some of 
them settled by the time RFC 821 was published) do not move us 
forward and, at least for those participants who tend to tune 
out when they see a large series of irrelevant messages, may 
actually constitute an impediment.

Let's try to focus on the core issues and get the work done. 
Please.

      john

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Soobok Lee <[email-masked]>
Date: Thu, 1 Feb 2007 22:22:03 +0900

> 	only should not make it, we should not even be

(This is the closing remark  of this thread)

I agree with you  and have been so. 
I don't support permitting  "eai@address <ascii@address>" here, 
rather the opposite !

What I tried to point out with my examples is that
 "eai@address <ascii@address>" are silently permitted now, while
 eai@address [ascii@address] will be rejected, by most
 existing non-EAI-capable parsers.

That explains why I favor [ascii@address] over <ascii@address>.
 
Permitting  "eai@address [ascii@address]" in [utf8headers] 
has no problem, but permitting "eai@address <ascii@address>"
make conflicts.

This will clear my position for [ascii@address].

If you and others have other opinions, please make new subjects and
thread about this.

> Let's try to focus on the core issues and get the work done. 

Sure!

Soobok

---

From: Kari Hurtta <[email-masked]>
Date: 01 Feb 2007 20:34:09 +0200

> If you and others have other opinions, please make new subjects and

With [ ] there is own problems.  I have stated them already.

/ Kari Hurtta

---

From: Soobok Lee <[email-masked]>
Date: Fri, 2 Feb 2007 10:16:04 +0900

> With [ ] there is own problems.  I have stated them already.

Your examples against [] around domain literals were

  From:  <NON-ASCII-LOCAL@[something]>
  From:  <NON-ASCII-LOCAL@[something][support@[something]]>
  From:  <NON-ASCII-LOCAL@[something][[email-masked]]>
  From:  <NON-ASCII-LOCAL@[something]>

Domain literals are there mainly for specifying IP address.
So, "[something]" above should be replaced with [128.124.1.1] etc.

  From:  <NON-ASCII-LOCAL@[128.124.1.1]>
  From:  <NON-ASCII-LOCAL@[128.124.1.1][support@[128.124.1.1]]>
  From:  <NON-ASCII-LOCAL@[128.124.1.1][[email-masked]]>
  From:  <NON-ASCII-LOCAL@[128.124.1.1]>

  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1][support@[128.124.1.1]>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1][[email-masked]]>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]>

I can't find any visual confusibility here. No danger at least to me.

  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]<support@[128.124.1.1>>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]<[email-masked]>>
  From:  <NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1]>

I replaced [support*] with <support*>. I can't find any difference.
Do you think the latter is safer than the former one in visual confusibility?

And, there may be increased parser implementations complexity around
using [ascii@address], as you pointed out.
But, that is a minor cost that implementors cannot bu pay, I think.

With [ascii@address], we have a "gain", since we can permit this convenient form,
  From:  NON-ASCII-LOCAL@[IPv6:2001:db8:100:f101::1] [[email-masked]]
  From:  [email-masked]  [[email-masked]]

With [ascii@address], we need <> surrounding them.
That is another cost.

Soobok

<quote from rfc2822>
Resnick                     Standards Track                    [Page 16]

RFC 2822                Internet Message Format               April 2001

   characters), then the dot-atom form SHOULD be used and the
   quoted-string form SHOULD NOT be used. Comments and folding white
   space SHOULD NOT be used around the "@" in the addr-spec.

addr-spec       =       local-part "@" domain

local-part      =       dot-atom / quoted-string / obs-local-part

domain          =       dot-atom / domain-literal / obs-domain

domain-literal  =       [CFWS] "[" *([FWS] dcontent) [FWS] "]" [CFWS]

dcontent        =       dtext / quoted-pair

dtext           =       NO-WS-CTL /     ; Non white space controls

                        %d33-90 /       ; The rest of the US-ASCII
                        %d94-126        ;  characters not including "[",
                                        ;  "]", or "\"

   The domain portion identifies the point to which the mail is
   delivered. In the dot-atom form, this is interpreted as an Internet
   domain name (either a host name or a mail exchanger name) as
   described in [STD3, STD13, STD14].  In the domain-literal form, the
   domain is interpreted as the literal Internet address of the    <==========
   particular host.  In both cases, how addressing is used and how
   messages are transported to a particular host is covered in the mail
   transport document [RFC2821].  These mechanisms are outside of the
   scope of this document.

</quote>

<quote from rfc2821>

Klensin                     Standards Track                    [Page 36]

RFC 2821             Simple Mail Transfer Protocol            April 2001

      esmtp-param     = esmtp-keyword ["=" esmtp-value]
      esmtp-keyword   = (ALPHA / DIGIT) *(ALPHA / DIGIT / "-")
      esmtp-value     = 1*(%d33-60 / %d62-127)
            ; any CHAR excluding "=", SP, and control characters
      Keyword  = Ldh-str
      Argument = Atom
      Domain = (sub-domain 1*("." sub-domain)) / address-literal
      sub-domain = Let-dig [Ldh-str]

      address-literal = "[" IPv4-address-literal /
                            IPv6-address-literal /
                            General-address-literal "]"
            ; See section 4.1.3

      Mailbox = Local-part "@" Domain

4.1.3 Address Literals

   Sometimes a host is not known to the domain name system and
   communication (and, in particular, communication to report and repair
   the error) is blocked.  To bypass this barrier a special literal form
   of the address is allowed as an alternative to a domain name.  For
   IPv4 addresses, this form uses four small decimal integers separated
   by dots and enclosed by brackets such as [123.255.37.2], which
   indicates an (IPv4) Internet Address in sequence-of-octets form.  For
   IPv6 and other forms of addressing that might eventually be
   standardized, the form consists of a standardized "tag" that
   identifies the address syntax, a colon, and the address itself, in a
   format specified as part of the IPv6 standards [17].

   Specifically:

      IPv4-address-literal = Snum 3("." Snum)
      IPv6-address-literal = "IPv6:" IPv6-addr
      General-address-literal = Standardized-tag ":" 1*dcontent
      Standardized-tag = Ldh-str
            ; MUST be specified in a standards-track RFC
            ; and registered with IANA

      Snum = 1*3DIGIT  ; representing a decimal integer
            ; value in the range 0 through 255
      Let-dig = ALPHA / DIGIT
      Ldh-str = *( ALPHA / DIGIT / "-" ) Let-dig

      IPv6-addr = IPv6-full / IPv6-comp / IPv6v4-full / IPv6v4-comp
      IPv6-hex  = 1*4HEXDIG
      IPv6-full = IPv6-hex 7(":" IPv6-hex)
      IPv6-comp = [IPv6-hex *5(":" IPv6-hex)] "::" [IPv6-hex *5(":"
                 IPv6-hex)]
            ; The "::" represents at least 2 16-bit groups of zeros
            ; No more than 6 groups in addition to the "::" may be
            ; present
      IPv6v4-full = IPv6-hex 5(":" IPv6-hex) ":" IPv4-address-literal
      IPv6v4-comp = [IPv6-hex *3(":" IPv6-hex)] "::"

Klensin                     Standards Track                    [Page 38]

RFC 2821             Simple Mail Transfer Protocol            April 2001

                   [IPv6-hex *3(":" IPv6-hex) ":"] IPv4-address-literal
            ; The "::" represents at least 2 16-bit groups of zeros
            ; No more than 4 groups in addition to the "::" and
            ; IPv4-address-literal may be present

</quote>

---

From: Soobok Lee <[email-masked]>
Date: Fri, 2 Feb 2007 10:28:36 +0900

> With [ascii@address], we need <> surrounding them.

==> Errata:  With <ascii@address>, we need <> surrounding them.

---

From: Kari Hurtta <[email-masked]>
Date: 02 Feb 2007 20:15:42 +0200

> So, "[something]" above should be replaced with [128.124.1.1] etc.

But not according of grammar. Grammar says that inside of [ ] can be 
almost anything.  On grammar [ ] acts as " "

That is a problem.

-------------------------------------------------------------------------
From: Kari Hurtta <[email-masked]>
Subject: Re: [EAI] draft-ietf-eai-utf8headers-02 : alt-address  again
Newsgroups: gmane.ietf.ima
Date: 02 Nov 2006 06:59:39 +0200

Frank Ellermann <[email-masked]> writes in gmane.ietf.ima:

> Kari Hurtta wrote:
>  
> >    From: KÃ¤yttÃ¤jÃ¤tuki <KÃ¤yttÃ¤jÃ¤tuki@[something]>
> >    From: KÃ¤yttÃ¤jÃ¤tuki <KÃ¤yttÃ¤jÃ¤tuki@[something][support@[something]]>
> >    From: KÃ¤yttÃ¤jÃ¤tuki <KÃ¤yttÃ¤jÃ¤tuki@[something][[email-masked]]>
> >    From: KÃ¤yttÃ¤jÃ¤tuki <support@[something]>
>  
> > How easy it is to parse ?    This is un-ambiguous, but still quite 
> > dangerous.
> 
> Easy, I don't see the danger here, <dot-atom> can't be empty.  
> The [something] is always a <domain-literal>.

Yes. It is un-ambiguous. I was more considered how tokenizer can be used.

   Käyttäjätuki <Käyttäjätuki@[something][support@[something]]>

Maybe someone want tokekize as

        Käyttäjätuki
        WSP
        <
        Käyttäjätuki
        @
        [something]
        [
        support
        @
        [something]
        ]

On there was problem   [   versus [something]   selection/conflict.

That is easy if top-down parser is used, but not otherwise.

/ Kari Hurtta

---

From: Soobok Lee <[email-masked]>
Date: Sat, 3 Feb 2007 16:05:29 +0900

> almost anything.  On grammar [ ] acts as " "

Are there any other domain-literal usages/definitions beside ipv4 and ipv6?

Soobok

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 03 Feb 2007 15:55:39 +0100

> Are there any other domain-literal usages/definitions beside ipv4 and ipv6?

IPvFuture in RFC 3986, General-address-literal in RFC 2821, 
and speculations about other literals in discussions about 
NetNews Message-IDs.

Frank

---

From: John C Klensin <[email-masked]>
Date: Sat, 03 Feb 2007 12:29:08 -0500

--On Saturday, 03 February, 2007 16:05 +0900 Soobok Lee
<[email-masked]> wrote:

>> But not according of grammar. Grammar says that inside of [ ]
>> can be  almost anything.  On grammar [ ] acts as " "
>> 
>> That is a problem.
> 

No.  But, when 2821 was written, an explicit decision was made
to provide for other cases in the event that they came along.
That was precisely why the syntax rule was either 
    [ <some digits> ]
was an IPv4 address (with some possibilities for syntax errors)
and 
    [keyword: ...] was some other sort of address literal.
There were proposals to handle IPv6 is some other way, such as
requiring that the parser recognize ":" rather than "." in the
string, but the conclusion was that we should allow for explicit
labeling rather than either "no plan if something succeeded
IPv6" or heuristics on the contents of the string.

If one encountered
    local-part@[some-string]
and the first character of that string was not a digit, a
correct interpretation of 2821 almost certainly requires that
one parse the string looking for an address literal type
keyword.  If it can't be found, or isn't "IPv6:" (or something
else considered valid at the time), then it is an error.

But, while my personal preference would be to just accept 
  <utf8-address@utf8-domain <alt-addr@alt-domain>>
and get on with it --not because I especially like it but
because it appears to be the rough consensus and "get on with
it" is becoming important -- I don't see what this has to with
the discussion.  

In 2821 and I think 2822, address literals are permitted only
where a domain can appear, and domains can appear only after
"@".  So
   local-path@[address-literal]
and
   <@[address-literal]:local-part@domain>
are well-defined instance of address literals.  But in all of
   local-part@domain [string]
and
   <local-part@domain> [string]
and
   <local-part@domain [string]>
"[string]" cannot, contextually, be an address literal.  Barring
extensions, it is a plain, ordinary, syntax error.  The only
issue it raises with address literals is that I assume that some
MUAs would report the error as "address literal out of context"
or "bad address-- no local part" rather than "where did that '['
came from".  And that might be very confusion to users if was
encountered in a non-upgraded but excessively syntax-permissive
implementation (one that didn't notice the UTF-8 characters in
the real local part until it had reported syntax errors).

    john

---

From: Soobok Lee <[email-masked]>
Date: Sun, 4 Feb 2007 09:57:24 +0900

> it" is becoming important -- I don't see what this has to with

What I try to point out is that, 
  o  domain-literals contain  IP address sequences, and it won't 
        affect the confusibility of 4) and 5)
  o  Comparing 4) with 5), 4) seems safer than 5), at least to me.
  o  we can permit 2) and 3) as valid ones if we choose [].
        This is a outstanding benefit and beyond our own personal preferrences.
  o  2) seems  the natural and smooth extension  of 1)
  o  If we choose <>, we can't permit  6) 

1) To: [email-masked], [email-masked] 

2) To: [email-masked] [[email-masked]], LCL2@@address.tld [[email-masked]]

3) To: LCL@[128.1.2.3] [[email-masked]], LCL2@@[128.1.2.3] [[email-masked]]

4) To: <LCL@[128.1.2.3] [[email-masked]]>, <LCL2@@[128.1.2.3] [[email-masked]]>

5) To: <LCL@[128.1.2.3] <[email-masked]>>, <LCL2@@[128.1.2.3] <[email-masked]>>

6) To: LCL@[128.1.2.3] <[email-masked]>, LCL2@@[128.1.2.3] <[email-masked]>

7) To: <[email-masked] [[email-masked]]>, <LCL2@@address.tld [[email-masked]]>

6)  will be parsed into 
   DISPLAY_NAME <[email-masked]>, DISPLAY_NAME2 <[email-masked]>
   by most existing robost email addres parsers.

> "[string]" cannot, contextually, be an address literal.

Sure. [string] is always full ascii email address. pleass read above
for my intention.

Soobok

---

From: John C Klensin <[email-masked]>
Date: Sun, 04 Feb 2007 11:42:19 -0500

--On Sunday, 04 February, 2007 09:57 +0900 Soobok Lee
<[email-masked]> wrote:

>   o  domain-literals contain  IP address sequences, and it
> won't          affect the confusibility of 4) and 5)
>   o  Comparing 4) with 5), 4) seems safer than 5), at least to
> me.   o  we can permit 2) and 3) as valid ones if we choose [].
>         This is a outstanding benefit and beyond our own
> personal preferrences.   o  2) seems  the natural and smooth
> extension  of 1)   o  If we choose <>, we can't permit  6) 
> 
> 

I assume that the third "@" in the second line, here and below,
is a typographical error: certainly I can't figure out what else
it might be.

>    DISPLAY_NAME <[email-masked]>, DISPLAY_NAME2
> <[email-masked]>    by most existing robost email addres
> parsers.

Please also note, first, that you have no idea what a valid
domain literal looks like once one passes beyond IPv6, other
than it must start with an ASCII string that conforms to LDH
rules followed by a colon.  In particular, something like
[HIP:semi-random-identifier] might turn out to be valid
eventually: there is no requirement that the string consist of
decimal or hex digits.

Second, please note forms 4a and b and 5a and b:

4a) To: <LCL@[128.1.2.3] [ascii@[128.1.2.3]>,
   <LCL2@[128.1.2.3] [ascii2@[128.1.2.3]]>

4b) To: <[email-masked] [ascii@[128.1.2.3]>,
   <LCL2@domain-name [ascii2@[128.1.2.3]]>

5a) To: <LCL@[128.1.2.3] <ascii@[128.1.2.3]>>,
    <LCL2@[128.1.2.3] <ascii2@[128.1.2.3]>>

5b) To: <LCL@domain-name <ascii@[128.1.2.3]>>,
    <LCL2@domain-name <ascii2@[128.1.2.3]>>

I am unconvinced that there is any significant difference
between the two forms.  I can construct really weak arguments
for preferring either, but few if any strong ones.

     john

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Feb 2007 00:17:13 -0800

At 7:37 PM -0800 1/29/07, Harald Tveit Alvestrand wrote:

>>  13) To:  eai@address <ascii@address>       <-----
>>  14) To:  eai@address [ascii@address]
>>  15) To:  eai@address
>>  16) To:  <eai@address>
>
>  I'm happy with not allowing cases 13 and 15 in the UTF8SMTP ABNF.
>
>  Let the nonconformant UTF8SMTP UA implementations fail as they may.

I agree.  I don't see the use in perpetuating a potentially ambiguous 
syntax given that only upgraded software will be capable of 
generating/parsing UTF8 addresses anyway.  Why not take the 
opportunity to require the angle brackets?

(It might be useful to keep in mind that originally, even the "@" was 
not required, and " at " could be used instead.  Tightening the 
syntax can be a good thing.)
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
It takes a wonderful brain and exquisite senses to produce a few
stupid ideas.                                        --Santayana

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Feb 2007 00:17:13 -0800

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Feb 2007 00:17:07 -0800

At 6:45 PM -0500 1/30/07, John C Klensin wrote:

>  Sigh.  Quotation marks have never been required, or even
>  recommended in that case.  For your amusement, the exact rule is
>  the reason why I stopped using a period after the "C" in "John C
>  Klensin", which doesn't use quotation marks either.  I suggest
>  that you might find reading the specification useful.

We tried very hard in DRUMS to permit unquoted dots in phrases! 
After all, no one likes to see his or her name in sneer quotes.  Too 
bad we couldn't find a way to do so.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Get your facts first, and then you can distort them as much as you please.
--Mark Twain

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Feb 2007 00:59:23 -0800

>  Speak as the editor of utf8header, my intension is not to allow 13 
> and 15. As discussed in previous posts, I think it makes sense that 
> 15 is allowed as we use ascii@address right now.

Sorry, for being confused.  Do you want to allow or disallow 
unbracketed UTF8 addresses?

I say let's ban them.
-------------- Randomly-selected tag: ---------------
The curse of all Art is that the devotee or disciple is always more
certain than the Priest.                          --Rudyard Kipling

---

From: John C Klensin <[email-masked]>
Date: Thu, 08 Feb 2007 05:44:09 -0500

--On Thursday, 08 February, 2007 00:59 -0800 Randall Gellens
<[email-masked]> wrote:

>>  Speak as the editor of utf8header, my intension is not to
>>  allow 13  and 15. As discussed in previous posts, I think it
>> makes sense that  15 is allowed as we use ascii@address right
>> now.
> 

While I would _strongly_ prefer to permit them as long as there
is no display name field, I think permitting them would force us
to deal with a whole series of other issues about characters
that may or may not be the same as others and what they would do
to parsers.  Since that long list of cases has completely
confused me, and probably some others, about some of what we are
talking about, what I am recommending/ would prefer is:

	(a) Any Mailbox or uMailbox (in the strict definitions
	of 2821 and eai-smtpext, respectively) that contains any
	non-ASCII character(s) MUST appear in brackets.
	
	(b) Any display-name (in the strict definition of 2822)
	that contains any non-ASCII character(s) MUST have the
	entire phrase (2822 definition) in quotes and follow
	2822's rules about quoted phrases, including the rules
	for embedded quotes.

Again, I don't really like this, but I the alternatives seem to
lead to bad cases and perhaps even some attack vectors.  Tedious
explanations and examples on request, but anyone who asks had
better be ready to understand Unicode normalization and the
different places at which it can be applied.

And, before we go down that rathole again, the above definitions
apply to the forms of headers the go on the wire.  What the user
interface of an MUA decides to permit on input or display is
Someone Else's Problem, as long as they get the "wire" form
correct as specified above.

    john

---

From: Soobok Lee <[email-masked]>
Date: Thu, 8 Feb 2007 21:41:22 +0900

> 	non-ASCII character(s) MUST appear in brackets.

Here is a peculiar email address example worth to consider 
 and leave in this list:

   To:  <[email-masked]>,<[email-masked]>

      The string between first "<" and "@evil.tld>" is 
         the single valid local-part if:

        First  @  is  U+FE6B or U+FF20, not U+0040
        Second @  is  U+0040
        First  ,  is  U+FE50 or U+FF0C, not U+002C
        First  <  is  U+003E
        First  >  is  U+FF1E or U+FE65, not U+003E
        Second <  is  U+FF1C or U+FE64, not U+003C
        Second >  is  U+003E

        (compatability characters homographs)
        http://www.unicode.org/charts/PDF/UFE50.pdf
        http://www.unicode.org/charts/PDF/UFF00.pdf
   
   To:  <[email-masked]> 

       (after replacing all compatability chars with hyphens)

   In EAI, utf8-local-part's utf8-dot-atom (and quoted-string)
	 has NO restriction on non-ascii characters at all.
 
   Should we prohibit all these compatability characters of 
         ASCII punctuations  ,:;[]<>()@  
         in utf8-local-part ?

<quote from utf8headers>

   utf8-addr-spec =  utf8-local-part "@" utf8-domain 
   
   utf8-local-part=  utf8-dot-atom / quoted-string / obs-local-part

   utf8-dot-atom = [CFWS] utf8-dot-atom-text [CFWS]

   utf8-dot-atom-text = 1*utf8-atext *("." 1*utf8-atext)

   utf8-atom     = [CFWS] 1*utf8-atext [CFWS]

   utf8-atext   =  ALPHA / DIGIT /
                   "!" / "#" /     ; Any character except
                   "$" / "%" /     ; controls, SP, and specials.
                   "&" / "'" /     ; Used for atoms
                   "*" / "+" /
                   "-" / "/" /
                   "=" / "?" /
                   "^" / "_" /
                   "`" / "{" /
                   "|" / "}" /
                   "~" /
                   UTF8-xtra-char

</quote>

Soobok

---

From: Soobok Lee <[email-masked]>
Date: Thu, 8 Feb 2007 21:57:06 +0900

>         First  ,  is  U+FE50 or U+FF0C, not U+002C

[correct] First  <  is  U+003C
[add]     First  .  is  U+FF0E or U+FE52, not U+002E

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 08 Feb 2007 16:23:34 +0100

>    To:  <[email-masked]>,<[email-masked]>

All combinations from 
<abc&#xFE6B;address.tld&#xFE65;&#xFE50;&#xFE64;[email-masked]> to
<abc&#xFF20;address.tld&#xFF1E;&#xFF0C;&#xFF1C;[email-masked]> with
<http://purl.net/net/ucode/FE6B>, <http://purl.net/net/ucode/FF20>,
<http://purl.net/net/ucode/FE65>, <http://purl.net/net/ucode/FF1E>,
<http://purl.net/net/ucode/FE50>, <http://purl.net/net/ucode/FF0C>,
<http://purl.net/net/ucode/FE64>, <http://purl.net/net/ucode/FF1C>.

"Nice" example for the security considerations.  But it doesn't
affect John's proposals a + b.  Or do you want more radical rules
like NFKC or SASLprep ?

Frank

---

From: Tony Finch <[email-masked]>
Date: Thu, 8 Feb 2007 15:33:53 +0000

> Here is a peculiar email address example worth to consider

Note that the addresses in the message header do not have to relate to the
addresses in the message envelope, so spoofing attacks similar to yours
are trivial without i18n.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTIES CROMARTY: EAST VEERING SOUTHEAST 5 OR 6, OCCASIONALLY 7, PERHAPS GALE
8 LATER. MODERATE OR ROUGH. SNOW SHOWERS. GOOD.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 08 Feb 2007 16:42:05 +0100

Randall Gellens wrote:
 
> We tried very hard in DRUMS to permit unquoted dots in phrases

I tried very hard to get rid of this idiosyncrasy in USEFOR, but
it's now one of two "acceptable" (MUST accept, MUST NOT generate)
obs-cenities adopted from RFC 2822... <sigh />

No leading, trailing, or adjacent dots as THE LAW is already odd.

But adding an exception from this syntax only for the US middle
initials or the DE title abbreviations is really weird, it has
side effects for stuff like phrases in a Keywords header field.

Frank

---

From: Soobok Lee <[email-masked]>
Date: Fri, 9 Feb 2007 20:11:51 +0900

> affect John's proposals a + b.  Or do you want more radical rules

What I tried to point out is that requiring <> don't guarantee more "safety".

Instead of requiring <>, we can prohibit  ,:;[]()<>@\"  and 
their NFKC-equivalents  in local-part explictly in [utf8headers].

There are many other look-alikes that are not their NFKC-equivalents, 
but let's ignore them here, and accept this as a cost.

Soobok

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 09 Feb 2007 14:59:27 -0000

On Thu, 08 Feb 2007 08:17:07 -0000, Randall Gellens <[email-masked]>  
wrote:

> We tried very hard in DRUMS to permit unquoted dots in phrases! After  
> all, no one likes to see his or her name in sneer quotes.  Too bad we  
> couldn't find a way to do so.

But you DID find a way to do so. It is called the <obs-phrase> which MUST  
be accepted, and it says:

    Note: The "period" (or "full stop") character (".") in obs-phrase is
    not a form that was allowed in earlier versions of this or any other
    standard.  Period (nor any other character from specials) was not
    allowed in phrase because it introduced a parsing difficulty
    distinguishing between phrases and portions of an addr-spec (see
    section 4.4).  It appears here because the period character is
    currently used in many messages in the display-name portion of
    addresses, especially for initials in names, and therefore must be
    interpreted properly.  In the future, period may appear in the
    regular syntax of phrase.

Clearly it should have been called the <extended-phrase> (how can it be  
"obs" when it was never in 822), and Pete has admitted on the ietf-822  
list that that should have been so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 09 Feb 2007 15:13:02 -0000

>         Second <  is  U+FF1C or U+FE64, not U+003C

All of which merely goes to show that local-parts MUST be normalized to  
NFKC, and our drafts should state that clearly (if they don't do so  
already). It comes back to the 'fax' test and the 'telephone' test which I  
mentioned right at the start of these discussions which should apply to  
all <addr-spec>s. Interoperability starts with the scrap of paper off  
which rou initially read the message, and ends with what the eye looking  
at the screen actually 'sees'. Moreover, it is the From and Reply-To  
headers that we should be particularly concerned with, because it is the  
envelope from that causes delivery to the correct destination (though even  
that should be normalized similarly).

Didn't we agree that sasl-prep contained just about the right degree of  
normalization for our purposes?

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 09 Feb 2007 15:18:00 -0000

> is no display name field,

And why not. Nobody has demonstrated any harm arising from that particular  
cause, and it would be bizarre to discard the long-standing conventions  
contained in RFC [2]822 unless actual harm could be demonstrated.

> I think permitting them would force us
> to parsers.

You deal with the issues that _will_ break parsers, not with the ones that  
won't.

> 	2822's rules about quoted phrases, including the rules

I disagree. None of the problems being reported would be cured by doing  
either of those things. If it ain't broke ...

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 10 Feb 2007 01:54:01 +0100

> What I tried to point out is that requiring <> don't guarantee

I think we're talking about different things, the a + b proposal
was about simplifying the task of a parser, but not some kind of
anti-phishing scheme.

> and their NFKC-equivalents  in local-part explictly

Maybe.  I certainly won't miss anything working only as quoted
pair.  And months ago, long before EAI was a WG, John proposed
to get rid of all obscure "features" in local parts (the cruft
in the RFC 3696 errata).

Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sat, 10 Feb 2007 11:02:00 +0100

--On 9. februar 2007 15:13 +0000 Charles Lindsey <[email-masked]> 
wrote:

> already). It comes back to the 'fax' test and the 'telephone' test which
> I mentioned right at the start of these discussions which should apply to
> all <addr-spec>s.

My memory says that you proposed the fax test, and the WG rejected it.
After all, the fax test fails for many occurences of USER05 vs USERO5.

> Interoperability starts with the scrap of paper off
> envelope from that causes delivery to the correct destination (though
> even that should be normalized similarly).
>

I have not agreed to that. I have seen lots of text suggesting that wise 
postmasters will only define mailboxes that are in SASLPrep or stronger, 
but I have seen lots of text suggesting that it would be unwise to require 
such normalization at any point between the sender and recipient.

               Harald

---

From: Soobok Lee <[email-masked]>
Date: Mon, 12 Feb 2007 23:46:01 +0900

> was about simplifying the task of a parser, but not some kind of

I understand that
 1) looks clearer to our eyes than 2).

  1) To: <[email-masked]>,<[email-masked]>
  2) To: [email-masked],[email-masked]

But, I can't imagine that future parsers drop support of form 2)
for simplicity of implementations.

This is another well-known example using BIDI control RLO
   (U+202E,right-to-left override). I applied this into 
   utf8-local-part.

   From: (RLO)[email-masked]

 is visually rendered to recipients as:

   From: [email-masked]

 This is angle-bracked one.

   From: <(RLO)[email-masked]>

 is rendered as:

   From: <<[email-masked]

Is the angled form safer in this case?  Maybe.

<script>
document.write(unescape('<p>To: [email-masked]'));
document.writeln("<br>");
document.write(unescape('<p>To: &lt;[email-masked]&gt;'));
document.writeln("<br>");
</script>

(All BIDI control characters behavior looks complex to me. 
I had misused them in idna-ipdate list. The above javascript scriptlet
s for verification purpose, especialy for me myself to be sure ).

> to get rid of all obscure "features" in local parts (the cruft

I am not sure whether I understood your word, but,
unlike utf8-dot-atom, quoted-string local-part  can contain _any_ characters 
including these quoted-pairs  \\ and \".

>From my reading RFC2822, quoted pairs cannot appear in utf8-dot-atom 
and dot-atom , but can appear in dcontent(domain literal) and
qcontent (every quoted-string). (Correct me if I am wrong.)

Soobok

----------------------------------------------------------------------
<quote from rfc2822>

qcontent        =       qtext / quoted-pair

quoted-pair     =       ("\" text) / obs-qp

obs-qp          =       "\" (%d0-127)

The only places in this
   standard where quoted-pair currently appears are ccontent, qcontent,
   dcontent, no-fold-quote, and no-fold-literal.

<quote from utf8headers>

utf8-local-part =  utf8-dot-atom / quoted-string / obs-local-part

</quote>

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 Feb 2007 11:18:26 -0500

> Is the angled form safer in this case?  Maybe.

Anyone who permits <RLO> in an email address had better
understand what he or she is doing and what the implications of
it might be.  We aren't going to be able to help avoid the mess
that could cause without writing global rules about the use of
Unicode and what characters can appear in local-parts, and we
have agreed to not do that.   My own opinion is that it would be
as stupid to insert such a character as it would be to insert
X'0E' or (X'18' X'28') at the beginning of an ASCII local-part.
One can do it, but the capability has been there all along and
no one who is sensible and trying to get mail sent and delivered
takes advantage of it.

     john

---

From: Chris Newman <[email-masked]>
Date: Mon, 12 Feb 2007 15:24:26 -0800

Actually, RFC 2822 does require recipients to accept unquoted dot in phrase 
(see the "obs-phrase" rule).  It's one of the few additions to the obsolete 
grammar that was not in RFC 822.

It should have been called out in a separate section as something we might move 
from the "MUST NOT generate" to "MAY generate" category in the future.  But at 
least it's there.

                - Chris

Randall Gellens wrote on 2/8/07 0:17 -0800:

> We tried very hard in DRUMS to permit unquoted dots in phrases! After all, no
> one likes to see his or her name in sneer quotes.  Too bad we couldn't find a
> way to do so.
> --
> Randall Gellens

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 13 Feb 2007 21:20:01 -0000

> After all, the fax test fails for many occurences of USER05 vs USERO5.

It was not a matter of accepting or rejecting it, because it is not a  
feature to in in(ex)cluded in any draft. Rather it is a self-evidently  
useful yardstick that should aid in deciding between various proposals  
that might be under discussion.

> I have not agreed to that.#

I think we agreed that SASL-prep was the best fit for any normalization  
that we might want to include/recommend/whatever.

> I have seen lots of text suggesting that wise postmasters will only  
> define mailboxes that are in SASLPrep or stronger,

Whether any normalization we may decide to include/recommend/whatever  
should take the form of "advice to poastmasters", or of stronger  
requirements for tests to be performed by agents in prescribed  
circumstances is an orthogonal issue as to whether it is best expressed in  
terms of sasl-prep or otherwise.

But, for sure, the issue certainly needs covering in some manner within  
our documents, and it seems to me that that is precisely what Soobok was  
calling for, given the nature of the examples he gave, which are far more  
troublesome than the well-known confusion between O and 0.

One gathers also that the internationalization of domains names is also  
currently stuck on much the same problem.

> but I have seen lots of text suggesting that it would be unwise to  
> require such normalization at any point between the sender and recipient.

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 14 Feb 2007 10:07:46 +0100

> that might be under discussion.
It is, however, rejected, so the claim that it's "self-evidently useful" 
has self-evidently failed.