Subject: [EAI] mailto: was RE: Rechartering

From: Shawn Steele <[email-masked]>
Date: Fri, 24 Jul 2009 04:48:35 +0000

I think we're digressing from rechartering to mailto :)

> Then it seems to me that you are in big trouble because
> deploying EAI with Downgrade --even in the syntax-only form
> Charles suggests-- means that users will encounter it working
> one way in some places (e.g., accepting non-ASCII addresses but
> not alternate address syntax) and in other ways (e.g., accepting
> non-ASCII and alternate addresses) in others, even within the
> same system.   That is, IMO, much worse even that having
> extended (non-ASCII) addresses work in some places but not in
> others.

>> I'd agree, but [email-masked] should still work :)0

> Sure.  But unless it can be parsed out of the above, that isn't
> interesting.

I disagree.  Assuming that [email-masked] is your Unicode alias and [email-masked] is your ASCII alias, you can always use mailto:[email-masked] and it should get to you regardless of any <> or whatnot.

I actually find the < > downgrade syntax fairly unlikely to be helpful.  Presumably your mail server understands the complete EAI syntax since you're providing both addresses.  Likewise my client knows about EAI if it understands <unicode <ASCII>>.  So pretty much the only place that <unicode <ASCII>> is going to gain anything is when my mail server is not EAI aware, but my client and your server are EAI aware.  It seems more likely to me that my server and client would both be EAI aware.  So, to me, the most likely scenarios would be when some in-between relay wasn't EAI aware, yet any such sender or receiver side relay would necessarily be EAI aware.  Relays that aren't tightly coupled with the sender/receiver side tend to be blacklisted anyway.

So to me, the <Unicode <ASCII>> syntax is mostly interesting merely to populate my address book with both addresses.

FWIW: mailto:Unicode@Unicode on Windows+Safari/IE will invoke Outlook with the address as expected.  I seriously doubt we'll block that behavior, and I think it should be permitted in the mailto draft.

>  Note that, if one had a Unicode-native system and
> simply removed the "ASCII address" restriction,
>    [email-masked]
> would parse as the address the user would expect while
>    <[email-masked] <[email-masked]>>
> would almost certainly get some sort of "invalid end of address"
> or "invalid unquoted space in the middle of the address" error.
> And, if it tried to "fix" that error, as many MUAs and some
> other systems would, life would get interesting.

Pretty much my feeling.

> Again, if you are in a hurry, the key question is whether we
> need downgrading.  If we don't, a large fraction of the issues
> I'm raising just disappear and I have no trouble working through
> a schedule (in your model) or a requirements plan (in Harald's).

In mailto: I don't find downgrade very helpful.  As I said, you'd have to have an updated client anyway to handle the new syntax, while mailto:unicode & mailto:ASCII are likely to work already.  I don't really object to the mailto: updates because I don't really see them having any practical impact :) and they provide some completeness.

I think downgrade is primarily interesting when I send mail from an EAI aware client to an EAI unaware client.  In that case the mail is downgraded and the recipient is able to reply to me.  In nearly every other case either the user has a Unicode address that may or may not work, or an ASCII address that works.  I think the downgrade behavior in this scenario is fairly complete.

So I think we pretty much sort of agree.  You think that without downgrade the experimental RFCs are somewhat stable.  I agree.  You ask if I think downgrade is required.  I'd split that out into downgrade when I send a mail, and "the rest of downgrade."  Downgrade when mailing from an EAI aware system to an EAI unaware system is useful and also somewhat reasonably defined, and I'd include that in my stable behavior.  mailto and most of the rest of downgrade I think is less helpful.

The scenarios I see are:
1) EAI to EAI, in which downgrade is unhelpful
2) Me giving you a Unicode address on a post-it, in which case downgrade is unhelpful (because I didn't give you the ASCII form)
3) Me giving you my ASCII address on a post-it, in which case downgrade is irrelevent because it's gonna work anyway.
4) Me mailing you from EAI aware to EAI unaware systems, in which case downgrade is helpful
    a) My mail gets to you, and if you reply to me, it'll work
    b) You probably won't get my Unicode address, but that's OK, because you don't know how to use it anyway.
    c) If you happen to have a client that can upgrade my address, then you'll have the info to send mail to me (that'll be downgraded on the way, probably)
5) Something like mailto:
    a) It'll continue to work with a downgraded ASCII address.
    b) If it happened to work with Unicode, it'll continue to work with Unicode.
    c) If it doesn't work with Unicode and is EAI unaware, then downgrade won't work anyway since it won't understand whatever has to happen to the protocol (like mailto)

So I think we're pretty close for the non-downgrade stuff.  For downgrade I think we're pretty close for the 95% useful cases.  For the remaining downgrade completeness stuff like mailto:, I agree that there're interesting issues and I don't think they're helpful for that 5%.  So I'd go ahead and standardize the non-downgrade stuff, the downgraded headers stuff, and I wouldn't worry about the other downgrade scenarios right now.

-Shawn

---

From: John C Klensin <[email-masked]>
Date: Fri, 24 Jul 2009 05:12:09 -0400

--On Friday, July 24, 2009 04:48 +0000 Shawn Steele
<[email-masked]> wrote:

>> one way in some places (e.g., accepting non-ASCII addresses
>> but not alternate address syntax) and in other ways (e.g.,
>> accepting non-ASCII and alternate addresses) in others, even
>> within the same system.   That is, IMO, much worse even that
>> having extended (non-ASCII) addresses work in some places but
>> not in others.
> 
>> Sure.  But unless it can be parsed out of the above, that
>> isn't interesting.
> 
> I disagree.  Assuming that [email-masked] is your
> Unicode alias and [email-masked] is your ASCII alias,
> you can always use mailto:[email-masked] and it should
> get to you regardless of any <> or whatnot.

And I don't know what you are suggesting.   If it is "use the
mailto: with the ASCII local part" then EAI is irrelevant.   If
it is "supply two separate mailto: URIs", then you are, in
effect, suggesting downgrading only at the sending MUA or
submission server.   I'm strongly in favor of doing as much
downgrading as is necessary and possible there, but it doesn't
require any protocol changes or new syntax.

> I actually find the < > downgrade syntax fairly unlikely to be
> helpful.  Presumably your mail server understands the complete
> EAI syntax since you're providing both addresses.  Likewise my
> client knows about EAI if it understands <unicode <ASCII>>.
> So pretty much the only place that <unicode <ASCII>> is going
> to gain anything is when my mail server is not EAI aware, but
> my client and your server are EAI aware.  It seems more likely
> to me that my server and client would both be EAI aware.  So,
> to me, the most likely scenarios would be when some in-between
> relay wasn't EAI aware, yet any such sender or receiver side
> relay would necessarily be EAI aware.  Relays that aren't
> tightly coupled with the sender/receiver side tend to be
> blacklisted anyway.

And that is precisely one of the arguments why downgrading on
the fly and mid-network is not likely to be useful enough to be
worth all the trouble, extra syntax, etc.

> So to me, the <Unicode <ASCII>> syntax is mostly interesting
> merely to populate my address book with both addresses.

And, depending on how your address book is designed --a topic
_far_ outside the scope of the WG-- a better way might be
something isomorphic with
   <email>Unicode</email>
   <alt-email>ASCII</alt-email>
or
   <email>Unicode <alt>ASCII</alt></email>
saving a lot of parsing aggravation when the addresses are
needed, _especially_ if the downgrade point is the originating
system.

>...
>> I'm raising just disappear and I have no trouble working
>> through a schedule (in your model) or a requirements plan (in
>> Harald's).
 
> In mailto: I don't find downgrade very helpful.  As I said,
> you'd have to have an updated client anyway to handle the new
> syntax, while mailto:unicode & mailto:ASCII are likely to work
> already.  I don't really object to the mailto: updates because
> I don't really see them having any practical impact :) and
> they provide some completeness.

Mostly agreed, but, if we specify the two-address syntax,
mailto: ought to be able to access it somehow.   That is another
case of "providing for on-the-fly downgrading complicates a lot
of things without a lot of payoff".

> I think downgrade is primarily interesting when I send mail
> from an EAI aware client to an EAI unaware client.  In that
> case the mail is downgraded and the recipient is able to reply
> to me.  In nearly every other case either the user has a
> Unicode address that may or may not work, or an ASCII address
> that works.  I think the downgrade behavior in this scenario
> is fairly complete.

Complete, yes.   Reasonably guaranteed to work, no.  And it does
suggest another case that might be worth considering, which is
permitting downgrade syntax and logic only for backward-pointing
addresses (MAIL, "From:", "Sender:", "Reply-to:") but not
forward-pointing ones (RCPT, "To:", etc.)

> So I think we pretty much sort of agree.  You think that
> without downgrade the experimental RFCs are somewhat stable.
> I agree.  You ask if I think downgrade is required.  I'd split
> that out into downgrade when I send a mail, and "the rest of
> downgrade."  Downgrade when mailing from an EAI aware system
> to an EAI unaware system is useful and also somewhat
> reasonably defined, and I'd include that in my stable
> behavior.  mailto and most of the rest of downgrade I think is
> less helpful.

I think we agree although I'd like to push back mildly on even
the backward-pointing addresses just because of the costs of
dealing with the new syntax at all.

> 2) Me giving you a Unicode address on a post-it, in which case
> downgrade is unhelpful (because I didn't give you the ASCII
> form) 3) Me giving you my ASCII address on a post-it, in which
> case downgrade is irrelevent because it's gonna work anyway.
> 4) Me mailing you from EAI aware to EAI unaware systems, in
> which case downgrade is helpful     a) My mail gets to you,
> and if you reply to me, it'll work

Well, maybe.   It'll work iff the downgraded info survives the
gateways, etc., some of which may be EAI-unaware to the point of
being hostile.

>     b) You probably won't
> get my Unicode address, but that's OK, because you don't know
> how to use it anyway.

right

>     c) If you happen to have a client
> that can upgrade my address, then you'll have the info to send
> mail to me (that'll be downgraded on the way, probably) 5)
> Something like mailto:
>     b) If it happened to work with Unicode, it'll continue to
> work with Unicode.     c) If it doesn't work with Unicode and
> is EAI unaware, then downgrade won't work anyway since it
> won't understand whatever has to happen to the protocol (like
> mailto)
> 
> So I think we're pretty close for the non-downgrade stuff.
> For downgrade I think we're pretty close for the 95% useful
> cases.  For the remaining downgrade completeness stuff like
> mailto:, I agree that there're interesting issues and I don't
> think they're helpful for that 5%.  So I'd go ahead and
> standardize the non-downgrade stuff, the downgraded headers
> stuff, and I wouldn't worry about the other downgrade
> scenarios right now.

I could probably live with this, but want to see the outcome of
more testing.

     john

---

From: Shawn Steele <[email-masked]>
Date: Fri, 24 Jul 2009 15:48:14 +0000

> And I don't know what you are suggesting.

I'm saying that for mailto: there are 2 possibilities with the current draft:

1) If I fear clients don't understand EAI, then I use the ASCII local part, in which case, yes, EAI is irrelevant.
2) If I don't think they understand EAI, then I use the Unicode locale part.

The 3rd option: Provide both (downgrade), is useless.  If my client isn't updated to support 2), then its as bad as only providing Unicode (except in a few cases where some intermediate server must downgrade, which is likely rare in practice).

Long-term, 2) is sufficient as most users should become EAI aware.  As a transition period I think that 1 + 2 is sufficient for EAI.

> worth all the trouble, extra syntax, etc.

It's useful in the limited place where I send mail from an EAI aware server to an unaware system.  In that case the mail will still make it and I'll still get a reply.  So I think downgrade is interesting for UTF8SMTP (the 95% case), and other places are pretty uninteresting.

>needed, _especially_ if the downgrade point is the originating

I said I didn't think the current mailto: downgrade stuff was helpful :)

> forward-pointing ones (RCPT, "To:", etc.)

That makes sense

> dealing with the new syntax at all.

That makes sense too, just drop the Unicode addresses and use the downgraded ones?  It'd certainly be simpler, and replies would still go back to the sender.  When the recipient's side was upgraded, then they could get the Unicode address, and it wouldn't really be helpful 'til then anyway.

> I could probably live with this, but want to see the outcome of

More testing is always good.  My fear is that China is basically headed this way anyway, so it seems that we're on track to end up here anyway, for better or for worse.

-Shawn

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 27 Jul 2009 11:47:39 +0100

On Fri, 24 Jul 2009 05:48:35 +0100, Shawn Steele  
<[email-masked]> wrote:

> I actually find the < > downgrade syntax fairly unlikely to be helpful.   
> Presumably your mail server understands the complete EAI syntax since  
> you're providing both addresses.  Likewise my client knows about EAI if  
> it understands <unicode <ASCII>>.  So pretty much the only place that  
> <unicode <ASCII>> is going to gain anything is when my mail server is  
> not EAI aware, but my client and your server are EAI aware.  It seems  
> more likely to me that my server and client would both be EAI aware.   
> So, to me, the most likely scenarios would be when some in-between relay  
> wasn't EAI aware, yet any such sender or receiver side relay would  
> necessarily be EAI aware.  Relays that aren't tightly coupled with the  
> sender/receiver side tend to be blacklisted anyway.

But the in-between server has no business looking at the headers inside  
the message, so it should not care is they contain To: <...<...>>. The  
important thing fot the in-between server (which preumably does not  
advvertise the UTF8 capability, and so SHOULD NOT have been offered the  
message in the first place) is whether it will be confused by the way in  
which the alt-address appears in the envelope (which is NOT that  
<...<...>> format). If it finds something in the RCPT TO that it thinks it  
knows how to forawrd to, it will try to forward it AS-IS (and may well  
succeed unless it is one of those Sendmails that still doesn't transmit  
7-bit headers cleanly). If it thinks it does not know how to forward it to  
the RCPT TO, it will try to bounce it to the MAIL FROM (if it had not  
already rejected with a 5xx). That might indeed fail to work.

But noway should it be looking inside the headers, except to add its own  
Received (and maybe to reorder the headers which some systems insist on  
doing). It it can't do either of those things without parsing every  
individual header, then it is indeed in trouble, but are respectable  
servers out there really like that?

But as Shawn point out, all other scenarious OUGHT to work, provided those  
that DO downgrade when faced with an ongoing server that does not  
advertise UTF8 do so correctly.

So servers are (or should not be) the problem. The only problem is all  
those other bits of software which ask you to type in an email address and  
then don't like what they see. They do not lie on the critical path of  
trnasmitting mail. And Microsoft will fix them pretty promptly IF it  
regards the Chinese market as sufficiently lucrative to make it worth  
their while (which, of course, they might not).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5