Subject: [EAI] Discussion of draft-ietf-eai-mailinglist-01.txt

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 13 Mar 2007 19:56:40 -0000

The mailing-list draft has been short of discussion. Time to rectify that.

3 Scenarios Involving Mailing Lists

Whilst this section covers many interesting cases, it is not clear that  
all the possible scenarios have been addressed.

I therefore suggest classifying systems against three orthogonal axes:

1. Does it have a UTF8 submission address?
2. Does it accept subscriptions to UTF8 addresses?
3. Does it accept UTF8SMTP messages?

In principle, that gives 8 different kinds of mailing lists, though not  
all of them are sensible. But the requirements can be discussed separately  
for each of those axes.

1. If it has a UTF8 submission address, it SHOULD publish an ASCII  
alt-address as well. Clearly, it needs to sit behind a UTF8SMTP-enabled  
MDA. If it also intends to use a UTF8 Return-Path, that too should have an  
ASCII alt-address, and it will need a UTF8SMTP-enabled MSA with at least  
the capability to accept that address in a MAIL FROM. There are also  
implications for the List-* headers (see below).

2. If it accepts UTF8 subscriptions, it SHOULD require an ASCII  
alt-address as well for each subscriber. It also requires a mechanism to  
accept requests to add those subscription addresses (preferably specified  
in the form <utf8@utf8<ascii@ascii>>). And presumably it needs a  
UTF8SMTP-enabled MSA with at least the capability to downgrade the  
envelope and any From/Reply-To header field in a message.

3. If it accepts UTF8SMTP messages, it needs a UTF8SMTP-enabled MSA with  
at least the capability to downgrade messages. It it is really smart, it  
might prepare both UTF8SMTP and downgraded versions of each message, and  
choose which to send based on the capability of the MTA identified by each  
recipient's MX record.

Many of those points are made in the present draft, but not necessarily  
broken down according to the particular kind of mailing list. It may  
indeed be quite sensible to open existing mailing lists to UTF8  
subscribers even though message content is still to be in ASCii-English.  
And also to admit UT8FSMTP messages to lists which still require  
ASCII-only subscriptions.

4 Mailing List Header Fields

Please can someone who understands the subtle difference tell us whether  
we should now be referring to "URI"s rather than "URL"s.

     By and large, the data contained in these mailing list header fields
     are URLs which often contain email addresses.  The same mechanism
     should be used for these fields as with other fields specifically
     discussed in the UTF8-Headers document [EAI-UTF8Headers].  Generally
     therefore, for fields that contain an internationalized email
     address, it could be expressed as a UTF8 string.

Leaving alt-addresses aside for the moment, how does this differ from just  
using IRIs instead of URIs in these fields? In any case, it is an  
extension to the syntax of these fields as defined in RFCs 2369 and 2919,  
with a downgrade provision either here or in our downgrade draft, so it  
would be as easy to extend by allowing IRIs as by making special  
allowances for UTF8 addresses in mailtos. Moreover, IRIs come with a nice  
downgrade to URIs already provided by RFC 3987.

     These fields might contain other URLs, such as HTTP.  In these
     cases, there are no EAI-specific considerations, since these
     non-mail-related URLs are out of scope for internationalized email
     documents, and have been addressed elsewhere, such as RFC3987
     "Internationalized Resource Identifier (IRI)" [RFC3987].

But it would be bizarre to provide different extension mechanisms for  
mailto: URIs than for http: URIs. Much simpler to use IRIs for both cases.

     Because the email addresses are expressed as "mailto" URLs, further
     specifications for presentation and inclusion of alt-addresses as
     well as other considerations may be necessary, other than simply
     following RFC3987 "Internationalized Resource Identifier (IRI)"
     [RFC3987] specifications.  This will be further discussed in Section
     6.

But this is the _real_ problem, and Section 6 does not really add anything  
beyond what is said here. Essentially, what we need is an extension to the  
mailto: for alt-addresses (whether it appears in a URI or an IRI). And  
such an extension could well use our "<utf8@utf8<ascii@ascii>>" syntax.

The question is how to bring this about? Martin has declared willingness  
to bring it into his new mailto draft at some stage (and now we at least  
have an agreed notation for him to use). But I don't think it would be a  
good idea to invent our own mechanism, with its own downgrade, for use in  
the interim. Better to forego the possibility of alt-addresses for the  
moment, with a NOTE to explain that an extended mailto syntax is expected  
to be defined in the near future. But, in fact, there is another way:

The alternative is to use two URIs or IRIs in those headers which need it.  
RFC 2369 is not particularly well written, but if you examine it closely  
you will see that comma-separated lists of URLs are allowed. So one could  
have, for example:

    List-Post: <utf8-submission@utf8>, <ascii-submission@ascii>

and RFC 2369 already tells you to examine them from left-to-right and to  
use the first which you have the capability to use.

6 Further Discussion

Much of this section seems to repeat what has been (or should be) said in  
the earlier sections.

     ...  Alternatively, it may be useful
     to consider having a mechanism, such as an additional SMTP command,
     for the receiving MTA (in this case the mailing list) to request the
     alt- address.  This may be useful in other scenarios as well,
     especially those concerning multiple recipients.

But if you want to invent new SMTP commands, then our smtp draft is the  
place to do it.

7 IANA Considerations

     None.

If you are going to extend the various List-* headers (whether by allowing  
IRIs or any other way of admitting UTF8 into them), then the IANA registry  
of header fields needs to be updated (RFC 3864).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Randall Gellens <[email-masked]>
Date: Sun, 18 Mar 2007 03:11:31 -0700

>  The mailing-list draft has been short of discussion. Time to rectify that.

I agree.  Thank you for doing so.

>  Whilst this section covers many interesting cases, it is not clear 
> that all the possible scenarios have been addressed.
>
>  In principle, that gives 8 different kinds of mailing lists, though 
> not all of them are sensible. But the requirements can be discussed 
> separately for each of those axes.

Sounds reasonable.

> alt-address as well. Clearly, it needs to sit behind a 
> UTF8SMTP-enabled MDA. If it also intends to use a UTF8 Return-Path, 
> that too should have an ASCII alt-address, and it will need a 
> UTF8SMTP-enabled MSA with at least the capability to accept that 
> address in a MAIL FROM. There are also implications for the List-* 
> headers (see below).

I'm inclined to think that even lists which use a UTF8 submission 
address should use an ASCII return-path.  Seems much safer.  I'd like 
to know what others think of this, though.

> alt-address as well for each subscriber. It also requires a 
> mechanism to accept requests to add those subscription addresses 
> (preferably specified in the form <utf8@utf8<ascii@ascii>>). And 
> presumably it needs a UTF8SMTP-enabled MSA with at least the 
> capability to downgrade the envelope and any From/Reply-To header 
> field in a message.

Seems reasonable (I think some of this is already said).

>
>  3. If it accepts UTF8SMTP messages, it needs a UTF8SMTP-enabled MSA 
> with at least the capability to downgrade messages. It it is really 
> smart, it might prepare both UTF8SMTP and downgraded versions of 
> each message, and choose which to send based on the capability of 
> the MTA identified by each recipient's MX record.

That would require a very close coordination between the list agent 
and the MTA/MSA that is beyond what is normally expected.

>
>  Many of those points are made in the present draft, but not 
> necessarily broken down according to the particular kind of mailing 
> list. It may indeed be quite sensible to open existing mailing 
> lists to UTF8 subscribers even though message content is still to 
> be in ASCii-English. And also to admit UT8FSMTP messages to lists 
> which still require ASCII-only subscriptions.

I'm not sure about this last point.  Why accept UTF8SMTP messages to 
a list which has only ASCII-address subscribers?

>  Please can someone who understands the subtle difference tell us 
> whether we should now be referring to "URI"s rather than "URL"s.
>
>  Leaving alt-addresses aside for the moment, how does this differ 
> from just using IRIs instead of URIs in these fields? In any case, 
> it is an extension to the syntax of these fields as defined in RFCs 
> 2369 and 2919, with a downgrade provision either here or in our 
> downgrade draft, so it would be as easy to extend by allowing IRIs 
> as by making special allowances for UTF8 addresses in mailtos. 
> Moreover, IRIs come with a nice downgrade to URIs already provided 
> by RFC 3987.
>
>  But it would be bizarre to provide different extension mechanisms 
> for mailto: URIs than for http: URIs. Much simpler to use IRIs for 
> both cases.
>
>  But this is the _real_ problem, and Section 6 does not really add 
> anything beyond what is said here. Essentially, what we need is an 
> extension to the mailto: for alt-addresses (whether it appears in a 
> URI or an IRI). And such an extension could well use our 
> "<utf8@utf8<ascii@ascii>>" syntax.
>
>  The question is how to bring this about? Martin has declared 
> willingness to bring it into his new mailto draft at some stage 
> (and now we at least have an agreed notation for him to use). But I 
> don't think it would be a good idea to invent our own mechanism, 
> with its own downgrade, for use in the interim. Better to forego 
> the possibility of alt-addresses for the moment, with a NOTE to 
> explain that an extended mailto syntax is expected to be defined in 
> the near future. But, in fact, there is another way:
>
>  The alternative is to use two URIs or IRIs in those headers which 
> need it. RFC 2369 is not particularly well written, but if you 
> examine it closely you will see that comma-separated lists of URLs 
> are allowed. So one could have, for example:
>
>  and RFC 2369 already tells you to examine them from left-to-right 
> and to use the first which you have the capability to use.

I'm concerned that many existing implementations of List-* headers 
wouldn't work in this case.  (Either they wouldn't handle multiple 
addresses, or if they did, they wouldn't handle UTF8 addresses.)

It might be safer to have new List-* header fields for EAI addresses.

So, an EAI list might include two sets of each List-* header, one for 
ASCII address using the existing List-* header names, and one for EAI 
addresses, using the new List8-* names.

>
>
>  6 Further Discussion
>
>  Much of this section seems to repeat what has been (or should be) 
> said in the earlier sections.
>
>  But if you want to invent new SMTP commands, then our smtp draft is 
> the place to do it.

Yes, this should be deleted.

>
>
>  7 IANA Considerations
>
>      None.
>
>  If you are going to extend the various List-* headers (whether by 
> allowing IRIs or any other way of admitting UTF8 into them), then 
> the IANA registry of header fields needs to be updated (RFC 3864).

Yes, thank you.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
There is no human problem which could not be solved if people would
simply do as I advise.                                -- Gore Vidal

---

From: Yangwoo Ko <[email-masked]>
Date: Mon, 19 Mar 2007 05:04:09 +0900

Randall Gellens wrote:
> address should use an ASCII return-path.  Seems much safer.  I'd like to 
> know what others think of this, though.

By making alternative addresses optional rather than mandatory, we are 
expecting that future non ASCII world users usually would not see all 
ASCII addresses at all. However, if a return-path is not for casual 
human users' mailing experience but for list administrations and 
restricting return-path to all ASCII increases the delivery ratio of 
administrative messages, "SHOULD use an ASCII return-path" might be OK. 
Otherwise, SHOULD seems too strong.

 > (snip)

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 19 Mar 2007 13:00:15 -0000

>> willingness to bring it into his new mailto draft at some stage (and  
>> now we at least have an agreed notation for him to use). But I don't  
>> think it would be a good idea to invent our own mechanism, with its own  
>> downgrade, for use in the interim. Better to forego the possibility of  
>> alt-addresses for the moment, with a NOTE to explain that an extended  
>> mailto syntax is expected to be defined in the near future. But, in  
>> fact, there is another way:
>>
>>  The alternative is to use two URIs or IRIs in those headers which need  
>> it. RFC 2369 is not particularly well written, but if you examine it  
>> closely you will see that comma-separated lists of URLs are allowed. So  
>> one could have, for example:
>>
>>  and RFC 2369 already tells you to examine them from left-to-right and  
>> to use the first which you have the capability to use.
>

Well RFC 2369 declares that such comma-separated lists are part of the  
standard, so any implementation that barfs on them is broken.

But, first of all, can we look at how these headers are actually used? To  
a large extent, surely, they are for human use. A human wants to  
unsubscribe from the list, so he asks his browser to show all the headers,  
finds the one that says List-Unsubscribe, and clicks on what he sees there  
(which will presumably be a mailto:).

So which of these headers is likely to be looked at an acted upon by some  
software that need to understand them? What is current practice here?

The obvious candidate is the List-Post header, which might be examined by  
some MUA in response to pressing a "Followup" button (in fact, I wish MUAs  
would provide such a button instead of forcing you to use Reply or  
Reply-to-All, neither of which do what you actually want). Such a facility  
would certainly expect to see a mailto: and to try to mail the followup to  
it, and if it found a utf8 address there might get upset (a  
UTF8SMTP-capable MUA, of course, might look to see if there was a second  
ascii address and use that if its MSA demanded downgrading).

But I doubt any existing agent is actually going to barf if is sees more  
than one address. More likely, it will try the first one it sees, and  
never look any further.

As to what happens when a non-UTF8SMTP agent sees a utf8 address, then  
that is a fair question. Note that it could easily see such an address  
even in a dongraded message, since any IRI would simply have been  
downgraded to a URI, and the utf8 address wold still be there, but %hex  
encoded.

Again, if a human sees that, and also sees a second ASCII address, he will  
surely click on the right one. An automated system is going to be confused  
whatever we put in there, unless we specify a downgrade mechansim that  
specifically looks at the mailto:s in the List-* headers and removes any  
utf8 addresses and just leaves the ascii ones.
> addresses, using the new List8-* names.

That seems like too much departure from the present usage. We should avoid  
that unless we absolutely cannot see any other way.

But, first, we really do need to know what current software actually does  
with those headers, other than to display them.

---

From: Philip Guenther <[email-masked]>
Date: Mon, 19 Mar 2007 07:18:41 -0600

On Mon, 19 Mar 2007, Charles Lindsey wrote:
...
> But, first of all, can we look at how these headers are actually used? To a 
> large extent, surely, they are for human use. A human wants to unsubscribe 
> from the list, so he asks his browser to show all the headers, finds the one 
> that says List-Unsubscribe, and clicks on what he sees there (which will 
> presumably be a mailto:).
>

Pine notices the headers and provides menu items to unsubscribe, 
subscribe, ask for help, etc.  Here's the screen for the mailing list 
commands it gave for your message.

--------------
                                Mail List Commands

Message 78 has information associated with it that explains how to
participate in an email list. An email list is represented by a single
email address that users sharing a common interest can send messages to
(known as posting) which are then redistributed to all members of the
list (sometimes after review by a moderator).

List participation commands in this message include:

  *  A method to get information about the list and instructions on how to
     join.

     Select HERE to seek help.

  *  Methods to remove yourself from the list (Unsubscribe).

      1. Select HERE to UNsubscribe.

      2. Select HERE to UNsubscribe.

  *  Methods to add yourself to the list (Subscribe).

      1. Select HERE to Subscribe.

      2. Select HERE to Subscribe.

  *  A method to send a message to the entire list (Post).

     Select HERE to post a message.

  *  A method to view archive of messages sent to the list.

     Select HERE to view the archive.

--------------

...
> But I doubt any existing agent is actually going to barf if is sees more than 
> one address. More likely, it will try the first one it sees, and never look 
> any further.

Pine gives options for all that it finds, as seen above.

Philip Guenther

---

From: Kari Hurtta <[email-masked]>
Date: 19 Mar 2007 19:39:12 +0200

"Charles Lindsey" <[email-masked]> writes in gmane.ietf.ima:

> But, first of all, can we look at how these headers are actually used?
> To  a large extent, surely, they are for human use. A human wants to
> unsubscribe from the list, so he asks his browser to show all the
> headers,  finds the one that says List-Unsubscribe, and clicks on what
> he sees there  (which will presumably be a mailto:).
> 
> So which of these headers is likely to be looked at an acted upon by
> some  software that need to understand them? What is current practice
> here?
> 
> The obvious candidate is the List-Post header, which might be examined
> by  some MUA in response to pressing a "Followup" button (in fact, I
> wish MUAs  would provide such a button instead of forcing you to use
> Reply or  Reply-to-All, neither of which do what you actually
> want). Such a facility  would certainly expect to see a mailto: and to
> try to mail the followup to  it, and if it found a utf8 address there
> might get upset (a  UTF8SMTP-capable MUA, of course, might look to see
> if there was a second  ascii address and use that if its MSA demanded
> downgrading).
> 
> But I doubt any existing agent is actually going to barf if is sees
> more  than one address. More likely, it will try the first one it
> sees, and  never look any further.

On my MUA   reply-to-list (Rl)  uses always first mailto -link
from List-post header field. There is also soem other reply commands
which uses first mailto -link. All these reply commands selects
first available mailto link (reply-to-owner (Ro) uses List-Owner).

On mail-to-list (Il) it is possible to select which link from
List-Post header field used. Same apply also for other posting
commands (mail-to-owner (Io) -- List-Owner, list-help (Ih)
-- List-Help and so on).  All these header fields are supported 
(assuming  that they include some mailto links).

/ Kari Hurtta

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 20 Mar 2007 11:57:51 -0000

> commands it gave for your message.

Cool!

Would it be possible for you to check what it does if it sees several URIs  
in one of those headers (e.g., does it only give access via the first, or  
does it allow you to pick the one you want, or does it explode)? And what  
happens if it sees an address in UTF8 (%hex encoded within the URI, of  
course)?

Not that we can base our standard just on what Pine does, or course, but  
if it can be shown that most current MUAs at least handle these cases  
gracefully, then that could suggest a way forward to where an UTF8SMTP  
upgrade to those MUAs could do the whole thing right.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 20 Mar 2007 12:02:51 -0000

> first available mailto link (reply-to-owner (Ro) uses List-Owner).

OK, which MUA is that? It seems it is behaving sensibly so far.

Can you check what happens if you give it a UTF8 address (%hexc encoded  
inside the URI, of course)? DOes it barf, or does it pass it on toe the  
MSA and let the MSA barf?

---

From: Kari Hurtta <[email-masked]>
Date: 20 Mar 2007 17:25:26 +0200

> OK, which MUA is that? It seems it is behaving sensibly so far.

http://www.elmme-mailer.org/        That is my version of Elm

> Can you check what happens if you give it a UTF8 address (%hexc
> encoded  inside the URI, of course)? DOes it barf, or does it pass it
> on toe the  MSA and let the MSA barf?

I have written that code, but I do not remeber exactly. Probably
it goes to MSA (sendmail or whatever).   I do not remember that I check
addresses for 8-bit content.

I will examine that. 

/ Kari Hurtta

---

From: Kari Hurtta <[email-masked]>
Date: 20 Mar 2007 19:10:06 +0200

> addresses for 8-bit content.

Yes. It was passed to sendmail.

hurtta@Hurtta06k:~/mail/elmme+$ fgrep List-Post Test.5
List-Post: <mailto:[email-masked]>
hurtta@Hurtta06k:~/mail/elmme+$

I used command 'Rl'   (reply to list).

Debug log confirms this:

mailmsg2.c: Preparing mail for sending: one part (not multipart)
mailmsg2.c: Preparing mail for sending: part 0 -- 429 bytes written to temp
mailmsg2.c: Preparing mail for sending: total 429 bytes body, ~398 bytes headers
mailer.c  : Sending mail via sendmail mailer, 1 recipients
syscall.c : start_run: [0] /usr/sbin/sendmail
syscall.c :            [1] -oi
syscall.c :            [2] -oem
syscall.c :            [3] -f
syscall.c :            [4] <khurtta@XXXXXXXX>
syscall.c :            [5] -N
syscall.c :            [6] success,failure,delay
syscall.c :            [7] --
syscall.c :            [8] [email-masked]
syscall.c :     infd=12, outfd=-1
syscall.c :     fd=13
syscall.c : Child: added SHELL=/bin/sh
syscall.c : start_run: child pid=11746
syscall.c : run_already_done=0 (w=0)
<..>
mailer.c  : Sleeping ( 2 seconds ) for completion!
schedule.c: wait_for_timeout:    START | seconds=2
syscall.c : got_sigchld: Setting handle_sigchld
schedule.c: wait_for_timeout=0  END (errno=4)
mailer.c  :  -- sleeping interrupted
syscall.c : raw_exit: no state change
posixsig.c: convert_status=0 (sig): exit_code=64
syscall.c : run_already_done: exit_code=64, sig=0
syscall.c : run_already_done=1
mailer.c  : Finished sending mail via sendmail mailer, ret=1 exit_stat=64
termbuffer: FlushBuffer: 7 bytes writted to 3, 0 left
error message=mailer returned error status 64
out_utils.: ERROR message: mailer returned error status 64

/ Kari Hurtta