Subject: [ietf-822] Different "replying" modes in a MUA

From: Jan Kundrát <[email-masked]>
Date: Fri, 04 Jan 2013 17:44:33 +0100

Hi,
I'm struggling to come up with the best approach for designing the "reply" feature in Trojitá. As far as I know, there's no single "best practices" document to follow (and many "XYZ considered harmful" ones and an interesting one about how Evolution works [1]). Here's what I'd like to do:

- Always offer multiple ways to reply. These actions will be represneted by an expandable button (i.e. a button with an arrow on the right which performs the "reasonable thing" by default but allows you to click the arrow to see the other actions).

- The rules for picking up the recipients will be the following:

1) If there's a List-Post header and its value is not set to "NO" (or an equivalent) and there's at least one mailto: URL in there, the "Reply to List" is enabled and listed as the default option. When user selects this action, all of the Sender, From, Reply-To, Cc and Bcc are ignored.

2) "Reply All" option is always available and will generate a list of recipients using the following rules:

  - All addresses in the message's From and Reply-To will be used in the "To" field
  - All addresses in the message's To will be in the Cc
  - All addresses in the message's Cc will be in the Cc
  - All addresses in the message's Bcc will be in the Bcc

After this is complete, a list will be "sanitized":
  - Duplicate enteries in each To/Cc/Bcc are be removed
  - Addresses already in To are removed from Cc and Bcc
  - Addresses already in Cc are removed from Bcc

The List-Post and the Sender headers are ignored when doing the Reply-All thing.

The "Reply All" is the second candidate for a default (i.e. the default when "Reply List" is not available).

3) "Private Reply" option is always available and produces a message with the following recipient(s) in the "To" field:

  - If the message contains a Reply-To, each of those which are at the same time *not* listed in any List-Post addresses are used. If the resulting set is non-empty, the From header is ignored. This means that any address in the Reply-To which is also listed in the List-Post is silenty ignored and anything not in the List-Post is used.
  - The Sender header is always ignored.
  - If the recipients list is empty at this point, everything from the From field is used.
  - If the resulting set contains more than one address, the duplicates are eliminated.

I am very interested in hearing what you think about this scheme. I realize that this is a topic with a huge potential for a good flame, and I suspect there are people who have very different opinions as to what is best. Despite that, I'll be happy to hear what issues are lurking in the algorithm I describe and what drawbacks I'm eniterly missing. Please also feel free to point me to any existing discussion as long as it's different from [1], [2] and [3] which I've read already.

When the discussion settles, I plan to impleemnt the outcome in Trojitá (along with a test case with plenty of examples matching real-world scenarios). Thanks for your help!

With kind regards,
Jan

[1] http://david.woodhou.se/reply-to-list.html
[2] http://www.unicom.com/pw/reply-to-harmful.html
[3] http://woozle.org/~neale/papers/reply-to-still-harmful.html

-- 
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/

---

From: SM <[email-masked]>
Date: Fri, 04 Jan 2013 10:15:43 -0800

Hi Jan,
At 08:44 04-01-2013, Jan Kundrát wrote:
>I'm struggling to come up with the best approach 
>for designing the "reply" feature in Trojita. As 
>far as I know, there's no single "best 
>practices" document to follow (and many "XYZ 
>considered harmful" ones and an interesting one about how Evolution works [1]).

I read the following in RFC 5322:

   'When the "Reply-To:" field is present, it indicates the address(es)
    to which the author of the message suggests that replies be sent.'

You message did not contain a "Reply-To:" 
field.  As such I am left will the following choices:

  (i)   Reply to the author

  (ii)  Reply to the mailing list

  (iii) Reply to the author and the mailing list

I am replying to you and copying the message to 
the mailing list.  It is a personal decision and not a "best practice".

>  Here's what I'd like to do:
>
>- Always offer multiple ways to reply. These 
>actions will be represneted by an expandable 
>button (i.e. a button with an arrow on the right 
>which performs the "reasonable thing" by default 
>but allows you to click the arrow to see the other actions).
>
>1) If there's a List-Post header and its value 
>is not set to "NO" (or an equivalent) and 
>there's at least one mailto: URL in there, the 
>"Reply to List" is enabled and listed as the 
>default option. When user selects this action, 
>all of the Sender, From, Reply-To, Cc and Bcc are ignored.

Ok, you are using "List-Post:" to identify 
whether the reply to the list is possible.

>2) "Reply All" option is always available and 
>will generate a list of recipients using the following rules:
>

The Bcc would be to your address.  It's like 
replying to yourself.  The "To" is similar if 
your address is in there.  I could remove it or, 
if my MUA has the option, set a preference.

If there is a "Reply-To:" I don't use the 
"From:".  If the "Reply-To:" is set by the 
mailing list I could use the "From:" if I would 
like the reply to go to all addresses.

>The "Reply All" is the second candidate for a 
>default (i.e. the default when "Reply List" is not available).
>
>
>3) "Private Reply" option is always available 
>and produces a message with the following recipient(s) in the "To" field:
>
>  - If the message contains a Reply-To, each of 
> those which are at the same time *not* listed 
> in any List-Post addresses are used. If the 
> resulting set is non-empty, the From header is 
> ignored. This means that any address in the 
> Reply-To which is also listed in the List-Post 
> is silenty ignored and anything not in the List-Post is used.
>  - If the recipients list is empty at this 
> point, everything from the From field is used.

Ok.

There was a discussion about Bcc (see thread at 
http://www.ietf.org/mail-archive/web/ietf-smtp/current/msg00112.html 
). There may be some good advice in there.

Regards,
-sm

---

From: Keith Moore <[email-masked]>
Date: Fri, 4 Jan 2013 23:14:59 -0500

(oh wow.  here we go again.)

1. I disagree with the RFCs about this, and always have.   (I state this up front just to make it clear that I am not proposing a novel way of interpreting the RFCs; I'm just saying that I disagree with them.   So there's no need to try to reconcile my view with what the RFCs say - we're just diametrically opposed.)

2. Regardless of the presence or absence of Reply-To or any other field, it is ALWAYS the recipient's job to decide to whom he wants to address replies.  

3. It is the job of the recipient's mail user agent ("agent" = acting on the recipient's behalf) to obey the recipient's wishes.   In particular, (a) it should allow the recipient to reply to whomever he wishes to reply and (b) make that behavior as obvious to the recipient as possible, so that the MUA's behavior will be unsurprising to the recipient of the subject message (the author of the reply).   

(Note that "to whomever he wishes to reply" might include addresses from any of the headers of the subject message, and/or addresses  that don't appear in the subject message at all.)

4. UAs that change their behavior based on the presence or absence of fields in the subject message (especially fields that aren't normally displayed) generally FAIL to meet the "unsurprising behavior" test.

5. It's not unusual for a recipient to want to change the set of recipients of a reply, after starting to compose the reply, but before sending it.  (For that matter, it's not unusual for the recipient to want to change the set of reply recipients after sending the reply, but Internet Mail wasn't designed to deal with that.)

6. The best address to which to send a reply is not something that can reliably be determined from the headers of the subject message.   Only the author of the reply understands the reason for replying to the message, and therefore only the author of the reply understands to whom the reply is best addressed.

There's a natural tension here between the expressed interests of several parties: the author of the subject message, the author of a reply to that subject message, and any lists through which that message might have been sent.  But the thing that makes the most sense, and produces the least surprising behavior, is to always entrust the author of the reply with deciding who should receive the reply.

---

now, having said that, here's what I'd recommend:

1. There should always be "Reply" and "Reply All" buttons, even if the two do the same thing.
2. If there's a List-Post header in the subject message, adding a "Reply List" button makes sense.
3. The recipient of the subject message (i.e. the author of the reply) should be able to change the list of recipients at any time prior to sending the reply, including being able to toggle between Reply/ReplyAll/ReplyList, adding additional recipients, and removing others.

Note: The Reply-To field just confuses things because it doesn't really allow the sender to specify his preferences as to where replies should go; it only takes the place of From.   But that behavior is so well-entrenched and widely expected that you probably shouldn't change it.   So when the recipient hits "Reply" or "ReplyAll" to a message for which a Reply-To field is present, you should use the address(es) in the Reply-To field instead of those in the From field.   But you should still allow the recipient the option of replying to the address(es) in the From field, even when the Reply-To field is present.

The best way I've thought of to do this is for the compose window to present filled-in message headers listing all of the possible recipients, with the currently selected recipients in black and the others grayed out.   So if the recipient hits "Reply" then the only addresses initially shown in black are those from the Reply-To field (or the From field if no Reply-To field was present).   But any address could be toggled from black-to-gray or vice versa by clicking on it.   And the Reply / ReplyAll / ReplyList buttons should still be present in the compose window of the reply, so that the recipient can easily reset the set of active recipients to any of those defaults.

Keith

On Jan 4, 2013, at 11:44 AM, Jan Kundrát wrote:

> _______________________________________________
> ietf-822 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-822

---

From: Ned Freed <[email-masked]>
Date: Fri, 04 Jan 2013 22:44:11 -0800 (PST)

> (oh wow.  here we go again.)

Indeed.

> 1. I disagree with the RFCs about this, and always have.   (I state this up
> front just to make it clear that I am not proposing a novel way of interpreting
> the RFCs; I'm just saying that I disagree with them.   So there's no need to
> try to reconcile my view with what the RFCs say - we're just diametrically
> opposed.)

While I agree with almost all of your subsequent points, I'm not convinced
this constitutes a position that is "diametrically opposed" to what the RFCs
say.

> 2. Regardless of the presence or absence of Reply-To or any other field, it
> is ALWAYS the recipient's job to decide to whom he wants to address replies.

Quite correct. In the case of replies user intent is very complicated and
at best can be approximated by a small set of UA settings.

> 3. It is the job of the recipient's mail user agent ("agent" = acting on the
> recipient's behalf) to obey the recipient's wishes.   In particular, (a) it
> should allow the recipient to reply to whomever he wishes to reply and (b) make
> that behavior as obvious to the recipient as possible, so that the MUA's
> behavior will be unsurprising to the recipient of the subject message (the
> author of the reply).

That summarizes the requirements I personally have for a UA in this regard very
nicely.

> (Note that "to whomever he wishes to reply" might include addresses from any
> of the headers of the subject message, and/or addresses  that don't appear in
> the subject message at all.)

> 4. UAs that change their behavior based on the presence or absence of fields
> in the subject message (especially fields that aren't normally displayed)
> generally FAIL to meet the "unsurprising behavior" test.

I'll also note that it's one thing to pull an address out of a field that's
displayed by default. At least in that case the user knows where it came from,
and has a little context as to what it means. It's quite another to pull it out
of field that isn't displayed. This can be a real surprise. 

And since most users agents don't allow the set of fields displayed by default
to be altered, the worst cases of this problem can't even be ameliorated.

> 5. It's not unusual for a recipient to want to change the set of recipients
> of a reply, after starting to compose the reply, but before sending it.  (For
> that matter, it's not unusual for the recipient to want to change the set of
> reply recipients after sending the reply, but Internet Mail wasn't designed to
> deal with that.)

I'm not sure how popular it is, but I personally do it all the time.

> 6. The best address to which to send a reply is not something that can
reliably be determined from the headers of the subject message.   Only the
author of the reply understands the reason for replying to the message, and
therefore only the author of the reply understands to whom the reply is best
addressed.

As I noted, this is why your second point is so important.

> There's a natural tension here between the expressed interests of several
> parties: the author of the subject message, the author of a reply to that
> subject message, and any lists through which that message might have been sent. 
> But the thing that makes the most sense, and produces the least surprising
> behavior, is to always entrust the author of the reply with deciding who should
> receive the reply.

Agreed.

> 1. There should always be "Reply" and "Reply All" buttons, even if the two do

Agreed.

> List" button makes sense.

I'll also note that it is important that the field be checked prior to offering
the option. The choice of a URL for the list-post field  syntax along with
providing essentially no guidelines as to its use is a bit unfortunate.

> 3. The recipient of the subject message (i.e. the author of the reply) should
> be able to change the list of recipients at any time prior to sending the
> reply, including being able to toggle between Reply/ReplyAll/ReplyList, adding
> additional recipients, and removing others.

An ancillary question that needs to be answered here is whether or not changes
made at this point should affect other header elements. I'm inclined to think
the answer is "no", but I'll note that there are potential security issues when
addresses that would normally be included in the reply are removed but message
ids associated with those addresses remain.

> Note: The Reply-To field just confuses things because it doesn't really allow
> the sender to specify his preferences as to where replies should go; it only
> takes the place of From.   But that behavior is so well-entrenched and widely
> expected that you probably shouldn't change it.   So when the recipient hits
> "Reply" or "ReplyAll" to a message for which a Reply-To field is present, you
> should use the address(es) in the Reply-To field instead of those in the From
> field.   But you should still allow the recipient the option of replying to the
> address(es) in the From field, even when the Reply-To field is present.

My personal preference is actually to ignore reply-to when doing a replyall,
but I recognize that's a distinctly minority taste.

> The best way I've thought of to do this is for the compose window to present
> filled-in message headers listing all of the possible recipients, with the
> currently selected recipients in black and the others grayed out.   So if the
> recipient hits "Reply" then the only addresses initially shown in black are
> those from the Reply-To field (or the From field if no Reply-To field was
> present).   But any address could be toggled from black-to-gray or vice versa
> by clicking on it.   And the Reply / ReplyAll / ReplyList buttons should still
> be present in the compose window of the reply, so that the recipient can easily
> reset the set of active recipients to any of those defaults.

I like that idea.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Mon, 07 Jan 2013 20:22:27 -0800

This note follows the other comments you've already received. I think my 
comments are compatible with the ones you've already received.

Some inline comments, and then more summary ones:

On 1/4/2013 8:44 AM, Jan Kundrát wrote:
> 1) If there's a List-Post header and its value is not set to "NO"
...
>   - All addresses in the message's From and Reply-To will be used in the

Just to note this specific point: The specific language in RFC 5322 is 
somewhat milder than the original design of Reply-to and is now:

      "When the "Reply-To:" field is present, it
    indicates the address(es) to which the author of the message suggests
    that replies be sent."

Reply-to was invented to give the orignal author an explicit means for 
specifying a /replacement/ to the From address, for replies, not for 
augmenting it.

So your proposed use of /both/ Reply-To and From is not strictly a 
protocol violation, but runs counter to the intent of the author who 
created the Reply-To. In fact, that's also the intent of mailing lists 
that add a Reply-To.

At that, I think the wording from RFC733 onward is not nearly as crisp 
and complete as it should have been, to make this clear.  Sigh.

> when "Reply List" is not available).

I don't quite understand what this last sentence statement means.

> 3) "Private Reply" option is always available and produces a message
> with the following recipient(s) in the "To" field:
>
>   - If the message contains a Reply-To, each of those which are at the

"each of those" ??  each of those what? Do you mean that any address is 
the Reply-to which is not also listed in the List-Post is used.

> same time *not* listed in any List-Post addresses are used. If the
> resulting set is non-empty, the From header is ignored. This means that
> any address in the Reply-To which is also listed in the List-Post is
> silenty ignored and anything not in the List-Post is used.

It appears that you intend Private Reply to mean what Reply originally 
meant:  Send this reply /only/ to the author.

The important semantic enhancement being attempted is to use information 
in List-Post to make it more likely that the reply really won't go to a 
group.  It's an interesting idea; I've no idea whether it will succeed.

Now for extended comments:

The topic really is user interface design, not protocol design.  Hence, 
the knowledge, intent, psychology and, ultimately, expectations of the 
author and the recipient are the controlling factors.  The message 
generated for the recipient should naturally reflect the recipients 
/and/ the original author's intent.

The cited Principle of Least Astonishment is a usefully concise 
reference to the question of expectation, which arguably synthesizes the 
other human factors.  One of the problems with citing the Principle is 
the implication that expectations can be readily or completely discerned.

By contrast, the rule I've observed among skilled UI designers is that 
the application of the best cognitive, usability, etc. modeling, for 
doing UI, merely produces a good /start/ and never is sufficient to 
produce a final result.  For any interesting innovation in the UI, it's 
necessary to experiment with real usage and modify as experience is gained.

Two natural assumptions drive many people's thinking about UI design:

    1)  Users always benefit from additional information

    2)  Users always benefit from having more control

For interactive interface (UI) design, both of these assumptions are 
more often wrong than right.  Too much information or being forced to 
make too many decisions both create an excessive cognitive burden.

The challenge are to get the right /balance/ of the right information 
and at the right time.  Users don't pay attention to all information 
equally or even consistently.  What I might notice today, I might not 
tomorrow.

And users essentially never do the kind of diligent address field review 
that some of the notes on this thread seems to expect.

So, your ruleset has features that are unusual and somewhat 
complicated.  In spite of the design appearing to be entirely 
reasonable, I think it highly unlikely that an average user will 
understand it or remember it.  Hence, the complexity is almost certain 
to result in... Astonishment.

One aspect of the design that's potentially interesting is to have 
interactive modification of address lists by selecting addresses to 
remove from a candidate list.  Current interfaces only allow that action 
in a kind of character-oriented text editing mode.  It sounds as if you 
want something like menu selection.

d/
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Tue, 08 Jan 2013 21:25:54 -0500

> meant:  Send this reply /only/ to the author.

Indeed, it's useful to understand that back in ancient history, "Reply" 
was assumed to only go to the author, and this helps to explain why the 
"Reply-To" field seems to have a peculiar meaning. But user agents 
capable of some variant of "reply all" have been around for a very long 
time.  (since the early 1980s, IIRC)

> /and/ the original author's intent.

The problem is, of course, that the author's intent and the recipient's 
intent may not be the same.  But since the author cannot reliably 
anticipate the nature of a recipient's response, and also since the 
recipient responding is responsible for any replies that he sends, 
clearly the recipient who is composing the reply must be the one to 
ultimately choose the recipient(s) of a reply.

> reference to the question of expectation, which arguably synthesizes 
> the other human factors.  One of the problems with citing the 
> Principle is the implication that expectations can be readily or 
> completely discerned.
>
> produce a final result.  For any interesting innovation in the UI, 
> it's necessary to experiment with real usage and modify as experience 
> is gained.

I absolutely agree with this.

> And users essentially never do the kind of diligent address field 
> review that some of the notes on this thread seems to expect.

Users do, on the other hand, suffer embarrassment or worse (sometimes 
much worse) when their replies go to a different set of recipients than 
they expected or intended.

To the extent that users "essentially never" review addresses in 
replies, this might in part be because most user agents aren't designed 
to facilitate effective review of such addresses.   If they let you edit 
reply recipients at all, it's usually via some "character-oriented text 
editing mode"  that seems cumbersome and antiquated in comparison to the 
way the GUI facilities specification of recipients of an original 
message: with name lookup and completion (for better or worse, often 
automagically populated from previously received messages) and automatic 
filling-in of the address in proper Internet Mail syntax.   With the 
"character oriented text editing mode" of editing reply addresses, you 
typically have to be responsible for getting the syntax right yourself.  
If ordinary users don't make use of this facility very often, it's 
little wonder.

Also, it seems to be current fashion in UA design to abbreviate the list 
of recipients of a message, hide their addresses, etc. as if that 
information were somehow superfluous.   This makes it even more 
difficult for a recipient to see to whom his reply will go.

(Those of us who remember paper memos also remember that it could be 
very important to read the listed set of recipients of a memo in order 
to understand the context of a message, and to choose carefully to whom 
you'd send a reply.   This is in principle no less true for email, 
though it's possible that poor MUA UI design has encouraged email users 
to be careless.)

Keith