Subject: [EAI] [eai] #11: Capture the scenario of "SHOULD not use UTF-8 in Message-ID" for future advice draft

From: "eai issue tracker" <[email-masked]>
Date: Mon, 19 Sep 2011 14:58:02 -0000

#11: Capture the scenario of "SHOULD not use UTF-8 in Message-ID" for future
advice draft

 The "SHOULD NOT use UTF-8" could be used in advice doc explaining the
 advantage of being more conservative

 --
 "Implementers of message-id generation algorithms may prefer to restrain
 their output to ASCII since that has a few minor advantages, such as when
 constructing References fields in mailing-list threads where some senders
 use EAI and others not."

 suggestion from Arnt Gulbrandsen

 ---
 Just in case we put a text like this in (I don't mind one way or another),
 please remove "minor".

 suggestion from Martin Durst

-- 
-----------------------------------+----------------------------------------
 Reporter:  jyee@…                 |       Owner:     
     Type:  enhancement            |      Status:  new
 Priority:  minor                  |   Milestone:     
Component:  scenarios              |     Version:     
 Severity:  Candidate WG Document  |    Keywords:     
-----------------------------------+----------------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/eai/trac/ticket/11>
eai <http://tools.ietf.org/wg/eai/>

---

From: [email-masked]
Date: Mon, 19 Sep 2011 07:58:11 -0700 (PDT)

A slightly different version of this text has already been added to 5335bis
(and noted in the change list). However, I used a MAY, not a SHOULD.

				Ned

> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 19 Sep 2011 10:25:01 -0700

>>   constructing References fields in mailing-list threads where some senders

I looked over the draft, and this language made me very uncomfortable.

The reason is that -- once again -- it has this draft trying to juggle a hybrid 
(UTF-8/ASCII-only) world, rather than a much simpler UTF-8 only world.  Indeed, 
the language that results looks awkward.

A simple question is why this juggling is appropriate here but not in the rest 
of the core drafts, including this one?

For the situations that need this special treatment, why do they /not/ also need 
it for all other occurrences of UTF-8.  And in answering that question, consider 
whether those are edge conditions or likely to be typical.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: [email-masked]
Date: Mon, 19 Sep 2011 11:06:52 -0700 (PDT)

> I looked over the draft, and this language made me very uncomfortable.

I don't see why. It's a fact that using 8-bit utf-8 in message-ids (1) isn't
really necessary and (2) creates a situation where replies are forced to be EAI
messages when otherwise they may not have needed to be. Saying that  generators
of such fields need to be aware of the consequences seems completely reasonable
to me, and it also aligns quite well with the nearby sections on line limits,
encoded words, and encoding restrictions, all of which have very much the same
overall character as this.

> the language that results looks awkward.

No, it really isn't doing that.

> of the core drafts, including this one?

Since the juggling you refer to is in fact nonexistent, this question is 
unanswerable.

> it for all other occurrences of UTF-8.

Because they aren't comparable cases. Again, there is no real need to generate
utf-8 message-ids - these are fields for automatic processing, not presentation
to the user. The same cannot be said for addresses, subject fields, etc. etc.
And yes, there are other automatically processed fields where we're now
allowing 8-bit utf-8, but AFAIK none of them have the additional cascading
effect message-ids have. (And if someone can identify additional cases they
should also be noted in this specification.)

In short, this is a special case, one that arises from a combination of
factors, and for that reason it's worth noting.

> And in answering that question, consider

The message-id issue is most certainly going to create issues for clients.

				Ned

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 19 Sep 2011 20:38:20 +0200

On 19 September 2011 19:25, Dave CROCKER wrote:

> A simple question is why this juggling is appropriate here but
> not in the rest of the core drafts, including this one?

The technical problems of "folks want to use local parts in their
scripts and languages" vs. "messages need a unique part to form a
globally unique message ID" are _technically_ unrelated.

That unique@ and local@ are _historically_ almost the same syntax
has no _technical_ reason I'm aware of.  Besides you would know it
better.  For the purposes of NetNews message IDs are as important
as say IP for the purposes of TCP.

That's as simple as 1-2-3.  Don't waste time with MAYs or SHOULDs,
anything below a MUST is "broken by design", details don't matter,
or make it worse.

-Frank

---

From: [email-masked]
Date: Mon, 19 Sep 2011 11:40:05 -0700 (PDT)

> globally unique message ID" are _technically_ unrelated.

Exactly. The syntax may be similar but the semantics are very, very different.

> has no _technical_ reason I'm aware of.

The reason is pretty simple: Parsimony. Why invent additional syntax
when you don't need it?

> Besides you would know it

> anything below a MUST is "broken by design", details don't matter,

That's what the previous versions did but the consensus call went against you
on that. Deal with it.

				Ned

---

From: Joseph Yee <[email-masked]>
Date: Mon, 19 Sep 2011 15:05:06 -0400

> anything below a MUST is "broken by design", details don't matter,

Hi Frank,

The recent WG rough consensus agreed to allow UTF-8 in Message-ID.  

When we look into EAI advice draft in future, would you be interest to write about advice dealing with NNTP, or better (IMHO), look into -bis or extension RFC for NNTP in a new WG/BOF?

Thanks
Joseph, co-chair of EAI

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 19 Sep 2011 21:49:54 +0200

On 19 September 2011 21:05, Joseph Yee wrote:

> would you be interest to write about advice dealing with NNTP,
> or better (IMHO), look into -bis or extension RFC for NNTP in
> a new WG/BOF?

Well, "somebody" would have to fix RFC 5538 section 6 at some
point in time, or submit a "post facto erratum", if that is the
correct Latin for this nit.  Generally I don't expect any new
NNTP efforts before say 2020, so that is no big problem at the
moment.  Gateway operators are used to the idea that they have
to do something with Message-IDs, and while UTF-8 makes this
job more interesting it won't be completely different.  Maybe
they'll simply reject message/global.

For a new USEFOR effort it's similar, the last round took more
than one decade; folks remotely interested in this topic are
likely not tempted to revisit it in this decade.  In theory I
should revive my old SPF EAI draft, but with UTF-8 message IDs
there are greener pastures from my POV, FNV hashes, 4408bis,
UTF-4, SCRAM-MD5, or some non-IETF stuff I'm interested in.

For EAI-2011 I first want to see if it gets traction before I
care about the consequences (excl. RFC 5538 and mailto-bis).

Kind regards,
 Frank

---

From: "John Levine" <[email-masked]>
Date: 19 Sep 2011 20:56:25 -0000

> Gateway operators are used to the idea that they have to do
> something with Message-IDs, and while UTF-8 makes this job more
> interesting it won't be completely different.

Practical Mail->News gateways have to do all sorts of gross stuff due
to news systems enforcing syntax rules more strictly than mail
systems.  Munging message IDs would not be any worse than what they do
now.

Although I understand the rationale for wanting ASCII message IDs in
mail to make non-EAI replies easier, I'm having trouble imagining
plausible scenarios in which it would make a difference.

Scenario 1: A sends B an EAI message.  B replies, and somehow manages
to leave out all the elements that made A's message need EAI.  This
suggests that A and B both have ASCII addresses, A's message was cc'ed
to EAI recipient C who B leaves out of the reply, or something like
that.  Yes, it could happen, but it doesn't impress me as likely
enough to be a design point.

Scenario 2: A sends B an EAI message, something in the path uses local
knowledge to downgrade the message to ASCII.  But if it's going to
rewrite all the other headers, it can rewrite the message-ID, too.

So I'm OK with it being a MAY, but it doesn't seem worth much effort.
It's also one of those things where no matter what we say, MUAs and
MTAs will make them UTF-8 anyway.

R's,
John

---

From: [email-masked]
Date: Mon, 19 Sep 2011 14:01:00 -0700 (PDT)

> enough to be a design point.

I somewhat agree and somewhat disagree. Replies to subsets of recipients are
very common. And my suspicion is that initially at least EAI usage will mostly
be in the extended address area. So I think sort of thing will come up a fair
bit.

OTOH, since the original message sent to these ASCII address recipients was EAI
and for good reason, what's the real value in being able to send a non-EAI
reply? For this to be useful there has to be some sort of routing difference
between B and A as senders involving non-EAI-capable systems. This, I think is
what makes this case obscure and not really worth worrying about.

> rewrite all the other headers, it can rewrite the message-ID, too.

Maybe, maybe not. It depends on whether or not it's an algorithmic rewrite or
not. Having to keep a mapping database is expensive.

> MTAs will make them UTF-8 anyway.

Agreed on both points.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Mon, 19 Sep 2011 17:52:47 -0400

Folks,

Can we avoid looping back through this entire discussion again?

While I happen to agree with Ned's explanation of the scenarios,
the important thing is that we have been through them before.  

We know that there are some edge-case scenarios in which replies
to messages that require EAI facilities only because UTF-8 can
end up in slightly funny states   We know that gateways to other
environments (netnews excluded but by no means exclusively) tend
to exhibit more sensitivity to changes than would occur in a
more perfect world.  Trying to base normative text on how often
those cases will arise and how much pain they will cause --
beyond knowing that the values are non-zero -- requires ability
to predict the future with good accuracy, which is a skill I
haven't observed in the WG or the IETF generally.

Joseph has announced consensus around permitting UTF-8 in
message headers.  It seems to me that Ned's text reflects that
consensus, including pointing out that systems may want to pay
special attention to those edge cases.

Unless people think this is important enough to (further) stall
progress and have new things to add to the prior case analyses
and positions, can we move on?

   john

--On Monday, September 19, 2011 14:01 -0700 [email-masked]
wrote:

>> > something with Message-IDs, and while UTF-8 makes this job
>> > more interesting it won't be completely different.
> 
>> Practical Mail->News gateways have to do all sorts of gross
>> stuff due to news systems enforcing syntax rules more
>> strictly than mail systems.  Munging message IDs would not be
>> any worse than what they do now.
>...