Subject: Re: [EAI] Netnews

From: Shawn Steele <[email-masked]>
Date: Wed, 27 Jan 2010 23:54:50 +0000

I think netnews is it's own protocol, since it's news, not mail, though I agree that the concept shouldn't be too hard.

I'm curious how a 2009 RFC ended up not clearly preferring Unicode, particularly when the RFC it obsoletes hasn't changed for 12 years....

-Shawn

----------------------------------------------------------------------

Message: 1
Date: Wed, 27 Jan 2010 10:24:36 -0000
From: "Charles Lindsey" <[email-masked]>
Subject: Re: [EAI] draft of the new charter for EAI WG
To: IMA <[email-masked]>
Message-ID: <[email-masked]>
Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-1

On Tue, 26 Jan 2010 09:33:08 -0000, YAO Jiankang <[email-masked]> wrote:

> Dear all,
>
>      The draft of new charter is proposed below. We can finalize the 
> new charter based on the following text.
>
>     This message is sent with permission from co-chairs.
>
>     any comments are welcome.

Can we have as an additional item extending EAI to Netnews? That needs to be done, and whether it is done by this group of by the USEFOR WG is up for discussion (Harald may be able to advise). Essentially, it is just a matter of defining an entension to the Newsgroups header (some work on this was done earlier on by USEFOR). Everything else in EAI should work out of the box, and downgrading would only be required for gateways to other media such as email.

--
Charles?H.?Lindsey?

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Jan 2010 12:51:06 -0000

On Wed, 27 Jan 2010 23:54:50 -0000, Shawn Steele  
<[email-masked]> wrote:

> I think netnews is it's own protocol, since it's news, not mail, though  
> I agree that the concept shouldn't be too hard.
>
> I'm curious how a 2009 RFC ended up not clearly preferring Unicode,  
> particularly when the RFC it obsoletes hasn't changed for 12 years....

USEFOR got bogged down in internal wrangling. Initially, it wanted to make  
everything in UTF-8 . But then it got bogged down in internal wrangling,  
and that bit got relegated to "future work". The remnant of USEFOR is now  
probably too small to get much further.

The technique has already been demonstrated to work on the present network  
and there already exists an experimental group with a utf-8 name -  
dk.test.utf8-Ã¦Ã¸Ã¥ (which is actually utf8 although I wrote it in  
8859-1). It is available on news.dotsrc.org, if not elsewhere.

As I said, little needs to be done, apart from the Newsgroups header  
(which will need some severe normalization of Newsgroup names to avoid  
variations in spelling).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: John C Klensin <[email-masked]>
Date: Thu, 28 Jan 2010 11:12:35 -0500

--On Thursday, January 28, 2010 12:51 +0000 Charles Lindsey
<[email-masked]> wrote:

>> I think netnews is it's own protocol, since it's news, not
>> mail, though   I agree that the concept shouldn't be too hard.
>> 
>> I'm curious how a 2009 RFC ended up not clearly preferring
>> Unicode,   particularly when the RFC it obsoletes hasn't
>> changed for 12 years....
> 
> USEFOR got bogged down in internal wrangling. Initially, it
> wanted to make everything in UTF-8 . But then it got bogged
> down in internal wrangling, and that bit got relegated to
> "future work". The remnant of USEFOR is now probably too small
> to get much further.
> 
> The technique has already been demonstrated to work on the
> present network and there already exists an experimental group
> with a utf-8 name - dk.test.utf8-Ã¦Ã¸Ã¥ (which is
> actually utf8 although I wrote it in 8859-1). It is available
> on news.dotsrc.org, if not elsewhere.
> 
> As I said, little needs to be done, apart from the Newsgroups
> header (which will need some severe normalization of Newsgroup
> names to avoid variations in spelling).

Charles,

While this is tempting for all of the reasons you mention, each
time we start digging into the details from one side or the
other, we discover that netnews is not email and that email is
not netnews.  That gets us into trouble.  

>From my point of view, part of what has sometimes gotten EAI
into trouble is a lack of deep understand of email issues by
some of the participants who are much more concerned with
internationalization.  Because the number of active WG
participants who understand netnews well is even lower, adding
it to the charter as a priority task strikes me as an invitation
to either delays (as we all familiarize ourselves with the
issues) or to bad results.

In addition, almost every time we have had a group that tried to
address both netnews and email, we have ended up with conflicts
about which one should take priority, e.g., whether one or the
other should be changed to make the other one more efficient or
easier.

The EAI task is hard enough, important enough, and potentially
problematic enough wrt the deployed email infrastructure that I
just don't see making non-critical-path tasks part of the effort
as desirable.   Conversely, if the addons for netnews really
turn out to be as simple and obvious as your notes imply, then
they should be an appropriate matter for a short, individual
submission, document once EAI finishes its work.

Sorry.

     john

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Jan 2010 19:19:54 -0000

> other should be changed to make the other one more efficient or

Indeed. that is why people got cold feet when USEFOR wanted to di it years  
ago. The problems of gatewaying to email is what sank it, and convenient  
gatewaying between email and news is an important need (a lot off it goes  
on, which is why they sometimes need to be looked at together). But now  
that email is getting into a position where it can propagagte such things,  
that argument is gone.
> submission, document once EAI finishes its work.

OK, that makes sense. But if such a submission is upcoming, do you want to  
see it here? I will raise the matter on that dk.* group, and see who bites.

---

From: John C Klensin <[email-masked]>
Date: Thu, 28 Jan 2010 16:08:45 -0500

--On Thursday, January 28, 2010 19:19 +0000 Charles Lindsey
<[email-masked]> wrote:

>...
> OK, that makes sense. But if such a submission is upcoming, do
> you want to see it here? I will raise the matter on that dk.*
> group, and see who bites.

My personal opinion on that is, I think, similar to Joseph's.
Happy to see it on this list _after_ the key EAI documents are
finished and at least on their way to IESG.  Earlier than that
would only be a distraction.   And I would hope that any
document of that nature would be reviewed by at least a few
people, other than yourself, who are thoroughly familiar with
Netnews, Email, and i18n issues.  That kind of expert review may
be more important than review by a larger, but less educated,
group.

    john

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Jan 2010 14:16:54 -0000

On Thu, 28 Jan 2010 18:48:39 -0000, Julien ÉLIE <[email-masked]>  
wrote:

I gather Julien still has problems posting to this list. But it is nice to  
have at least one more Netnews expert on this thread.

> Hi Charles,
>
>> The technique has already been demonstrated to work on the present  
>> network  and there already exists an experimental group with a utf-8  
>> name - dk.test.utf8-Ã¦Ã¸Ã¥ (which is actually utf8 although I wrote it  
>> in  8859-1).
>
> If the newsgroup has been made moderated, I doubt it would have worked  
> fine.
> The submission address would have been  
> <dk-test-utf8-Ã¦Ã¸Ã¥@moderators.isc.org>
> and INN "sanitizes" everything when mail is concerned...
>
> For instance innmail:
>
> # Fix up any addresses.
> foreach ( @ARGV ) {
>     s![^-a-zA-Z0-9+_.@%]!!g ;
> }

Insofar as INN is an email agent, it really ought to be brought up-to-date  
by incporporating EAI. It certainly propagates that group fine within  
Usenet.

---

From: Julien ÉLIE <[email-masked]>
Date: Fri, 29 Jan 2010 19:28:45 +0100

Hi Charles,

> Insofar as INN is an email agent, it really ought to be brought up-to-date  by 
> incporporating EAI. It certainly propagates that group fine within  Usenet.

Yes, propagation is fine.

Incidentally, maybe it would also be useful to define internationalized
distributions (for the Distributions: header field).
We are currently limited to ASCII...
As byte-string comparisons are usually done, I believe UTF-8 could also
be used for them.

-- 
Julien ÉLIE

« -- Ouvre l'oil, et le bon !
  -- L'autre, je peux pas encore l'ouvrir, je risque pas de me tromper ! » (Astérix)

---

From: Julien ÉLIE <[email-masked]>
Date: Fri, 29 Jan 2010 19:15:55 +0100

Hi John,

> not netnews.  That gets us into trouble.

Email and netnews are linked in some aspects.

> as desirable.

I understand that.

> would only be a distraction.

But what if some of the things EAI come up with are difficult to
integrate in netnews?

News articles use for instance From:, Sender: and Approved: header
fields.  They will contain internationalized email addresses and
it is still unsure whether netnews will cope up with that.
(Would downgrade also be possible for them?)

Of course I hope everything will go fine and there will be no
problem for netnews.  But better be sure and do not lose from
sight that netnews also uses a few email fields.

-- 
Julien ÉLIE

« Vous savez, les idées, elles sont dans l'air. Il suffit que quelqu'un
  vous en parle de trop près, pour que vous les attrapiez ! » (Raymond Devos)

---

From: John C Klensin <[email-masked]>
Date: Fri, 29 Jan 2010 17:04:19 -0500

> As byte-string comparisons are usually done, I believe UTF-8

This is exactly why I'm resistant to trying to do the Netnews
work using the EAI list.  While the synergies are good, the
distractions aren't.

In this specific case, please do not just assume that, if a
string comparison works for ASCII, it will be sufficient for
UTF-8.  There are normalization issues (whether to normalize
and/or case fold before comparing; which method to use) that
need to be examined very carefully to prevent all sorts of nasty
surprises.  It may be more important to make a decision than
what that decision actually is, but it has to be sufficiently
well-documented that responsibilities are understood and
behavior is predictable.

      john