Subject: Re: [I18nrp] charter -01

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 15 Aug 2018 14:42:34 +0000

(Sorry, I just joined this list, so I'm responding to a message from the list archive and not one that I received directly.)

> Internationalization Directorate (i18ndir) Charter

"Internationalization" sometimes drifts into areas that I would describe as "localization". That is, use in some local-only context. Does anyone else see this distinction, and if so, is it something to be concerned about?

Scott

---

From: Nico Williams <[email-masked]>
Date: Wed, 15 Aug 2018 11:36:10 -0500

On Wed, Aug 15, 2018 at 02:42:34PM +0000, Hollenbeck, Scott wrote:
> "Internationalization" sometimes drifts into areas that I would
> describe as "localization". That is, use in some local-only context.
> Does anyone else see this distinction, and if so, is it something to
> be concerned about?

At Sun we used to speak of I18N, L10N (localization), and G11N
(globalization).

I18N = everything to do with Unicode and other charsets/codesets,
       including whatever on-the-wire negotiation thereof

L10N = everything to do with localization, including how to express
       language choices, locales, and so on, but also including
       on-the-wire language/locale negotiations

G11N = I18N + L10N

For the IETF most of L10N is of limited relevance, in that Internet
protocols that carry localized/localizable strings need to support
language negotiation, but locales are almost right out because we mostly
want to only use Unicode (UTF-8) on the wire.  Except that locales are
in scope for protocols like SSHv2, where data in arbitrary codesets will
be exchanged (even though messages for users from the server are always
in UTF-8) because the protocol carries arbitrary data in "channels".

Because of this we could get away with using I18N to mean G11N.

But still, the fact is that SSHv2's language negotiation is insufficient
for what users need.  Users work around this by sending POSIX LANG/LC_*
environment variable settings and hope the server has matching locales.
This state of affairs is due to the IETF not having a notion of locale,
which in part is also due to shortcomings / non-universality of POSIX
locales (how would one express a locale interoperably between Windows
and POSIX-ish systems?).  In principle this can cause significant pain,
but in actuality it's mostly a non-problem -- a theoretical problem that
mostly only bothers _me_ :)  We're probably not going to remediate this
because SSHv2 is pretty much the only Internet protocol that needs this,
and it is done (SECSH WG has concluded), and no one is complaining
enough to bother fixing it.

Nico
--

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 15 Aug 2018 20:44:52 +0200

> "Internationalization" sometimes drifts into areas that I would describe as "localization". That is, use in some local-only context. Does anyone else see this distinction, and if so, is it somethin…

My view: not to be concerned about but to remember. To me Internationalization implies "doing things without context" while Localization implies a local context. Sure, there is a gray area in between, and a line to be drawn somewhere. If the group is Internationalization, the border is discovered by "walking towards localization" and stopping when context is needed, while Localization walk "from the other direction". Given this view (which I might not share with anyone) of course the boundaries might be found on different places, but that does not worry me.

What for me is important is that when looking at Internationalization one explicitly look at problems in general without context. A good focus as I find it being hard to do both Internationalization and Localization at the same time.

So no, I am not worried. But important to talk about.

   Patrik

---

From: Asmus Freytag <[email-masked]>
Date: Wed, 15 Aug 2018 14:16:37 -0700

> "Internationalization" sometimes drifts into areas that I would describe as "localization". That is, use in some local-only context. Does anyone else see this distinction, and if so, is it somethin…

Internationalization provides the architecture that makes localization 
possible.

Something like a locale-data repository may contain local data, but 
providing it in a central repository makes it part of an overall i18n 
effort, I would say.

A recommendation about IDN policies for a particular script may appear 
to be providing localization. But I would view it as part of a larger 
deliverable, that is recommendations for IDN policies in general, which 
to me is par tof an i18n effort, just like a locale data repository would.

You may find that following that perspective most specifications are 
more i18n whereas localization is more a concern for specific 
implementations. Your mileage may vary,

A./

---

From: "Jiankang Yao" <[email-masked]>
Date: Thu, 16 Aug 2018 09:43:43 +0800

From: Hollenbeck, Scott
Date: 2018-08-15 22:42
To: [email-masked]'
Subject: Re: [I18nrp] charter -01

>"Internationalization" sometimes drifts into areas that I would describe as "localization". That is, use in some local-only context. Does anyone else see this distinction,
>

Thanks for pointing it out.
Actually, I share some similar ideas with you when I did some research in idn, eai and Internationalization many years ago.

I am a Chinese speaker. If we talk about Internationalization in general in our culture, not in IETF scope, Internationalization means to use English/latin character to reach outside China, localization is to mean how to use Chinese Character to adapt to the local needs.

For Non-English speaker, Internationalization is to reach the English speaker, localization is to reach the local people of non-English.
For English speaker, Internationalization is to reach the non-English speaker, localization is to reach the local people of English.
So I am sure that the wording "Internationalization " in IETF is invented by some native English IETF expert.

Back to the wording of Internationalization in IETF, my understanding is that Internationalization is how to use Unicode which includes almost all local characters; localization is how to use the local encoding such as GB2312.

Yes, Internationalization  sometimes includes  some local-only context. For an example, in IDN, we sometimes need to consider Chinese character (simplified one, tradtional one), Arabic character(right to left); these characters may need to have some special rules to deal to be adapt to the local needs.

>
> and if so, is it something to be concerned about?
>

We need to refine the definition of Internationalization and loccalization in IETF.

Best Regards
Jiankang Yao

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Thu, 16 Aug 2018 14:23:24 +0000

From: Jiankang Yao <[email-masked]>
Sent: Wednesday, August 15, 2018 9:44 PM
To: Hollenbeck, Scott <[email-masked]>; i18nrp <[email-masked]>
Subject: [EXTERNAL] Re: Re: [I18nrp] charter -01

From: Hollenbeck, Scott<mailto:shollenbeck=[email-masked]>

Date: 2018-08-15 22:42

To: [email-masked]'<mailto:[email-masked]>

[SAH] Thanks for everyone's feedback. I'd like to suggest that one thing we can do with the charter to make the distinction clearer is to cite the definition of internationalization from Section 2 of RFC 6365:

OLD TEXT
The Internationalization Directorate is a specialized advisory group that assists the Area Directors of the Applications and Real-Time Area with regard to internationalization, in accordance with RFC 2418.

NEW TEXT
The Internationalization Directorate is a specialized advisory group that assists the Area Directors of the Applications and Real-Time Area with regard to internationalization (as defined in Section 2 of RFC 6365), in accordance with RFC 2418.

Scott

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Aug 2018 10:35:18 -0400

Hi.  I think it is time to return to, and think about, the
purpose of this effort and see where that leaves us.

I think I can safely claim to be one of the small number of
people who have been doing a disproportionate share of the work
in/for the IETF in this loosely-defined area in the last
quarter-century [1]. I've gotten frustrated enough by the
relatively recent inability to have useful and constructive
discussions about and to process documents that I initiated the
discussions that led to the BOF.  I decided to be fairly quiet
during the BOF and to remain silent so far in the charter
discussion to see where the list would take things.  I think it
is time to say something.

At least to me, the problem here is that we haven't been able to
process documents.  I am frankly much more concerned about
i18n-specific documents than I am about protocol specifications
that include i18n-issues or provisions, if only because, if the
foundations are not firm, the references and adaptations cannot
be satisfactory.   In recent years, we have seen that WGs don't
get (and, more important, keep) a critical mass of active expert
participants.  Reviews on IETF Last Calls either get silence or
a horrible S/N ratio, neither of which helps convince the IESG
that there is informed community consensus.  We have been
talking about the importance of recruiting more expertise and
activity but, with a few notable individual exceptions, have
made little progress.  In particular, attempted educational
efforts have either fallen flat or been declined entirely.
While we are making little progress in growing expertise and
participation, we manage to have the same disfunctional,
ignorant, and/or irrelevant discussions over and over again and
do so to the point that some of the existing pool of experts
have started asking themselves whether we _really_ have to go
through those issues, and make time-consuming and often
fruitless efforts to explain them, yet again.  If even a few of
those experts answer that question with "no, not another time"
and walk away from the IETF or those issues, the effect is to
further reduce the pool of available expertise [2].

I'm not optimistic about a directorate helping very much, at
least unless it comes with clear IESG agreement about more
status and authority than the typical "review team" and, in
particular, the ability to move work forward that much of the
IETF community does not understand [3], not just catch glitches
or push back on particular efforts.  It would be, IMO,
irresponsible on the part of the IESG to give it that status in
the absence of clear plans about how people are recruited for
(or excluded from) the directorate.  I also have some concerns
about whether a group that is clearly "in" the ART area and
whose function is to "assist[s] the Area Directors of the
Applications and Real-Time Area with regard to
Internationalization" is going to have sufficient leverage with
the IESG in those years in which, by happenstance, none of the
ART ADs are both committed to and comfortable with this work.

However, if the directorate is the best idea we have at this
point, then let's give it a try -- certainly nothing else seems
to be working.

That said, I believe the recent discussion illustrates part of
the underlying problem with internationalization work in the
IETF (and perhaps other IETF work -- maybe this is not a unique
problem).  Charter-01 was posted on the 13th.  We have now had
discussions on two topics -- language that attempted to clarify
scope wrt liaisons as not interfering or changing existing
relationships and responsibilities and questions about what 
"internationalization" really means or includes and what is
excluded.  In the latter case, we have survived for many years
without precise definitions and boundaries.  If we discover that
we need better ones (I'd be surprised), probably the right way
to do it is to put a revision to RFC 6365 on the agenda and see
if we have expertise and energy to write and process it, not to
hold up this charter.  For both issues, if we can't trust the
directorate leadership and relevant ADs to keep the directorate
from going off into the weeds, or if the directorate goes into
protocol lawyer mode about its charter, I suggest that the
directorate idea should be dead in the water because that is
just one of many possible serious failure modes.  A directorate
thus preoccupied is unlikely to be moving documents forward
(again, we have been here before).

Maybe the lack of comments about anything substantive -- such as
how the directorate will be staffed and actually work, how it is
going to recruit and train people, etc. -- means that those
questions can safely be left to the ADs and directorate
membership.  If so, perhaps we are done.   But, unless there are
issues that would affect the directorate plan in significant
ways that I haven't understood yet, let's not spend a lot of
time debating characteristics or t8y (or perhaps even the
boundaries between t8y and i7e, s7y, or i9y [4]).

best,
     john

[1] For those who don't know or don't remember, the effort that
led to MIME, the SMTP extension model, and media types was
initially organized to give us a way to handle non-English,
non-ASCII email without the recipient having to guess what was
going on.  And then there is not only RFC 20 (cited in the
draft) but RFCs 205 and 373 but they predate the IETF.

[2] I observe that, of those who have made significant
contributions in this area by chairing key WGs or authoring/
editing or contributing key text to documents, a non-trivial
proportion seem to have dropped out of the IETF or moved on to
other work in the IETF and largely dropped out of its i18n
efforts. Independent of how one assesses the expertise or
contributions of those people, they were at least exerting
significant effort over extended periods and their departure
doesn't make the expertise pool any larger.

[3] Every time someone stands up in a WG and says something that
might be interpreted as "I don't want to know about all of those
details; just tell me what to do" and is not immediately
escorted away from the microphone and reeducated, it probably
sets up back. It certainly impedes more helpful conversations.
At least we know what to do if similar people write "just use
UTF-8" into their documents.  Those who can identify an issue
with one language they read or speak and either assume that is
the only significant problem or that all other problems can be
extrapolated from it are not much more helpful and might be less
so.

[4] triviality, ignorance, stupidity, irrelevancy...

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Aug 2018 20:03:26 -0400

Jiankang,

Let me add a bit to Scott's response and perhaps start a
discussion that will clarify all of our thinking.   As I
indicated in an earlier note, I don't think these issues need to
be resolved to get to a decent charter for the directorate even
(or especially) if they constitute work items for that
directorate...

--On Thursday, August 16, 2018 09:43 +0800 Jiankang Yao
<[email-masked]> wrote:

>> "Internationalization" sometimes drifts into areas that I
>> would describe as "localization". That is, use in some
>> local-only context. Does anyone else see this distinction,

> Actually, I share some similar ideas with you when I did some
> research in idn, eai and Internationalization many years ago.

While I have departed a bit from them below (and probably
wouldn't try to write them in quite the same way were I doing it
today), you might find reading the definitions of
internationalization and localization in RFC 6365 helpful, at
least to get a different perspective.
 
> I am a Chinese speaker. If we talk about Internationalization
> in general in our culture, not in IETF scope,
> Internationalization means to use English/latin character to
> reach outside China, localization is to mean how to use
> Chinese Character to adapt to the local needs.

How would you describe the use of, e.g., Arabic script
characters in China?   And is use of Latin characters to write
Hanyu Pinyin a localization or an internationalization matter?
Is the distinction between Simplified and Traditional Chinese
characters a localization matter or an internationalization one?
Does the observation that CNNIC has sometimes argued that they
are two separate scripts and that several parties in Japan have
argued that Japanese and Chinese should be considered separate
scripts despite the large number of overlapping characters
change those answers at all (noting that Unicode, which unifies
not only Simplified and Traditional Chinese characters but
CJK[V], may have a different answer).  I'm not disagreeing with
you about practice in China, about which you are far more expert
than I am, just pointing out how slippery these terms become
when one tries to think about things globally... and globally
either all at once or as a sequence of local/global distinctions.

> For Non-English speaker, Internationalization is to reach the
> English speaker, localization is to reach the local people of
> non-English. For English speaker, Internationalization is to
> reach the non-English speaker, localization is to reach the
> local people of English. 

This really bothers me because it seems to imply that having a
Chinese speaker communicate with an Arabic one, a Tamil speaker
interact with a Hindi one, or even communication between a
speaker of French and one of German, are irrelevant and that the
only issues of interest are 
  <some local language> <-> English

I hope that was not what you intended; certainly it is not what
some of the rest of us mean.

> So I am sure that the wording
> "Internationalization " in IETF is invented by some native
> English IETF expert.

It wasn't invented in the IETF at all but adopted (somewhat
reluctantly by some of us) from terminology that started being
used in the character coding community (including Unicode but
not limited to it) long before the IETF became active in this
area.  I don't know if whoever proposed the terminology was a
native Engish speaker or not; there are at least a few likely
other possibilities.

> Back to the wording of Internationalization in IETF, my
> understanding is that Internationalization is how to use
> Unicode which includes almost all local characters;
> localization is how to use the local encoding such as GB2312.

Certainly not how I've used the latter term.  Localization may
well be in Unicode and becomes quite relevant when specific
scripts (and, more important, specific language-script
combinations) require special rendering, case mapping, or other
procedures to make presentation or comparison operations work
well.
 
> Yes, Internationalization  sometimes includes  some local-only
> context. For an example, in IDN, we sometimes need to consider
> Chinese character (simplified one, tradtional one), Arabic
> character(right to left); these characters may need to have
> some special rules to deal to be adapt to the local needs.

See above.

> We need to refine the definition of Internationalization and

I can't speak for either my co-author or the IESG but, if you
think that is useful and have some good ideas where to start, I
think an update to RFC 6365 may be in order.  Again, I don't
think we should hold the directorate charter up while we resolve
those definitional issues.

best,
   john

---

From: "Jiankang Yao" <[email-masked]>
Date: Fri, 17 Aug 2018 09:57:02 +0800

Dear John,

  Thanks a lot.
  Some comments below.

From: John C Klensin
Date: 2018-08-17 08:03
To: yaojk
CC: Hollenbeck, Scott; i18nrp
Subject: Internationalization and Localization (was: Re: [I18nrp] charter -01)

> characters a localization matter or an internationalization one?

Different people from different culture, different area and different country might have different interpretation about internationalization.
The answers might vary depending on different situations.

In the technology field of IETF, if we have a very clear definition of internationalization, we can have a clear answer to the questions raised above.

>   <some local language> <-> English

It proves that internationalization is a complex issue.

> Again, I don't

I agree with you here.

Best Regards
Jiankang Yao

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 17 Aug 2018 15:57:37 -0500

FWIW,

> point, then let's give it a try -- certainly nothing else seems

 I'm not an AD for an area that's likely to be affected a huge amount by
i18n (for instance, TCP + i18n = something like TCP), but speaking as
something of an outsider,

- I couldn't agree more about trying a directorate because that's something
we haven't actually tried before, as best I can tell. I'm not sure I can
remember another suggestion that's been made, that we haven't already tried
at least once.

- I don't know that "the best idea" is the only idea that we should ever
try, but ISTM that if doing a directorate improves some things for i18n,
that is progress, and a future IESG could certainly try another idea to
improve things that the directorate hasn't improved - but again, doing a
directorate is a start.

Spencer

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 28 Aug 2018 12:14:50 -0600

Hi John, thanks for your thoughtful post and my apologies about the
delayed reply.

On 8/16/18 8:35 AM, John C Klensin wrote:
> be satisfactory. 

Your framing of the topic highlights that we have at least two sorts of
work here: strengthening the foundations and, if you will, ensuring
coherence among the full range of IETF specifications that re-use those
foundations.

Different sorts of people might be good at the two sorts of work. More
knowledge and experience is needed to contribute to the foundationalist
efforts, whereas folks with less knowledge and experience can more
easily assist with ongoing coherentist efforts (e.g., reviews and
adaptations).

Whether it's best to attempt both sorts of task in the same team is a
good question. I tend to think it could be fine, as long as we go in
with our eyes open and assign particular tasks based on knowledge and
experience. In addition, doing both sorts of work with the same overall
team gives us more opportunities for learning and mentorship.

> In recent years, we have seen that WGs don't

I don't think we've done the right sort of sustained educational
efforts. In particular, the way that less knowledgeable and experienced
contributors learn on the job is by interacting with and being mentored
by more knowledgeable and experienced contributors. We haven't tried to
form a true team before, just a random collection of individuals (all of
whom are more knowledgeable and experienced - you especially) who once
in a while come down from on high to dispense their wisdom to spec
authors, working groups, and self-selected IETF participants at large
(e.g., through Sunday tutorials at IETF meetings). Although *that* sort
of educational effort hasn't yielded much fruit, I don't think it's the
sort of educational effort we need in order to be successful.

> further reduce the pool of available expertise [2].

Here again, I suggest that we try a different approach by building a
team and mentoring folks who are less knowledgeable and experienced but
who have expressed a willingness to learn.

> or push back on particular efforts. 

Not "just", yes. I believe that by participating in regular reviews and
glitch-catching, and by being involved in the foundation-shoring
efforts, directorate members who are less knowledgeable and experienced
will learn a lot about the topics under consideration. And if we also
document the things that anyone who cares about these topics needs to
learn, we can also do a better job of educating IESG members who might
be new to internationalization.

As to moving forward work that much of the IETF community does not
understand, perhaps the proof is in the pudding. We need to organize,
prioritize, collaborate, and together get some of the foundation-shoring
specs in good enough shape that those who know about these topics feel
they are ready to progress (with, perhaps, more detailed explanatory
writeups than might be generated in other domains). We have several
candidate specs, as mentioned in the run-up to the BoF:

https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/
https://datatracker.ietf.org/doc/draft-freytag-troublesome-characters/
https://datatracker.ietf.org/doc/draft-klensin-idna-5892upd-unicode70/
https://datatracker.ietf.org/doc/draft-klensin-idna-rfc5891bis/

Given our collective failure to move these specs forward, I'd understand
if some IESG members are skeptical and might not want to grant more
status and authority up front than the typical review team or
directorate has. So let's get to work and show them that we deserve it.

> It would be, IMO,
> (or excluded from) the directorate. 

I agree that clear plans are in order, not only for initial recruiting
but also for talent development as well as ongoing management of the
directorate. Do we have good examples of such plans, or can we leverage
best practices, from existing directorates and review teams?

Although I don't want to say "we'll figure it out as we go along", I'm
also leery of placing requirements on this directorate that no past
directorate has had to meet. Setting out some high-level plans seems
appropriate, however.

> I also have some concerns

That's a matter of managing up and educating the ART ADs.

> point, then let's give it a try -- certainly nothing else seems

With a bit more substantive planning with regard to success criteria,
management, recruiting, training, etc. (I can put together a strawman
proposal soon), I think a directorate is indeed worth trying.

Peter