Subject: language tags in PKIFreeText

From: Ronald Tschal�r <[email-masked]>
Date: Sat, 7 Aug 1999 14:18:12 -0700 (PDT)

The definition of PKIFreeText in RFC-2510 says:

     PKIFreeText ::= SEQUENCE SIZE (1..MAX) OF UTF8String
         -- text encoded as UTF-8 String (note:  each UTF8String SHOULD
         -- include an RFC 1766 language tag to indicate the language
         -- of the contained text)

Unfortunately, I could find no hint as to *how* the language tag should
be included (RFC-1766 only defines the tags themselves and a Content-Language
header for MIME-like structures). I presume the idea is that the string
start off with "<lang-tag>:" followed by the actual text?

  Cheers,

  Ronald

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 07 Aug 1999 14:45:20 -0700

>start off with "<lang-tag>:" followed by the actual text?

I don't think so. Embedding language tags in Unicode (of which UTF-8 is a 
transformation) is described in RFC 2482. This is the method that is 
expected to be used in all Unicode text that will contain language tags. 
However, there are not many applications that use language tagging, so I 
can't say for certain if that's what is supposed to be done in CMP.

FWIW, UTF-8 is described in RFC 2279 (this wasn't referenced in the CMP 
spec). Also note that RFC 1766 is being updated; the Internet Draft for the 
update is draft-alvestrand-lang-tags-v2-00.txt.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Ronald Tschal�r <[email-masked]>
Date: Sat, 7 Aug 1999 17:07:54 -0700 (PDT)

> expected to be used in all Unicode text that will contain language tags. 

Ah, wasn't aware of that spec. Maybe a refernce to it should be included
in the above text.

Looking 2482, it allows any number of tags anywhere in the text. For
PKIFreeText it seems to me that one tag per string, located at the
beginning, should suffice. So how about:

     PKIFreeText ::= SEQUENCE SIZE (1..MAX) OF UTF8String
         -- text encoded as UTF-8 String (note:  each UTF8String SHOULD
         -- include a single RFC 1766 language tag, as specified in RFC
	 -- 2482, located at the beginning of the string, to indicate
	 -- the language of the contained text)

> update is draft-alvestrand-lang-tags-v2-00.txt.

Thanks for the pointer - good to see language ranges included.

  Cheers,

  Ronald

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 07 Aug 1999 18:32:26 -0700

>         -- the language of the contained text)

I see three problems with this:

- RFC 2482 is an Informational RFC, and therefore there cannot have a 
normative reference to it in a standards-track RFC

- There are a fair number of people in the Unicode community who do not 
think much of language tagging; they consider it unneeded overhead that is 
rarely of value. Language tags are really only of value on text that will 
be machine-processed. Unfortunately, it is impossible to determine this 
ahead of time in most cases. For instance, PKIFreeText is meant to be read 
*by* a human, so the language tag is superfluous (either the human 
understands the language, or they don't). However, if that text is read 
*to* a human, such as by a text speaking program, then the language tag can 
be very valuable.

- Language tags in UTF-8 come out really long, like about 28 octets on 
average, due to their position in the character set (way out in plane 14).

I apologize for not having seen this in the CMP drafts; otherwise, I would 
have suggested that it be changed to a MAY or dropped altogether. Any UTF-8 
reader should be able to handle language tags just fine if they are in the 
text; nothing special is needed. This SHOULD may get dropped when going 
from Proposed to Draft Standard unless two implementations both use them in 
a noticeable way, which seems unlikely.

(Folks interested in IETF and internationalization, at least as it affects 
Internet mail, can find lots of information and pointers to RFCs at 
<http://www.imc.org/imc-intl/>.)

--Paul Hoffman, Director

---

From: Ronald Tschal�r <[email-masked]>
Date: Sat, 7 Aug 1999 21:24:52 -0700 (PDT)

> I see three problems with this:
[snip]
> *to* a human, such as by a text speaking program, then the language tag can 

I disagree with this. The spec says further down that:

   The freeText field may be used to send a human-readable message to
   the recipient (in any number of languages).  The first language used
   in this sequence indicates the desired language for replies.

In other words you get a list of languages, and the client can then
choose most suitable for display according to the clients preferences
or environment. I.e. PKIFreeText is much like multipart/alternative.
So labeling the individual variants is useful and neccessary.

> text; nothing special is needed.

Ok, so I'm confused. Are you refering to some other way of embeding
language tags in the text?

  Cheers,

  Ronald

---

From: Paul Koning <[email-masked]>
Date: Mon, 9 Aug 1999 09:52:07 -0400

>> >The definition of PKIFreeText in RFC-2510 says: > > PKIFreeText
 >> ::= SEQUENCE SIZE (1..MAX) OF UTF8String > -- text encoded as
 >> UTF-8 String (note: each UTF8String SHOULD > -- include an RFC
 >> 1766 language tag to indicate the language > -- of the contained
 >> text) > >Unfortunately, I could find no hint as to *how* the
 >> language tag should >be included (RFC-1766 only defines the tags
 >> themselves and a Content-Language >header for MIME-like
 >> structures). I presume the idea is that the string >start off with
 >> "<lang-tag>:" followed by the actual text?
 >> 
 >> I don't think so. Embedding language tags in Unicode (of which
 >> UTF-8 is a transformation) is described in RFC 2482. This is the
 >> method that is expected to be used in all Unicode text that will
 >> contain language tags.

 > Ah, wasn't aware of that spec. Maybe a refernce to it should
 > be included in the above text.

 > Looking 2482, it allows any number of tags anywhere in the
 > text. For PKIFreeText it seems to me that one tag per string,
 > located at the beginning, should suffice. 

Maybe so, but maybe not.  Does it make matters significantly easier to 
deviate from the existing RFC and define a data type that is almost
the same as something already defined but not quite, because it is
subject to new limitations?

"Free text" suggests that its use is pretty much open; that being the
case it doesn't seem terribly useful to throw in new restrictions.

	paul

---

From: Ronald Tschal�r <[email-masked]>
Date: Mon, 9 Aug 1999 12:04:05 -0700 (PDT)

> subject to new limitations?

First, given that PKIFreeText is defined as a *sequence of* UTF8String,
and given the texts

and 

this seemed to imply that the intent was one language per string. However,
that is not specified anywhere. So, if the spec allows any number of tags
anywhere in the strings then the following questions arise:

  - What if the first string contains a language tag in the middle of
    the string - is the "first language used" the default language assigned
    to the text before the language tag (which is presumably whatever
    language the client happens to choose as default), or is it that of
    the first language tag? I presume the former?

  - Is there any reason to ever include more than one string? After all,
    you can just create one string with all the languages separated by
    the language tags.
    
And yes, the handling becomes easier if it's one language tag per string
at the beginning, but it's not like a full free-format is impossible to
implement.

> "Free text" suggests that its use is prety much open; that being the

I was just trying to nail down what I thought was the intent. Maybe I
just read too much into those sentences.

  Cheers,

  Ronald

---

From: Daniel Finkelstein <[email-masked]>
Date: Tue, 10 Aug 1999 15:43:38 -0400

> case it doesn't seem terribly useful to throw in new restrictions.

If I may offer an analogy, albeit an unusual one: when swimming freestyle
one traditionally chooses the crawl, but every so often you see a breast
stroke or something the athlete prefers. But it is assumed the crawl will
be done.

Free isn't necessarily free if traditional steps in, or peer pressure.

--
Daniel Alex Finkelstein
New Technologies
phone   212-383-2951
pager   917-427-1630
fax     212-383-3289
Securities Industry Automation Corporation

---

From: "tog" <[email-masked]>
Date: Wed, 11 Aug 1999 08:06:03 -0700

This is somewhat of a complex issue. Depending on what the PKIFreeText is
used for, it may or may not need a language statement embedded with it  and
this is specific to what and how you intend to use this construct.

For instance if PKIFreeText is used to convey any "transaction specific"
data, there may be a need to specify the language type for any objects
encoded into the Digital Structure, but my feeling is that there should be a
generic applicable language definition tag for the processing of encased or
encapsulated objects inside a larger PKI process.

The reason is that in commercial processing it may be necessary to prove
that the program that used the PKI enablement did a specific set of XY&Z and
this language tag would be useful in establishing a context for the enclosed
objects. This exact same facility can be used in timestamping as well for
setting a an evaluatory context for complex stamping or other efforts where
totally online interpretation is necessary.

The concept to me seems like a perfect use of an OID tree. There are a
limited number of languages on this planet, and it would be pretty easy to
setup a parallel tree to the ISO character encoding tree to represent these
since for commercial purposes there are less than 20 or thirty instances of
different language tags. This in and of itself may be one of the things that
pushes us as a planetary race to come to grips with uniform methods of
representing the context of our interpretation models.

My feeling is that only a small subset of languages will really be necessary
and since this is really for commerce based  or 'legally binding'
applications, that this effort should not be too complex and that the system
described in RFC2482 is likely to work here just
fine.(http://www.ietf.org/rfc/rfc2482.txt )

Todd