Subject: R: R: [Megaco] The ABNF coding of VALUE doesn't allow to represent ageneric UTF-8 string/char

From: "Contardi Angelo" <[email-masked]>
Date: Mon, 27 Jun 2005 17:29:12 +0200

Hi Tom,

your solution is interesting as a "work-around" to avoid the constrains of actual ABNF
but i think the discussion here is focused on "how to EXTEND the actual ABNF syntax"
to allow ANY UTF-8 string/char and use this extension in the incoming H.248.1 V3,
not how to carry with the actual syntax a special "message format" that include
ANY ABNF syntax.
If i'm not wrong, H.248.9 use a structured message to carry ANY (?)
UTF-8 string/char, whith some "TAGs" to mark UTF-8 chars that can't be
TX within actual ABNF notation of VALUE.
So the target here is opposite: "how to extend the ABNF form of VALUE in a simple way to
allow the "transport" of any UTF-8 string/char", to AVOID the usage of specific
coder/decoder in the future packages.
Obviously this have a sense if the solution is "simple" (require a little more effort at
the "parser level") and "general" (allow to carry ALL UTF-8 strings and chars). 

-----Messaggio originale-----
Da: Tom Taylor [mailto:[email-masked]]
Inviato: lunedì 27 giugno 2005 15.02
A: Contardi Angelo
Cc: Steve Cipolli; Megaco IETF - Mail List (E-mail); Christian Groves
(E-mail)
Oggetto: Re: R: [Megaco] The ABNF coding of VALUE doesn't allow to
represent ageneric UTF-8 string/char

I dealt with the issue of non-printable characters when putting together 
the Advance Audio Server packages (H.248.9).  Please see clause 
6.2.5.1.2/H.248.9 for a discussion of the matter.

Contardi Angelo wrote:
> He Steve,
> 
> i don't understand these points:
> 
> 1) Where are included non printing chars from 0x01-0x31 and 0x127 in VALUE? Note that the "general
>    definition of packages" (pargraphs 12.x.x of H.248.1) allow those chars too. Otherwise it should
>    be stated somewhere the opposite.
> 
> 2) Why  %x80-F7 should be included in both "quoted" and "non quoted" form of value. Isn't it enought
>    to include them just in quoted form of VALUE ?  
> 
> 3) How to TX 0x22 (")? It's a NEED, not an optional to tx 0x22 too, because of, for instance,
>    the package "Text telephone package", described in the specification H.248.2, need to tx ANY
>    UTF-8 string, 0x22 included. I tink this package is used to tx a text writed by users and so
>    it may include 0x22 too. And actually 0x22 can't be tx in ANY VALUE form. So how do you
>    solve this problem? 
> 
> -----Messaggio originale-----
> Da: Steve Cipolli [mailto:[email-masked]]
> Inviato: venerdì 24 giugno 2005 21.11
> A: Contardi Angelo
> Cc: Sasha Ruditsky; Christian Groves; [email-masked];
> [email-masked]
> Oggetto: RE: [Megaco] The ABNF coding of VALUE doesn't allow to
> represent ageneric UTF-8 string/char
> 
> 
> Angelo,
> 
> Today all ASCII characters except double quotes can be specified by the VALUE production (some of the characters need to be in the quotedstring form).  Adding UTF-8 does not change this.  The part of UTF-8 that is < 0x80 is exactly ASCII and does not change or need to change the way in which VALUE is encoded.  
> 
> UTF-8 characters above 0x7F are encoded as multiple bytes in which each byte of the character is guaranteed to be > 0x7F.  So UTF-8 characters are either < 0x80 and are simply the equivalent of ASCII or they are encoded into multiple bytes in which every byte of the character is > 0x7F.  There will never be a circumstance where a byte of a UTF-8 multibyte character collides with _any_ ASCII character.
> 
> Perhaps the table below will help clarify
> 
> Scalar Value 			1st Byte 2nd Byte 3rd Byte 4th Byte
> 00000000 0xxxxxxx 		0xxxxxxx
> 00000yyy yyxxxxxx 		110yyyyy 10xxxxxx
> zzzzyyyy yyxxxxxx 		1110zzzz 10yyyyyy 10xxxxxx
> 000uuuuu zzzzyyyy yyxxxxxx 	11110uuu 10uuzzzz 10yyyyyy 10xxxxxx
> 
> Notice that "one byte" UTF-8 characters have their high order bit cleared (0) and that all bytes of all multi-byte UTF-8 encodings have the high-order bit set (1).  In other words all multi-byte character's bytes are > 0x7F.
> 
> So changing the productions to:
>   VALUE                = quotedString / 1*(SafeChar / %x80-F7)
>   quotedString         = DQUOTE *(SafeChar / %x80-F7 / RestChar / WSP) DQUOTE
> 
> as Sasha Ruditsky suggests will allow UTF-8 characters to be used in VALUEs.  ASCII characters (and UTF-8 equvialents < 0x80) that are in the RestChar set will need to be quoted just as they have always been.  UTF-8 characters above 0x7F may or may not be quoted, just as all the ASCII characters in the SafeChar set.
> 
> --Stephen
> 	
> 
> 
>>-----Original Message-----
>>From: Contardi Angelo [mailto:[email-masked]] 
>>Sent: Friday, June 24, 2005 12:25 PM
>>To: Steve Cipolli
>>Subject: R: [Megaco] The ABNF coding of VALUE doesn't allow 
>>to represent ageneric UTF-8 string/char
>>
>>
>>
...

_______________________________________________
Megaco mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/megaco