Subject: [Megaco] The ABNF coding of VALUE doesn't allow to represent a generic UTF-8 string/char

From: "Contardi Angelo" <[email-masked]>
Date: Fri, 24 Jun 2005 17:33:54 +0200

Hello,

from my previous  private  discussion with Christian Groves, i have deduce this:

The ABNF coding of VALUE doesn't  allow to  represent a generic  UTF-8 string or
char (RFC 3629) because of:

1) The FIRST byte of an UTF-8 char may be ANY ASCII char in the range  0x00-0x7E
   and, for instance, the  ASCII 0x00-0x07  and 0x22 (") are not  allowed in ANY
   ABNF form of VALUE.
   
2) Furthermore, UTF-8 chars "greater than 0x7E", need  "ASCII chars"  (to better
   say, Octets) in the range 0x80-0xFF (not ALL), not allowed in  ANY  ABNF form
   of VALUE.

So, to "correct" this problem, i can suggest two possible solutions:

1) The simple one, code the UTF-8 string/char  in "Octect Mode", as described in
   ANNEX B.3. This is a BAD solution from the efficiency  transmission  point of
   view because of it "halve the TX band": to TX one  UTF-8  char  (max 4 Octet)
   i must TX max 2 x 4 = 8 Octet (ASCII chars).

2) The difficult one, allow the ABNF quoted form of VALUE to  TX ALL ASCII chars
   0x01-0xFF (the range 0x80-0xFF are more  properly  named  OCTET in RFC2234 ),
   except 0x22 ("), that should be  ESCAPED with "\", as  already  done for ABNF
   Local and Remote Descriptor (see SDP).  Note that '\0' (0x00) is  NOT ALLOWED
   in this new quoted string form as in the present one, but it's  not a problem
   because in UTF-8 the char '\0' (0x00) is same as in ASCII (string terminator)
   and is NOT used to code "non ASCII" UTF-8 chars,  all those chars > 0x7F that
   require more than one ASCII chars to be encoded (from 2 to 4 ASCII chars). In
   fact the "extra chars" needed to code an UTF-8 char are all above 0x7F.

While the 1st mode doesn't  require any  modification  of ABNF syntax (but is it
applicable in any case ?), the 2nd one require this ABNF modification:

      quotedString         = DQUOTE *(quotedChar) DQUOTE
  	
      ; %x22 (") is allowed just if "escaped" with "\"
      quotedChar           = ( %x01-21 / "\" DQUOTE / %x23-FF )

In a parser implementation, i think, this  is not a  "terrible complication" and
can be solved in the same way of "octetString" of Local and RemoteDescriptor.
It require  also to  implement an  ESCAPE "strip(rx)/padding(tx)"  mechanism, as 
already required for SDP.

P.S.: I suppose NO ONE need to  send the  "string terminator" '\0' (0x00)  in an
      UTF-8 string or char. If my  assertion is false,  in the "solution 2)" the
      %x00 should also be "escaped". Viceversa,  the "solution 1)"  can  already
      "transport" %x00 octet.

Best regards

        o o o o o o o . . .   ___________________________________
        o      _____           ||        Angelo Contardi         |
      .][__n_n_|DD[  ====_____  |   [email-masked]   |
     >(________|__|_[_________]_|________________________________|
     _/oo OOOOO oo`  ooo   ooo  'o!o!o                      o!o!o`

_______________________________________________
Megaco mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/megaco