Subject: [Geopriv] Format of SSID in held-measurements-01.txt

From: David Waitzman <[email-masked]>
Date: Mon, 13 Sep 2010 15:49:14 -0400

I have a comment about the choice of data representation of SSID along with a suggested change.

The held-measurements draft makes a common assumption that the SSID (not the BSSID) is a printable character string.  This is expressed in the XML on page 52:
    <xs:simpleType name="ssidBaseType">
      <xs:restriction base="xs:token">
        <xs:maxLength value="32"/>
      </xs:restriction>
    </xs:simpleType>

Actually, xs:token has some important restrictions on its contents.  http://www.w3.org/TR/xmlschema-2/#token says:
"The ·value space· of token is the set of strings that do not contain the carriage return (#xD), line feed (#xA) nor tab (#x9) characters, that have no leading or trailing spaces (#x20) and that have no internal sequences of two or more spaces."

The IEEE 802.11 standards simply state a SSID is "0 to 32 octets".  Nothing about ASCII or UTF-8.  I think it is just a convenience hack that many 802.11 devices show them as if they are ASCII and some in UTF-8.

Therefore, the held-measurements draft can not represent all valid SSID values nor even all common ones (such as with leading/trailing/multiple spaces or a tab).  

Additionally, non-printable SSIDs have been used in the past to defeat wardriving tools, by having contents that the tools would crash upon decoding.  If we make a clear comment in the draft to *not* assume ASCII or legal UTF-8 values, we can encourage more robust implementations.

I suggest changing ssidBaseType to be something that can hold any octet value, like:
   <xs:simpleType name="ssidBaseType">
     <xs:restriction base="xs:string">
       <xs:pattern value="(([0-9A-Fa-f]{2})(-[0-9A-Fa-f]{2}){0,31})?"/>
     </xs:restriction>
   </xs:simpleType>

Example: 45-40-01-00-fe

This pattern is similar to what's used for macAddressType.

This way the *protocol* can express all legal underlying values and *implementations* can choose to pretty-print the SSID values if they wish.

A drawback to my suggestion is that the raw XML contents will no longer display human readable SSID values.  But I don't feel that the IETF mandates humans being able to easily read raw packet dumps.

--
David Waitzman
BBN Technologies
[email-masked] 410-290-6160

---

From: "Winterbottom, James" <[email-masked]>
Date: Tue, 14 Sep 2010 04:06:41 +0800

Thanks for the pickup and suggestion David.

Cheers
James

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of David Waitzman [[email-masked]]
Sent: Monday, September 13, 2010 2:49 PM
To: [email-masked]
Subject: [Geopriv] Format of SSID in held-measurements-01.txt

Example: 45-40-01-00-fe

--
David Waitzman
BBN Technologies
[email-masked] 410-290-6160

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 14 Sep 2010 08:19:59 +0800

The SSID as string is an error.

It would be relatively easy to change the base type used here to xs:hexBinary.

The only concern is that, in most cases, an SSID is a string.  Users set and view SSIDs as strings.

LDAP has a similar problem with its attributes and they solved it [RFC4514] differently.  And I proposed that we do the same in response to John Bressler:

  <http://www.ietf.org/mail-archive/web/geopriv/current/msg08717.html>

That solution recognizes the virtues of a text-based protocol and attempts to retain those virtues at a small cost in complexity.

--Martin

p.s. It's not immediately clear, but a token can include any string content: <http://www.w3.org/TR/xml/#AVNormalize>.  The text you cite is misleading.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of David Waitzman
> Sent: Tuesday, 14 September 2010 5:49 AM
> To: [email-masked]
> The held-measurements draft makes a common assumption that the SSID
> (not the BSSID) is a printable character string.  This is expressed in
> the XML on page 52:
> Actually, xs:token has some important restrictions on its contents.
> http://www.w3.org/TR/xmlschema-2/#token says:
> "The ·value space· of token is the set of strings that do not contain
> the carriage return (#xD), line feed (#xA) nor tab (#x9) characters,
> that have no leading or trailing spaces (#x20) and that have no
> internal sequences of two or more spaces."
> 
> The IEEE 802.11 standards simply state a SSID is "0 to 32 octets".
> Nothing about ASCII or UTF-8.  I think it is just a convenience hack
> that many 802.11 devices show them as if they are ASCII and some in
> UTF-8.
> 
> Therefore, the held-measurements draft can not represent all valid SSID
> values nor even all common ones (such as with leading/trailing/multiple
> spaces or a tab).
> 
> Additionally, non-printable SSIDs have been used in the past to defeat
> wardriving tools, by having contents that the tools would crash upon
> decoding.  If we make a clear comment in the draft to *not* assume
> ASCII or legal UTF-8 values, we can encourage more robust
> implementations.
> 
> This way the *protocol* can express all legal underlying values and
> *implementations* can choose to pretty-print the SSID values if they
> wish.
> 
> A drawback to my suggestion is that the raw XML contents will no longer
> display human readable SSID values.  But I don't feel that the IETF
> mandates humans being able to easily read raw packet dumps.
> 
> --
> David Waitzman
> BBN Technologies
> [email-masked] 410-290-6160

---

From: David Waitzman <[email-masked]>
Date: Tue, 14 Sep 2010 13:15:33 -0400

Martin,

> It would be relatively easy to change the base type used here to xs:hexBinary.

Using xs:hexBinary is also a good choice; I was trying to mirror how macAddressType are encoded.

>  <http://www.ietf.org/mail-archive/web/geopriv/current/msg08717.html>

I get geopriv as a digest, for now, so it's too easy to loose track of valuable individual emails like John's and your response to him.

Your suggestion in msg08717.html of:
>>> LDAP uses a simple scheme where a backslash followed by two hex characters is converted into an octet with that value:
>>> 
>>>   octetsAsString = *(vcharx / escaped)
>>>   vcharx = %x21-5b / %x5d-7e ; VCHAR minus backslash
>>>   escaped = %x5c 2HEXDIG

doesn't handle the UTF-8 encoding cases well.  It would escape all non-ASCII values in an UTF-8 string, so for practical real-world cases, would use escaping.  We did tests here with non-ASCII UTF-8 contents in real wireless devices and some were fine with them, like Apple Airports.  

Therefore, I still prefer my proposal or your hexBinary proposal.  At least the LDAP scheme is not lossy as compared to the current HELD-measurements draft.

> p.s. It's not immediately clear, but a token can include any string content: <http://www.w3.org/TR/xml/#AVNormalize>.  The text you cite is misleading.

I looked that up and don't get your point.  In <xs:element name="ssid" type="wifi:ssidBaseType" minOccurs="0"/> we not dealing with an Attribute's value.

---

One additional question for you: what's the difference between "name" and "ssid" in wifiType?

   name:  The broadcast name for the access point.
   ssid:  The service set identifier for the wireless network served by
      the access point.

  <xs:element name="name" type="wifi:ssidBaseType"  minOccurs="0"/>

The one example of a wifiType has these values with different capitalization:
         <name>Example</name>
          <ssid>example</ssid>
 
As far as I can understand, name should be identical to ssid.  Perhaps there was supposed to be some distinction with regards to "hidden" SSIDs?

--
David Waitzman
BBN Technologies
[email-masked] 410-290-6160

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 15 Sep 2010 12:52:33 +0800

David,

> As I expressed in my first message, users don't normally look at raw

This is an interesting argument, and one that I've seen used quite frequently (I've used it myself on occasion).  

When applied to the general question of text-vs-binary protocols, the answer tends to be entirely subjective.  Taken to its extreme, you end up with ASN.1+PER or similar solutions.  

I favour the protocol being usable in the default case.  Even if that sort of usage seems unlikely, it's been hugely beneficial in debugging thus far.

> Your suggestion in msg08717.html of: [...]
> doesn't handle the UTF-8 encoding cases well.  

LDAP solves this by adding UTFMB, a pattern that allows for multi-byte UTF-8 sequences:
  octetsAsString = *(vcharx / escapted / UTFMB)

In XML, the easiest solution is slightly different, and even quite simple:
  ([^\\]|\\[0-9a-fA-F]{2})*

That is, everything other than backslash, or backslash plus two hex characters.  To convert from a raw token value to a sequence of octets, UTF-8 encode all characters except backslash, then replace all backslash-escaped sequences with a single octet.  In reverse, decode UTF-8 and backslash-escape anything that doesn't decode, plus backslash.

The benefits are clear enough:

  <ssid>ManufacturerName</ssid>

...as opposed to:

  <ssid>4d616e7566616374757265724e616d65</ssid>

...with the occasional:

  <ssid>Wêird\5cNàmé</ssid>

> > p.s. It's not immediately clear, but a token can include any string
> content: <http://www.w3.org/TR/xml/#AVNormalize>.  The text you cite is
> misleading.
> 
> I looked that up and don't get your point.  In <xs:element name="ssid"
> type="wifi:ssidBaseType" minOccurs="0"/> we not dealing with an
> Attribute's value.

You have to follow the thread of definitions:  token is defined to have whiteSpace set to "collapse".  This definition references whiteSpace, which references the attribute value normalization section of the XML specification.  

You can check this behaviour with a conformant processor.  Try decoding:

 <foo xsi:type="xs:token"> &#x20;  
  &#x20; </foo>

You should get three spaces.
 
> One additional question for you: what's the difference between "name"

Good question.  I honestly can't remember now, and I can't find a good reason for the "name" field.  A "name" doesn't feature prominently in 802.11-2007, so it must be from somewhere else.  I'll ask around and try to find out for you, but I suspect that it's redundant and safe to remove.

--Martin

---

From: David Waitzman <[email-masked]>
Date: Wed, 15 Sep 2010 10:26:11 -0400

Martin,

> I favour the protocol being usable in the default case.  Even if that sort of usage seems unlikely, it's been hugely beneficial in debugging thus far.

I appreciate your argument: I used to be able to read parts of hex dumps of ASN.1 BER-encoded data from SNMP.  I cede the point that having human readability can help.

>  <ssid>Wêird\5cNàmé</ssid>

But there's a need to be defensive about illegal UTF-8 sequences, including 0 byte values.
See rfc3629.txt section 6 and also http://etutorials.org/Programming/secure+programming/Chapter+3.+Input+Validation/3.12+Detecting+Illegal+UTF-8+Characters/

I suggest a more defensive algorithm: anything not printable ASCII gets hex encoded.  That won't handle Wêird\5cNàmé but it will handle ManufacturerName and \20\00\ff\fbUgly\00\00\00.

--
David Waitzman
BBN Technologies
[email-masked] 410-290-6160

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 16 Sep 2010 09:28:54 +0800

> See rfc3629.txt section 6 and also
> http://etutorials.org/Programming/secure+programming/Chapter+3.+Input+V
> alidation/3.12+Detecting+Illegal+UTF-8+Characters/
> 
> I suggest a more defensive algorithm: anything not printable ASCII gets
> hex encoded.  That won't handle Wêird\5cNàmé but it will handle
> ManufacturerName and \20\00\ff\fbUgly\00\00\00.

The joy of writing a specification is that you don't have to specify what algorithm is used.  Either algorithm would only be an example - you can use the more conservative algorithm, I can choose to provide a more complicated algorithm :)

Taking "Wêïrd\5cNàmé" you get [0x57, 0xC3, 0xAA, 0xC3, 0xAF, 0x72, 0x64, 0x4E, 0xC3, 0xA0, 0x6D, 0xC3, 0xA9].  You could just as easily go with "W\c3\aa\c3\afrd\5cN\c3\a0m\c3\a9"; or even "\57\c3\aa\c3\af\72\64\4e\c3\a0\6d\c3\a9".  They all end up producing the same value.  Certainly, the latter two choices are easier to write code for.

This is what I intend to add:

   An SSID is a sequence of up to 32 octets.  SSIDs are typically presented as strings, which are converted to an octet sequence using either ASCII or UTF-8 encoding.  The "ssid" element is provided as a string that is converted to octets by UTF-8 encoding the string.  Octet values can be expressed directly using a backslash ('\') followed by two hexadecimal digits.

   The value of an SSID is the sequence of octets that
   is produced from the concatenation of UTF-8 encoded sequences of
   unescaped characters and octets derived from escaped components.

The XML will change to have a constraint:

  <xs:pattern value="([^\\]|\\[\da-fA-F]{2}){1,32}"/>

--Martin

---

From: David Waitzman <[email-masked]>
Date: Thu, 16 Sep 2010 10:36:32 -0400

Martin,

>   unescaped characters and octets derived from escaped components.

I think there might be a problem in the revised definition.  What if you have 32 ê values, which you encoded with the two octets \c3\aa?  Per your pattern, that is legal but it's 64 octets.  Perhaps we can get out of it with a statement in the text rather than trying to fix the pattern to be exact (a solution I don't prefer but it's manageable)?

Also I'd note in the text that 0 byte characters are legal in a SSID.
  
>  <xs:pattern value="([^\\]|\\[\da-fA-F]{2}){1,32}"/>

The SSID can be zero bytes, so the pattern (before fixing the above problem) would be:
<xs:pattern value="([^\\]|\\[\da-fA-F]{2}){0,32}"/>

--
David Waitzman
BBN Technologies
[email-masked] 410-290-6160