Subject: [jose] Canonical JSON

From: Anders Rundgren <[email-masked]>
Date: Thu, 05 Sep 2013 06:39:08 +0200

Just to assure myself of the slightly confused state of JSON escaping,
I downloaded the Java RI (Reference Implementation) from json.org and
found the following lines in the JSON serializer:

  if (c < ' ' || (c >= '\u0080' && c < '\u00a0')
              || (c >= '\u2000' && c < '\u2100')) {
            w.write("\\u");
            hhhh = Integer.toHexString(c);
            w.write("0000", 0, 4 - hhhh.length());
            w.write(hhhh);
         } else {
        w.write(c);
   }

IMHO, Canonical JSON doesn't have to escape anything but '\' and '"'.

For JSON data supplied as a part of a web-page, escaping must anyway
follow whatever the web-environment requires as proved by the RI.

That an unescaped '\n' in a string would cause a JS syntax error is
true but this only applies to "in-page" JSON.

JSON data received through XHR or WebSockets doesn't suffer from JS issues.
Well, a JSON _parser_ may beg to differ but that's another thing.
Canonical JSON would indeed require a minute parser update.

The other part of the game (representation of floating point data etc),
has a similar simple solution: Keep the textual format and do the
[potentially incorrect] conversions in "get" methods only.

Canonical JSON and JCS (JSON Clear-text Signature) will hopefully be
available for Android in a months or so.

Cheers
Anders

---

From: Tim Bray <[email-masked]>
Date: Wed, 4 Sep 2013 22:21:42 -0700

What is this “Canonical JSON” of which you speak? -T

On Wed, Sep 4, 2013 at 9:39 PM, Anders Rundgren <
[email-masked]> wrote:

> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose
>

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 05 Sep 2013 08:32:19 +0200

> What is this “Canonical JSON” of which you speak? -T

Canonical JSON is an unexpected spinoff of the SKS/KeyGen2 project:
http://openkeystore.googlecode.com/svn/resources/trunk/docs/sks-keygen2-exec-level-presentation.pdf

It is currently defined in the following work-in-progress document:
https://openkeystore.googlecode.com/svn/resources/trunk/docs/JSON-Clear-Text-Signature-Scheme.pdf

Cheers
Anders

> 
> 
> On Wed, Sep 4, 2013 at 9:39 PM, Anders Rundgren <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     I downloaded the Java RI (Reference Implementation) from json.org <http://json.org> and
>     [email-masked] <mailto:[email-masked]>

---

From: David Waite <[email-masked]>
Date: Thu, 5 Sep 2013 01:14:55 -0600

> true but this only applies to "in-page" JSON.

The JSON RFC disallows all unescaped control characters inside strings - including newlines, carriage returns, and tabs. You would indeed need to escape these characters for the JSON to be considered valid. 

Since the JSON Spec was nice enough to provide shorter versions of these commonly used values (e.g. \n vs \u000a ), you will need to define which form is canonical. It sounds like in general you are leaning toward using the an unescaped character if valid, then the shortened escape sequences, and finally the unicode escape sequence.

-DW

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 05 Sep 2013 10:39:04 +0200

On 2013-09-05 09:14, David Waite wrote:>
> The JSON RFC disallows all unescaped control characters inside strings -
> including newlines, carriage returns, and tabs. You would indeed need to
> escape these characters for the JSON to be considered valid.

Since JSON escaping for serializers already is all over-the-map, is ambiguous and like the Reference Implementation builds on empirical data such as "Some browsers can't deal with certain UTF-8 characters", the concept of valid JSON doesn't exist in the real world.

Therefore my take on Canonical JSON requires JSON tool-builders to introduce a flag which [presumably] by default preserves the existing behavior but would if set make the tool adhere to Canonical JSON.

> Since the JSON Spec was nice enough to provide shorter versions of these
> commonly used values (e.g. \n vs \u000a ), you will need to define which
> form is canonical.

Forward slashes are already treated in two different ways so it seems that there is no way you can introduce a canonical form without breaking existing systems.  It should be easier adding a flag than having to deal with a gazillion of systems that you have no knowledge of.

http://stackoverflow.com/questions/6011012/making-json-not-escape-forward-slashes

> It sounds like in general you are leaning toward using the an unescaped
> character if valid, then the shortened escape sequences, and finally the
> unicode escape sequence.

In Canonical JSON all escapes but \\ and \" are simply "outlawed".
This surely isn't "kosher" but since I'm not creating a standard, only working systems, I can do things a standardization WG cannot :-)

Cheers
Anders