Subject: [jose] On-line Demo/Test of JSON Clear Text Signatures

From: Anders Rundgren <[email-masked]>
Date: Sat, 07 Sep 2013 12:37:21 +0200

https://mobilepki.org/jcs/home

Enjoy!

Canonicalization

Precondition: Valid JSON data as described on http://www.json.org has been received.

Restriction: The only permitted (and required) escape sequences are \\ and \" because JSON supplied as a part of a web-page must anyway be escaped according to web-rules, while JSON supplied through HTTP etc. only needs to be parsable.

Another reason for this somewhat radical step is that the JSON world appears to be severely divided on escapes sequences in general:
http://stackoverflow.com/questions/6011012/making-json-not-escape-forward-slashes.

The canonicalization steps are as follows:
- Whitespace is removed which in practical terms means removal of all characters outside of quoted strings having a value <= ASCII space
- Properties are sorted in lexical UNICODE order using their unescaped form.  Zero-length properties are not allowed
- The JSON object associated with the Signature is recreated using the actual textual data.  Rationale: Numbers are ambiguously defined in JSON which means that encoding and decoding most likely will differ among JSON implementations.  There is another, more subtle issue as well.  If a sender for example assigns a large number such as 0.99999999999999999999 to a JSON property there is a possibility that a receiver due to limitations in arithmetic precision rather interprets it as 1.0.  To cope with these potential problems, a compliant parser must preserve the original textual representation internally in order to perform proper canonicalization

Anders

---

From: "Manger, James H" <[email-masked]>
Date: Mon, 9 Sep 2013 09:45:33 +1000

Anders,

JSON strings are perfectly unambiguous, they are simply not unique (like many text formats that have escapes).

I am glad you want to pick a unique (canonical) form, but if your choice involves not escaping newlines, for instance, then please show some courtesy to the rest of the community by not calling it JSON — because it simply will not interoperate with many other JSON-compliant systems.

For instance, try the console (press F12) of the popular Chrome browser:

> JSON.parse("\"hello\\nworld\"");
  "hello
  world"
> JSON.parse("\"hello\nworld\"");
  SyntaxError: Unexpected token

A laptop.org form might also only escape \ and ", but it also accepts arbitrary bytes that are not valid UTF-8. Perhaps that is useful in their circumstance, but it will not interoperate with general JSON systems so should not be called JSON.

JCS looked interesting, but if it doesn't bother to meet the easy bar of complying to the JSON string format then my interest vanishes.

--
James Manger

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Anders Rundgren
> Sent: Saturday, 7 September 2013 8:37 PM
> To: [email-masked]
> Subject: [jose] On-line Demo/Test of JSON Clear Text Signatures
> 
> Restriction: The only permitted (and required) escape sequences are \\
> and \" because JSON supplied as a part of a web-page must anyway be
> escaped according to web-rules, while JSON supplied through HTTP etc.
> only needs to be parsable.
> 
> Another reason for this somewhat radical step is that the JSON world
> appears to be severely divided on escapes sequences in general:
> - Whitespace is removed which in practical terms means removal of all
> characters outside of quoted strings having a value <= ASCII space
> - Properties are sorted in lexical UNICODE order using their unescaped
> form.  Zero-length properties are not allowed
> - The JSON object associated with the Signature is recreated using the
> actual textual data.  Rationale: Numbers are ambiguously defined in
> JSON which means that encoding and decoding most likely will differ
> among JSON implementations.  There is another, more subtle issue as
> well.  If a sender for example assigns a large number such as
> 0.99999999999999999999 to a JSON property there is a possibility that a
> receiver due to limitations in arithmetic precision rather interprets
> it as 1.0.  To cope with these potential problems, a compliant parser
> must preserve the original textual representation internally in order
> to perform proper canonicalization
> 
> Anders

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Anders Rundgren
Sent: Saturday, 7 September 2013 9:30 PM
To: [email-masked]
Subject: [jose] Yet another Canonical JSON

http://wiki.laptop.org/go/Canonical_JSON

It was nice to see that this scheme also outlaws all escapes but \\ and \".
It looks like a trend :-)

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 09 Sep 2013 21:20:38 +0200

>   SyntaxError: Unexpected token

If restoring \n would make you happ{y|ier} I would do that immediately.   But I guess you require the whole bunch of escapes, right?
Unfortunately JSON escaping is a mess seen from a canonicalization perspective:

>JSON.parse ('{"escape": "A/\/"}');
Object {escape: "A//"}

The JSON RI exhibits completely arbitrary ways of dealing with /-escapes including looking for </ combinations.

I'm worried about being entangled in a world of moderately compatible JSON systems where the spec will always be subject to minor "adjustments".

The bottom line seem to be that there are two fundamentally different ways of dealing with JSON canonicalization:
1. Perform CN through fairly sophisticated post-and pre-processing
2. Require changes in JSON encoders/decoder (CN option)

>
>
> A laptop.org form might also only escape \ and ", but it also accepts arbitrary bytes that are not valid UTF-8.
> Perhaps that is useful in their circumstance, but it will not interoperate with general JSON systems so should not be called JSON.

I also don't see the point with arbitrary binary data; for that Base64 seems like the only reasonable solution.

> JCS looked interesting, but if it doesn't bother to meet the easy bar of complying to the JSON string format then my interest vanishes.

How about this one?

JSON.parse ('{"amount":1.00}');
Object {amount: 1}

Using the proposed cheme it works right-out-of-the-box, using the "right" approach you end-up with error-prone and processing-intensive pre-processing.

I'm also beginning to question sorting of properties.  If array element order can be maintained, property ordering should be that as well.

How about this lowercase-camelcase thing?
publicKey {"rsa"
is much better than the current
PublicKey {"RSA"
?

Anyway, neither the spec nor the code is cast in concrete :-)

Cheers
Anders

---

From: "Manger, James H" <[email-masked]>
Date: Tue, 10 Sep 2013 10:20:01 +1000

> If restoring \n would make you happ{y|ier} I would do that immediately.
> But I guess you require the whole bunch of escapes, right?

Correct. 34 chars must be escaped to comply with JSON, including U+0000 to U+001F.

--
James Manger

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 12 Sep 2013 06:16:24 +0200

> Correct. 34 chars must be escaped to comply with JSON, including U+0000 to U+001F.

I will update the scheme but I'm a bit unsure on the result:

http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
 "a Unicode escape sequence occurring within a string literal in an
  ECMAScript program always contributes a character to the String value
  of the literal and is never interpreted as a line terminator or as a
  quote mark that might terminate the string literal"

Chrome:
JSON.parse ('{"esq":"\u000a"}');
SyntaxError: Unexpected token

I will adhere to Chrome's interpretation.

Anders

---

From: David Waite <[email-masked]>
Date: Wed, 11 Sep 2013 23:28:06 -0600

> SyntaxError: Unexpected token

Do you mean:
JSON.parse('{"esq":"\\u000a"}');

-DW

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 12 Sep 2013 09:50:44 +0200

> JSON.parse('{"esq":"\\u000a"}');

Thank you David,
I was obviously confused :-)

So \u000a will be treated as \n during input but be normalized to \n in output and during canonicalzation.

Anders

---

From: Anders Rundgren <[email-masked]>
Date: Fri, 13 Sep 2013 14:38:35 +0200

Hopefully the escaping is more tastful now...

https://openkeystore.googlecode.com/svn/resources/trunk/docs/JSON-Clear-Text-Signature-Scheme.pdf

The new sample message indicates where I think this stuff is headed (outside of esoteric protocols like KeyGen2).

Cheers
Anders