Subject: [jose] The use of \/ in JOSE

From: Anders Rundgren <[email-masked]>
Date: Mon, 02 Sep 2013 08:59:00 +0200

Spurred by James Manger's intelligent observations regarding JCS, I began to do some research on the state of \/ in JSON these days.

>From an extract of http://tools.ietf.org/id/draft-ietf-oauth-json-web-token-11.html

The following is an example of a JWT Claims Set:

{"iss":"joe",
 "exp":1300819380,
 "http://example.com/is_root":true}

Base64url encoding the bytes of the UTF-8 representation of the JSON Claims Set
yields this Encoded JWS Payload, which is used as the JWT Second Part
(with line breaks for display purposes only):

eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFt
cGxlLmNvbS9pc19yb290Ijp0cnVlfQ

AFAICT the serialized output doesn't perform the expected (?) escaping of /.

Although I think this just fine, I'm still puzzled by the lack of stringency that seems to be plaguing th JSON world.

In JCS I made it simple by claiming that \/ is not only unnecessary; it actually forbidden!

The JOSE WG probably need to specify which particular "camp" you belong to since a "standard" JSON serializer presumably wouldn't be compliant with your sample code.

Cheers
Anders

---

From: Richard Barnes <[email-masked]>
Date: Mon, 2 Sep 2013 09:18:39 -0400

Why would "/" be escaped?

On Monday, September 2, 2013, Anders Rundgren wrote:

> Spurred by James Manger's intelligent observations regarding JCS, I began
> to do some research on the state of \/ in JSON these days.
>
> From an extract of
> http://tools.ietf.org/id/draft-ietf-oauth-json-web-token-11.html
>
> Although I think this just fine, I'm still puzzled by the lack of
> stringency that seems to be plaguing th JSON world.
>
> The JOSE WG probably need to specify which particular "camp" you belong to
> since a "standard" JSON serializer presumably wouldn't be compliant with
> your sample code.
>
> Cheers
> Anders
>
> _______________________________________________
> jose mailing list
> [email-masked] <javascript:;>
> https://www.ietf.org/mailman/listinfo/jose
>

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 02 Sep 2013 16:57:18 +0200

> Why would "/" be escaped?

Well, the example shows a JSON object and then it would be logical
to escape / since this is a part of the JSON "standard".

However, when looking further I note that JOSE requires some kind
of non-standard/additional JSON normalizing before applying base64url:

http://tools.ietf.org/id/draft-ietf-jose-json-web-signature-14.html#rfc.section.5.3

Anders

---

From: "Manger, James H" <[email-masked]>
Date: Tue, 3 Sep 2013 10:28:43 +1000

> Well, the example shows a JSON object and then it would be logical to
> escape / since this is a part of the JSON "standard".

You can escape / as \/ in JSON, but you don't have to. "/" and "\/" are both equally valid JSON strings representing the same 1-character logical value. Any JSON parser needs to support both.
I believe the one useful use of \/ is to escape "</script>" as "<\/script>" when it appears as a JSON string in HTML so the HTML parser does not misinterpret it as the end of the script.
In a canonical form of JSON one form needs to be chosen: "/" is the best choice; it is the choice of ECMAScript’s JSON.stringify.
JOSE does not require a canonical form so "http://example.com" and "http:\/\/example.com" are both acceptable. I'm glad the former is chosen for the JOSE examples.

> However, when looking further I note that JOSE requires some kind of
> non-standard/additional JSON normalizing before applying base64url:
> 

No. The string comparison rules are just trying to be clear that JOSE implementations are expected to compare logical string values (not JSON serializations, which are not unique due to optional escape sequences) and are not expected to perform any Unicode normalization, such as NFC or NFKD.

--
James Manger

---

From: Tim Bray <[email-masked]>
Date: Mon, 2 Sep 2013 17:33:06 -0700

On Mon, Sep 2, 2013 at 5:28 PM, Manger, James H <
[email-masked]> wrote:

> You can escape / as \/ in JSON, but you don't have to. "/" and "\/" are
> both equally valid JSON strings representing the same 1-character logical
> value. Any JSON parser needs to support both.
>

Yes.

> I believe the one useful use of \/ is to escape "</script>" as
> "<\/script>" when it appears as a JSON string in HTML so the HTML parser
> does not misinterpret it as the end of the script.
>

No.  Because \/ is just /.

If you want \/ in your output you have to say \\/.

In any case, all this escaping is done at the JSON parser level, so
app-level software doesn’t know whether the JSON textual form contained /
or \/.

Yes, if you read and re-serialize a JSON text and foolishly escape
characters here and there, you will break signatures.  JSON doesn’t have a
canonical form and is not apt to get one.  -T

> In a canonical form of JSON one form needs to be chosen: "/" is the best
> choice; it is the choice of ECMAScript’s JSON.stringify.
> JOSE does not require a canonical form so "http://example.com" and
> "http:\/\/example.com" are both acceptable. I'm glad the former is chosen
> for the JOSE examples.
>
>
> No. The string comparison rules are just trying to be clear that JOSE
> implementations are expected to compare logical string values (not JSON
> serializations, which are not unique due to optional escape sequences) and
> are not expected to perform any Unicode normalization, such as NFC or NFKD.
>
> --
> James Manger
> [email-masked]

---

From: Breno de Medeiros <[email-masked]>
Date: Mon, 2 Sep 2013 17:47:54 -0700

On Mon, Sep 2, 2013 at 5:33 PM, Tim Bray <[email-masked]> wrote:
> On Mon, Sep 2, 2013 at 5:28 PM, Manger, James H
> <[email-masked]> wrote:
>>
> app-level software doesn’t know whether the JSON textual form contained / or
> \/.
>

And is in recognition of the fact that there is no JSON canonical form
that Jose has defined signature/integrity protection on the base64
encoded format. Any manipulations of the input are thus disallowed,
thus avoiding an entire class of interoperability failures.

-- 
--Breno

---

From: Anders Rundgren <[email-masked]>
Date: Tue, 03 Sep 2013 05:21:15 +0200

On 2013-09-03 02:33, Tim Bray wrote:
> On Mon, Sep 2, 2013 at 5:28 PM, Manger, James H <[email-masked] <mailto:[email-masked]>> wrote:
>

No, you don't.  The darn JSON serializer does the escaping for you regardless if you want it or not:

http://stackoverflow.com/questions/6011012/making-json-not-escape-forward-slashes

Also in Android :-)

>
> In any case, all this escaping is done at the JSON parser level, so app-level software doesn’t know whether the JSON textual form contained / or \/.
>
> Yes, if you read and re-serialize a JSON text and foolishly escape characters here and there, you will break signatures.  JSON doesn’t have a canonical form and is not apt to get one.  -T

If you add an option to a JSON serializer that will cost you some 3-20 lines you may indeed produce "Canonical JSON" that is both user-friendly ("readable") and signature-adapted.

Web-developers have successfully used non-escaped URLs in JavaScript since it was introduced by Netscape in the mid-90'ties.

Going back to the subject line,  JOSE standards apparently doesn't need any "improvements" in JSON serializers.
However, it comes at a price: Obscured messages.
If the "message" is just a token there's no problem to solve.
For more elaborate messaging schemes like KeyGen2 and payments the situation is slightly different, particularly from a debug/documentation point-of-view.

Cheers
Anders

>     JOSE does not require a canonical form so "http://example.com" and "http:\/\/example.com <http://example.com>" are both acceptable. I'm glad the former is chosen for the JOSE examples.
>
>
>     [email-masked] <mailto:[email-masked]>