Subject: [jose] #26: Allow for signature payload to not be base64 encoded

From: "jose issue tracker" <[email-masked]>
Date: Tue, 25 Jun 2013 06:09:27 -0000

#26: Allow for signature payload to not be base64 encoded

 Ticket #23 was closed because for signature computation, the header
 information must be base64 encoded due to security considerations.  This
 is not true for the payload.

 Payload does need to be restricted to the base64URL character set for the
 compact encoding, however it does not need to be restricted for the JSON
 encoding, it is the responsibility of JSON parsers/serializers not to
 change the encoding of a string field.

 For detached content, the payload can be binary without any transport
 problems.  For text the content needs to be UTF8 encoded (maybe) but not
 base64url encoded.  For compact messages it might need to be base64url
 encoded.

 Applications can and should (when necessary) specify that there is one
 encoding that needs to be used.  However for the general purpose library
 all that is needed is to say that we are passing in an octet string to be
 used.  A flag/function name choice can easily be used to determine if the
 passed in value needs to be utf8 encoded or utf8 and base64url encoded.  I
 have put this into my implementation with almost zero effort and it is
 still very easy to understand for callers.

-- 
-------------------------+-------------------------------------------------
 Reporter:               |      Owner:  draft-ietf-jose-json-web-
  [email-masked] |  [email-masked]
     Type:  defect       |     Status:  new
 Priority:  major        |  Milestone:
Component:  json-web-    |    Version:
  signature              |   Keywords:
 Severity:  -            |
-------------------------+-------------------------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/jose/trac/ticket/26>
jose <http://tools.ietf.org/jose/>

---

From: "jose issue tracker" <[email-masked]>
Date: Wed, 26 Jun 2013 00:22:45 -0000

Comment (by [email-masked]):

 Having multiple ways to do something is one of the classic pitfalls to
 avoid in standards.  They tend to generate interop problems because
 developers will often only build the way that they think they need, never
 getting around to the others.   I don’t think the potential benefits of
 having multiple ways to encode the payload outweigh this cost.

 Next, this feature would only work with the JSON serialization; it would
 not work for the Compact Serialization because the payload must be URL
 safe.  The base64url encoding accomplishes that, and so is a necessary
 part of the encoding.  Having the two serializations diverge in this
 manner seems suboptimal.

 Third, I am concerned that the payload will sometimes change during
 transmission if not encoded in some manner.  Tabs can change to spaces.
 LFs can change to CRLFs.  HTML is notorious for adding a blank at the end
 of each line.  Two spaces can change to a space and a non-breaking space.
 Etc.  Not encoding the payload would open us up to all of these potential
 modification-during-transmission problems – all of which would render the
 JWS useless by breaking the signature.

 Finally, this feature seems to be motivated by detached signatures, which
 per my comment on Ticket #25, can be accomplished at the application layer
 without adding general-purpose support for detached signatures.  Thus, I
 don’t think this related feature is needed either.

 For all these reasons, I believe that this issue should be closed as
 “won’t fix”.

  signature              |  Resolution:
 Keywords:               |

Ticket URL: <http://trac.tools.ietf.org/wg/jose/trac/ticket/26#comment:1>
jose <http://tools.ietf.org/jose/>

---

From: "Jim Schaad" <[email-masked]>
Date: Wed, 26 Jun 2013 11:29:52 -0700

Having multiple ways to do something can frequently buy you a great deal of benefit.  However, in this case it is my library that is doing this and not the standard.

The client gives the library an 8-bit string.  The library signs that string.  The library puts that string into the payload field.

The client needs to make the 8-bit string URL safe if it wants to use compact encoding.

Where is there two ways of doing something?

Jim

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> jose issue tracker
> Sent: Tuesday, June 25, 2013 5:23 PM
> To: [email-masked];
> [email-masked]
> Cc: [email-masked]
> Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded
> 
>  Having multiple ways to do something is one of the classic pitfalls to  avoid in
> standards.  They tend to generate interop problems because  developers will
> often only build the way that they think they need, never
>  getting around to the others.   I don t think the potential benefits of
>  Next, this feature would only work with the JSON serialization; it would  not
> work for the Compact Serialization because the payload must be URL  safe.
> The base64url encoding accomplishes that, and so is a necessary  part of the
> encoding.  Having the two serializations diverge in this  manner seems
> suboptimal.
> 
>  LFs can change to CRLFs.  HTML is notorious for adding a blank at the end  of
> each line.  Two spaces can change to a space and a non-breaking space.
> modification-during-transmission problems   all of which would render the
>  Finally, this feature seems to be motivated by detached signatures, which  per
> my comment on Ticket #25, can be accomplished at the application layer
> without adding general-purpose support for detached signatures.  Thus, I  don
> t think this related feature is needed either.
> 
>  For all these reasons, I believe that this issue should be closed as   won t fix .
> 
> --
> -------------------------+----------------------------------------------
> -------------------------+---
> -------------------------+----------------------------------------------
> -------------------------+---
> 
> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose

---

From: Mike Jones <[email-masked]>
Date: Wed, 26 Jun 2013 19:09:31 +0000

Maybe I'm confused about what you were proposing.  These are the two ways of doing something that I thought you were describing:

1.  Have the signature payload be base64url encoded (the status quo)
2.  Have the signature payload be unencoded in the JSON Serialization if a flag is set in the header

If you're proposing that the default be unencoded for the JSON Serialization, that has the pitfall that a simple syntactic transformation is no longer possible between the Compact Serialization and the JSON Serialization.  The two would have unnecessarily diverged.

If I misunderstood your proposal, maybe you could describe it in closer to normative language - for instance, defining the flag you propose to say that the payload is not base64url encoded and saying what its semantics would be for the JSON Serialization and the Compact Serialization.

BTW, your response didn't address the point that a primary purpose of the base64url encoding is to harden the payload against changes that might otherwise occur during transmission.  Not hardening the payload will almost certainly lead to hard-to-debug interop problems in practice.

				-- Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jim Schaad
Sent: Wednesday, June 26, 2013 11:30 AM
To: [email-masked]; Mike Jones
Cc: [email-masked]
Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded

Jim

> From: [email-masked] [mailto:[email-masked]] On Behalf 
> Of jose issue tracker
>  Having multiple ways to do something is one of the classic pitfalls 
> to  avoid in standards.  They tend to generate interop problems 
> because  developers will often only build the way that they think they need, never
>  Next, this feature would only work with the JSON serialization; it 
> would  not work for the Compact Serialization because the payload must be URL  safe.
> The base64url encoding accomplishes that, and so is a necessary  part 
> of the encoding.  Having the two serializations diverge in this  
> manner seems suboptimal.
> 
>  LFs can change to CRLFs.  HTML is notorious for adding a blank at the 
> end  of each line.  Two spaces can change to a space and a non-breaking space.
>  Finally, this feature seems to be motivated by detached signatures, 
> which  per my comment on Ticket #25, can be accomplished at the 
> application layer without adding general-purpose support for detached 
> signatures.  Thus, I  don t think this related feature is needed either.
> 
> -------------------------+--------------------------------------------
> -------------------------+--
> -------------------------+--------------------------------------------
> -------------------------+--
> Ticket URL: 
> <http://trac.tools.ietf.org/wg/jose/trac/ticket/26#comment:1>

---

From: John Bradley <[email-masked]>
Date: Wed, 26 Jun 2013 16:02:57 -0400

At the moment the signature is calculated over the base64url encoded representation of the payload and not the binary string.  

Having two different ways to calculate the signature seems like the wrong thing.  

If we were talking about always calculating the signature over the binary string and then dealing with encoding as a later step I would understand your point better but the flag is changing the processing as well as the representation.

However that would render the compact and JSON forms incompatible without recalculating the signature.

What are the issues with including a 8-bit string in a JSON sting?

What you are proposing seems like a large fundamental change, that would require a matching change in compact, that complicates that format.

Perhaps I am not understanding your proposal.

John B.

---

From: n-sakimura <[email-masked]>
Date: Thu, 27 Jun 2013 11:54:57 +0900

Let me understand the problem.

Are you suggesting that we should be able to do something like:

Case 1:

      {"protected":<integrity-protected shared header contents>",
       "unprotected":<non-integrity-protected shared header contents>",
       "payload":"this is a multi\nline text payload of which a line can 
be very very very long and wrapped during transmission. "
       "signatures":[
        {"header":"<per-signature unprotected header 1 contents>",
         "signature":"<signature 1 contents>"},
        ...
        {"header":"<per-signature unprotected header N contents>",
         "signature":"<signature N contents>"}],
      }

or even

Case 2:

      {"protected":<integrity-protected shared header contents>",
       "unprotected":<non-integrity-protected shared header contents>",
       "payload":{
	"this":"is a json",
	"payload":"which is not base64url encoded"
       }
       "signatures":[
        {"header":"<per-signature unprotected header 1 contents>",
         "signature":"<signature 1 contents>"},
        ...
        {"header":"<per-signature unprotected header N contents>",
         "signature":"<signature N contents>"}],
      }

Then, I am worried about allowing them because in general we would not 
know what would a transmission mechanism and user agents (including text 
editors) would do to them, like Mike mentioned. I do not think we should 
assume the property of the transmission mechanism in JOSE level. 
Assuming that would cause brittleness in the implementations.

For detached signature, I do not understand why you would want 
non-base64url encoded payload. The application should define how to hash 
the octet stream that it wants to calculate signature over it, and we 
can just include the base64url encoded hash in the "payload".
Why would you like to do other encoding for the hash?

-- 
Nat Sakimura ([email-masked])
Nomura Research Institute

PLEASE READ:
The information contained in this e-mail is confidential and intended 
for the named recipient(s) only.
If you are not an intended recipient of this e-mail, you are hereby 
notified that any review, dissemination, distribution or duplication of 
this message is strictly prohibited. If you have received this message 
in error, please notify the sender immediately and delete your copy from 
your system.

---

From: "Jim Schaad" <[email-masked]>
Date: Wed, 26 Jun 2013 20:18:29 -0700

What I am proposing is that it is an option to "harden" the content, but is
not required.  What is signed and what is transported are the same, but what
is signed may or may not be suitable for the compact encoding.

One would always be able to switch from the compact to the JSON
serialization, however the reverse transformation may not always be
possible.  This would be one more reason why it would not be possible, but
there are many others.

What I originally described would by how I would approach it from a library
basis.  However, there are many things that applications can do that are not
doable by a library and a library would only give some additional options to
what was doable.  This means that while I said that the application would be
required to base64url encode the content, it could do this by passing an
indicator to the library that this was the desired behavior (for example by
calling the SignForCompactEncoding function).

The primary purpose of base64 encoding is not to harden the content to
protect from changes, it is to allow you to do the compact encoding and
transport it in a URL.   If you are worried about things which are going to
muck with the content, they are going to be happy to parse it up, unbase64
encode it, muck with it and write it back out.  This does not prevent a
determined entity from doing the mucking.   If I pass the content as a
string field in a JSON encoded object, then nobody is going to much with it
by default.  They are going to need to know that the need to muck with it
before they start.

> From: Mike Jones [mailto:[email-masked]]
> Sent: Wednesday, June 26, 2013 12:10 PM
> To: Jim Schaad
> Cc: [email-masked]
> Subject: RE: [jose] #26: Allow for signature payload to not be base64
encoded
> 
> Maybe I'm confused about what you were proposing.  These are the two ways
> of doing something that I thought you were describing:
> 
> 1.  Have the signature payload be base64url encoded (the status quo) 2.
Have
> the signature payload be unencoded in the JSON Serialization if a flag is
set in
> the header
> 
> If you're proposing that the default be unencoded for the JSON
Serialization,
> that has the pitfall that a simple syntactic transformation is no longer
possible
> between the Compact Serialization and the JSON Serialization.  The two
would
> have unnecessarily diverged.
> 
> If I misunderstood your proposal, maybe you could describe it in closer to
> normative language - for instance, defining the flag you propose to say
that
> the payload is not base64url encoded and saying what its semantics would
be
> for the JSON Serialization and the Compact Serialization.
> 
> BTW, your response didn't address the point that a primary purpose of the
> base64url encoding is to harden the payload against changes that might
> otherwise occur during transmission.  Not hardening the payload will
almost
> certainly lead to hard-to-debug interop problems in practice.
> 
> 				-- Mike
> 
Jim
> Schaad
> Having multiple ways to do something can frequently buy you a great deal
of
> benefit.  However, in this case it is my library that is doing this and
not the
> standard.
> 
> The client gives the library an 8-bit string.  The library signs that
string.  The
> library puts that string into the payload field.
> 
> The client needs to make the 8-bit string URL safe if it wants to use
compact
> encoding.
> > because  developers will often only build the way that they think they
need,
> never
> > would  not work for the Compact Serialization because the payload must
be
> URL  safe.
> > of the encoding.  Having the two serializations diverge in this manner
> > seems suboptimal.
> >
won t fix .

---

From: "Jim Schaad" <[email-masked]>
Date: Thu, 27 Jun 2013 09:40:57 -0700

Nat,

The only thing that can be signed or encrypted is an octet string. (A slight over generalization. GOST for example hashes an integer, tree hashes can do strange things but a sufficiently true statement for our purposes.)

A that being said - yes case 1 would be something you can do.  Case #2 would not be something that you can do because it is not an octet string.

If you are looking at JavaScript, in order to hash something as an octet string it can be one of three things.  A UInt8Array, an Int8Array and, due to convention, a string where the top byte of all of the code points in the string is zero.

We currently don't have an efficient way to encode the Uint8Array case, although that might not be true if we are looking at new ECMA specifications which apparently are adding more defined types and serialization methods for those types.

> From: n-sakimura [mailto:[email-masked]]
> Sent: Wednesday, June 26, 2013 7:55 PM
> To: Jim Schaad
> Cc: [email-masked];
> [email-masked]; [email-masked]
>        "payload":"this is a multi\nline text payload of which a line can be very
> very very long and wrapped during transmission. "
>        "signatures":[
> For detached signature, I do not understand why you would want non-
> base64url encoded payload. The application should define how to hash the
> octet stream that it wants to calculate signature over it, and we can just
> include the base64url encoded hash in the "payload".

If I am not transporting the base64url encoded content, because it is not in the "payload" field, then why should I need to base64 encode it just to validate the signature.  This is the alto case.

Jim

> The information contained in this e-mail is confidential and intended for the
> named recipient(s) only.
> If you are not an intended recipient of this e-mail, you are hereby notified
> that any review, dissemination, distribution or duplication of this message is
> strictly prohibited. If you have received this message in error, please notify
> the sender immediately and delete your copy from your system.

---

From: Mike Jones <[email-masked]>
Date: Thu, 27 Jun 2013 16:49:12 +0000

Jim, you wrote "If I am not transporting the base64url encoded content, because it is not in the "payload" field, then why should I need to base64 encode it just to validate the signature.  This is the alto case."

So am I correct then, in thinking that your motivation for #26 is actually #25 - Detached content for the ALTO use case?  As John and Nat had written, this could be handled by the specs as they are today by signing a hash of the detached content and including a compact JWS representing that signature as an x-alto-signature header.  That doesn't require changing the signature calculation.

				-- Mike

-----Original Message-----
From: Jim Schaad [mailto:[email-masked]] 
Sent: Thursday, June 27, 2013 9:41 AM
To: 'n-sakimura'
Cc: [email-masked]; Mike Jones; [email-masked]
Subject: RE: [jose] #26: Allow for signature payload to not be base64 encoded

Nat,

>        "payload":"this is a multi\nline text payload of which a line 
> can be very very very long and wrapped during transmission. "
>        "signatures":[
> editors) would do to them, like Mike mentioned. I do not think we 
> should assume the property of the transmission mechanism in JOSE level.
> base64url encoded payload. The application should define how to hash 

Jim

> notified that any review, dissemination, distribution or duplication 
> of this message is strictly prohibited. If you have received this 
> message in error, please notify the sender immediately and delete your copy from your system.

---

From: "Jim Schaad" <[email-masked]>
Date: Thu, 27 Jun 2013 10:11:24 -0700

No, my motivation is to allow for a smaller version of the payload string.  In some cases quoting it will generate a smaller string than doing the base64 conversion on the string.

> Sent: Thursday, June 27, 2013 9:49 AM
> To: Jim Schaad; 'n-sakimura'
> Cc: [email-masked]; [email-masked]
> Jim, you wrote "If I am not transporting the base64url encoded content,
> because it is not in the "payload" field, then why should I need to base64
> encode it just to validate the signature.  This is the alto case."
> 
> So am I correct then, in thinking that your motivation for #26 is actually #25 -
> Detached content for the ALTO use case?  As John and Nat had written, this
> could be handled by the specs as they are today by signing a hash of the
> detached content and including a compact JWS representing that signature as
> an x-alto-signature header.  That doesn't require changing the signature
> calculation.
> 
> 				-- Mike
> 
> The only thing that can be signed or encrypted is an octet string. (A slight over
> generalization. GOST for example hashes an integer, tree hashes can do
> strange things but a sufficiently true statement for our purposes.)
> 
> A that being said - yes case 1 would be something you can do.  Case #2 would
> not be something that you can do because it is not an octet string.
> 
> If you are looking at JavaScript, in order to hash something as an octet string it
> can be one of three things.  A UInt8Array, an Int8Array and, due to
> convention, a string where the top byte of all of the code points in the string is
> zero.
> 
> We currently don't have an efficient way to encode the Uint8Array case,
> although that might not be true if we are looking at new ECMA specifications
> which apparently are adding more defined types and serialization methods for
> those types.
> 
> If I am not transporting the base64url encoded content, because it is not in
> the "payload" field, then why should I need to base64 encode it just to
> validate the signature.  This is the alto case.
> 
> Jim
> 
> >
> > --

---

From: Richard Barnes <[email-masked]>
Date: Thu, 27 Jun 2013 18:24:06 -0400

I think there are two separable issues here, both of which Mike and I
probably have divergent opinions on :)
1. Syntax: Should it be possible for the "payload" field JWS to be a JSON
string (instead of base64) when the payload is simply a UTF-8 string?
2. Crypto: Must the JWS Signing Input be base64url encoded.

The current answers to these questions are:
1. It is not possible.  Payload is always base64url encoded.
2. It is always base64url encoded.

The answers should be:
1. The payload should not be base64url encoded unless it cannot be
represented as a JSON string.
2. The JWS Signing Input should not be base64url encoded unless there is a
JWS Protected Header

Neither of these are complicated changes:
1. Add a "b64" header parameter that indicates that the payload is
base64url-encoded binary, as opposed to a UTF-8 string.  Specify that this
parameter is on by default in compact-formatted JWS objects.
2. Modify the signing/verification instructions to switch based on the
presence of a JWS Protected Header: "If the JWS Protected Header is absent
or empty, then the JWS Signing Input is simply the JWS Payload (not
encoded).  If the JWS Protected Header has a non-empty value, then the the
JWS Signing Input is the concatenation of the Encoded JWS Header, a period
('.') character, and the Encoded JWS Payload"

There are two reasons for these, both of which are important to make sure
that this spec is applicable to a broad variety of use cases:
1. Compactness.  As Jim notes, quoted strings are shorter than
base64url-encoded strings
2. Crypto-compatibility: Avoiding base64-encoding means that there's
nothing JWS-specific about the signature value.  So for example, you could
translate a JWS with no protected attributes to a CMS SignedData object
with no protected attributes.

This email brought to you by the number 2 and the letters J, W, and S :)

--Richard

[1] Note that the default value for this header must be false, or else you
lose the compactness advantage!
[2] You could also do this as a "Content-Encoding" equivalent, "cenc":
"b64".  That would give you "base64 agility", as some folks have asked for,
since you could have "b64" and "b64u" values.
[3] You could also do this like CMS, and have the JWS Signing Input in the
latter case be "the concatenation of the Encoded JWS Header, a period ('.')
character, and the Encoded JWS Payload Hash".  Where the Encoded JWS
Payload Hash is the base64url-encoded version of the hash you would
otherwise be signing.  This seems simpler to code, since you just digest
the body, then if there's a protected header, you digest again.  But it
also breaks with existing code.

On Thu, Jun 27, 2013 at 1:11 PM, Jim Schaad <[email-masked]> wrote:

> No, my motivation is to allow for a smaller version of the payload string.
>  In some cases quoting it will generate a smaller string than doing the
> base64 conversion on the string.
>
>
>

---

From: Mike Jones <[email-masked]>
Date: Thu, 27 Jun 2013 23:16:14 +0000

Even if we chose to support an unencoded payload representation as an option for the JSON Serialization (which I'm not advocating), having this be the default would be the wrong thing for two reasons:

1.  It won't work for the Compact Serialization, since it doesn't ensure that the representation is URL-safe.  The default should work for both serializations.

2.  It assumes a special case (a payload that can be represented as a transport-invariant JSON string), which will often not work.  The default should be the general case, which always works.

As for your suggestion to not use the base64url encoded payload in the signature computation, this seems to me like an attempt to reopen http://trac.tools.ietf.org/wg/jose/trac/ticket/23 (Make crypto independent of binary encoding (base64)) by another means, despite it already being resolved as "Won't Fix".  I don't believe that constantly trying to revisit closed issues is a productive activity, nor respectful of the working group's time, so I therefore request that you let that one go.

                                                                Sincerely,
                                                                -- Mike

From: Richard Barnes [mailto:[email-masked]]
Sent: Thursday, June 27, 2013 3:24 PM
To: Jim Schaad
Cc: Mike Jones; n-sakimura; [email-masked]; [email-masked]
Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded

I think there are two separable issues here, both of which Mike and I probably have divergent opinions on :)
1. Syntax: Should it be possible for the "payload" field JWS to be a JSON string (instead of base64) when the payload is simply a UTF-8 string?
2. Crypto: Must the JWS Signing Input be base64url encoded.

The answers should be:
1. The payload should not be base64url encoded unless it cannot be represented as a JSON string.
2. The JWS Signing Input should not be base64url encoded unless there is a JWS Protected Header

Neither of these are complicated changes:
1. Add a "b64" header parameter that indicates that the payload is base64url-encoded binary, as opposed to a UTF-8 string.  Specify that this parameter is on by default in compact-formatted JWS objects.
2. Modify the signing/verification instructions to switch based on the presence of a JWS Protected Header: "If the JWS Protected Header is absent or empty, then the JWS Signing Input is simply the JWS Payload (not encoded).  If the JWS Protected Header has a non-empty value, then the the JWS Signing Input is the concatenation of the Encoded JWS Header, a period ('.') character, and the Encoded JWS Payload"

There are two reasons for these, both of which are important to make sure that this spec is applicable to a broad variety of use cases:
1. Compactness.  As Jim notes, quoted strings are shorter than base64url-encoded strings
2. Crypto-compatibility: Avoiding base64-encoding means that there's nothing JWS-specific about the signature value.  So for example, you could translate a JWS with no protected attributes to a CMS SignedData object with no protected attributes.

--Richard

[1] Note that the default value for this header must be false, or else you lose the compactness advantage!
[2] You could also do this as a "Content-Encoding" equivalent, "cenc": "b64".  That would give you "base64 agility", as some folks have asked for, since you could have "b64" and "b64u" values.
[3] You could also do this like CMS, and have the JWS Signing Input in the latter case be "the concatenation of the Encoded JWS Header, a period ('.') character, and the Encoded JWS Payload Hash".  Where the Encoded JWS Payload Hash is the base64url-encoded version of the hash you would otherwise be signing.  This seems simpler to code, since you just digest the body, then if there's a protected header, you digest again.  But it also breaks with existing code.

On Thu, Jun 27, 2013 at 1:11 PM, Jim Schaad <[email-masked]<mailto:[email-masked]>> wrote:
No, my motivation is to allow for a smaller version of the payload string.  In some cases quoting it will generate a smaller string than doing the base64 conversion on the string.

> From: Mike Jones [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
> From: Jim Schaad [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>; Mike Jones;
> [email-masked]<mailto:[email-masked]>
> > From: n-sakimura [mailto:[email-masked]<mailto:[email-masked]>]
> > Cc: [email-masked]<mailto:[email-masked]>;
> > [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
> > Nat Sakimura ([email-masked]<mailto:[email-masked]>)

_______________________________________________
jose mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jose

---

From: Dick Hardt <[email-masked]>
Date: Thu, 27 Jun 2013 16:56:29 -0700

> string (instead of base64) when the payload is simply a UTF-8 string?

One of the goals of the token was that it not require encoding for it to be
transported in a request. IE that it be safe and SIMPLE to include in an
HTTP header or in a URL

> represented as a JSON string.

How does an implementation determine which parts are separated? Are '.'
escaped? Does the implementation need to parse the JSON to determine a
valid separator? How does it know the end of the JSON? That sounds really
hard to implement, or I am misunderstanding what you are proposing.

> parameter is on by default in compact-formatted JWS objects.

And how is the header separated from the payload?

> base64url-encoded strings

The tokens are already compact enough.

> with no protected attributes.

Not a useful feature from my point of view.

-- Dick

---

From: Edmund Jay <[email-masked]>
Date: Thu, 27 Jun 2013 17:01:35 -0700 (PDT)

I prefer keeping the base64url encoded payload as that will result in simpler code paths and better interoperability.
As Mike pointed out, there could be problems with compact serialization if you turn b64 off.

________________________________
 From: Mike Jones <[email-masked]>
To: Richard Barnes <[email-masked]>; Jim Schaad <[email-masked]> 
Cc: n-sakimura <[email-masked]>; "[email-masked]" <[email-masked]>; "[email-masked]" <[email-masked]> 
Sent: Thursday, June 27, 2013 4:16 PM
Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded

Even if we chose to support an unencoded payload representation as an option for the JSON Serialization (which I’m not advocating), having this be the default would be the wrong thing for two reasons:
 
1.  It won’t work for the Compact Serialization, since it doesn’t ensure that the representation is URL-safe.  The default should work for both serializations.

As for your suggestion to not use the base64url encoded payload in the signature computation, this seems to me like an attempt to reopen http://trac.tools.ietf.org/wg/jose/trac/ticket/23 (Make crypto independent of binary encoding (base64)) by another means, despite it already being resolved as “Won’t Fix”.  I don’t believe that constantly trying to revisit closed issues is a productive activity, nor respectful of the working group’s time, so I therefore request that you let that one go.
 
                                                                Sincerely,
                                                                -- Mike
 
From:Richard Barnes [mailto:[email-masked]] 
Sent: Thursday, June 27, 2013 3:24 PM
To: Jim Schaad
Cc: Mike Jones; n-sakimura; [email-masked]; [email-masked]
Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded

 
The answers should be:

 
 
--Richard

---

From: Matias Woloski <[email-masked]>
Date: Thu, 27 Jun 2013 21:21:05 -0300

+1. KISS.

On Thu, Jun 27, 2013 at 9:01 PM, Edmund Jay <[email-masked]> wrote:

> I prefer keeping the base64url encoded payload as that will result in
> simpler code paths and better interoperability.
>   ------------------------------
>  *From:* Mike Jones <[email-masked]>
> *To:* Richard Barnes <[email-masked]>; Jim Schaad <[email-masked]>
> *Cc:* n-sakimura <[email-masked]>; "[email-masked]" <[email-masked]>;
> "[email-masked]" <
> [email-masked]>
> *Sent:* Thursday, June 27, 2013 4:16 PM
>
> *Subject:* Re: [jose] #26: Allow for signature payload to not be base64
> encoded
>
>  Even if we chose to support an unencoded payload representation as an
> option for the JSON Serialization (which I’m not advocating), having this
> be the default would be the wrong thing for two reasons:
>
> 1.  It won’t work for the Compact Serialization, since it doesn’t ensure
> that the representation is URL-safe.  The default should work for both
> serializations.
>
> 2.  It assumes a special case (a payload that can be represented as a
> transport-invariant JSON string), which will often not work.  The default
> should be the general case, which always works.
>
> As for your suggestion to not use the base64url encoded payload in the
> signature computation, this seems to me like an attempt to reopen
> http://trac.tools.ietf.org/wg/jose/trac/ticket/23 (Make crypto
> independent of binary encoding (base64)) by another means, despite it
> already being resolved as “Won’t Fix”.  I don’t believe that constantly
> trying to revisit closed issues is a productive activity, nor respectful of
> the working group’s time, so I therefore request that you let that one go.
>
>                                                                 Sincerely,
>                                                                 -- Mike
>
> *From:* Richard Barnes [mailto:[email-masked]]
> *Sent:* Thursday, June 27, 2013 3:24 PM
> *To:* Jim Schaad
> *Cc:* Mike Jones; n-sakimura; [email-masked];
> [email-masked]
> *Subject:* Re: [jose] #26: Allow for signature payload to not be base64
> encoded
>
>  1. Syntax: Should it be possible for the "payload" field JWS to be a
> JSON string (instead of base64) when the payload is simply a UTF-8 string?
>  There are two reasons for these, both of which are important to make
> sure that this spec is applicable to a broad variety of use cases:
>  [3] You could also do this like CMS, and have the JWS Signing Input in
> the latter case be "the concatenation of the Encoded JWS Header, a period
> ('.') character, and the Encoded JWS Payload Hash".  Where the Encoded JWS

---

From: Richard Barnes <[email-masked]>
Date: Fri, 28 Jun 2013 18:52:55 -0400

On Thu, Jun 27, 2013 at 7:16 PM, Mike Jones <[email-masked]>wrote:

> be the default would be the wrong thing for two reasons:****
>
> ** **
>

It does work for the compact serialization; you just declare that the
default for the compact serialization is "b64": true.  That is, you declare
the compact serialization equivalent to the following JSON object:

{
  "unprotected": { "b64": true },
  "protected": /* JWS Protected Header */,
  "payload": /* JWS Payload */,
  "signature": /* JWS Signature */
}

> should be the general case, which always works.

It assumes that the protected content can be represented as a UTF-8 string.
 Which is true of some large fraction of the traffic on the Internet.

> the working group’s time, so I therefore request that you let that one go.

That was closed because the proposed solution would have broken legacy
code.  The current proposal does not, because it only changes the process
when there is no protected header, which can only happen in the JSON
serialization.  You're trying to inflict the limitations of the compact
format on the JSON format.

--Richard

> ****
> ** **
> ****
>                                                                 -- Mike***
> *
> ** **
> encoded****
> ** **
> probably have divergent opinions on :)****
>
> string (instead of base64) when the payload is simply a UTF-8 string?****
>
> 2. Crypto: Must the JWS Signing Input be base64url encoded.****
>
> ** **
>
> The current answers to these questions are:****
>
> 1. It is not possible.  Payload is always base64url encoded.****
>
> 2. It is always base64url encoded.****
>
> ** **
>
> The answers should be:****
>
> represented as a JSON string.****
>
> JWS Protected Header****
> ** **
> Neither of these are complicated changes:****
>
> parameter is on by default in compact-formatted JWS objects.****
>
> ('.') character, and the Encoded JWS Payload"****
>
> ** **
>
> that this spec is applicable to a broad variety of use cases:****
>
> base64url-encoded strings****
>
> with no protected attributes.****
>
> ** **
>
> This email brought to you by the number 2 and the letters J, W, and S :)**
> **
>
> ** **
>
> --Richard****
>
> ** **
>
> lose the compactness advantage!****
>
> since you could have "b64" and "b64u" values.  ****
>
> also breaks with existing code.   ****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> On Thu, Jun 27, 2013 at 1:11 PM, Jim Schaad <[email-masked]>
> wrote:****
>
> base64 conversion on the string.****
>
>
>
>
> > From: Mike Jones [mailto:[email-masked]]****
>
> https://www.ietf.org/mailman/listinfo/jose****
>
> ** **
>

---

From: Richard Barnes <[email-masked]>
Date: Fri, 28 Jun 2013 18:57:48 -0400

Hey Dick,

I guess I should have been clearer that I was only talking about the JSON
serialization.  My proposed change would have no impact on the compact
serialization.  Hopefully that answers your questions about separators,
etc.; they payload would just be a normal JSON string.

--Richard

On Thu, Jun 27, 2013 at 7:56 PM, Dick Hardt <[email-masked]> wrote:

> One of the goals of the token was that it not require encoding for it to
> be transported in a request. IE that it be safe and SIMPLE to include in an
> HTTP header or in a URL
>
>

---

From: Dick Hardt <[email-masked]>
Date: Fri, 28 Jun 2013 16:04:26 -0700

Hi Richard, that helps.

Our you looking at a way of effectively signing a JSON payload? A complete,
real world example would help alot.

---

From: Mike Jones <[email-masked]>
Date: Sat, 29 Jun 2013 10:14:47 +0000

In the spirit of compromise, and to help further the discussion on this topic, I thought I'd suggest a possible syntax for the JWS JSON Serialization for representing an unencoded payload as a JSON string.  One way of allowing either encoded or unencoded payloads would be to use different JSON member names for them.  For instance, one could allow either of these names, but not both, in any particular JWS using the JSON Serialization:

    "payload" - Contains base64url encoded JWS Payload representation
    "verbatim" - Contains unencoded JWS Payload representation

So then, for instance, these three alternative representations would all represent the same JWS Payload:
    "payload": "SGVsbG8gd29ybGQ"
    "verbatim": "Hello world"
    "verbatim": "Hello\u0020world"

As for the signature computation, in the "payload", case there would be no change from the present.  In the "verbatim" case, I assume that the logical thing to do would be to use the octets of the UTF8 representation of the payload string in the signing input - in this example, 72 101 108 108 111 32 119 111 114 108 100.

This approach would avoid the problem I had discussed earlier of there being a default that is different than the Compact Serialization.  There would be no default.  Those encoding the content would choose either the "payload" or "verbatim" member name.

This does not avoid the problem of there being two different ways to do the same thing in the standard, with the interoperability problems that can result.

I am still not advocating that we support two ways of representing payloads.  But if the working group does decide that it's important to do so, I think that this would be a credible approach for doing so.  I think it would accomplish what Jim is after here.

Thoughts?

                                                            -- Mike

From: [email-masked] [mailto:[email-masked]] On Behalf Of Dick Hardt
Sent: Friday, June 28, 2013 4:04 PM
To: Richard Barnes
Cc: Jim Schaad; n-sakimura; Mike Jones; [email-masked]; [email-masked]
Subject: Re: [jose] #26: Allow for signature payload to not be base64 encoded

Hi Richard, that helps.

Our you looking at a way of effectively signing a JSON payload? A complete, real world example would help alot.

On Fri, Jun 28, 2013 at 3:57 PM, Richard Barnes <[email-masked]<mailto:[email-masked]>> wrote:
Hey Dick,

I guess I should have been clearer that I was only talking about the JSON serialization.  My proposed change would have no impact on the compact serialization.  Hopefully that answers your questions about separators, etc.; they payload would just be a normal JSON string.

--Richard

On Thu, Jun 27, 2013 at 7:56 PM, Dick Hardt <[email-masked]<mailto:[email-masked]>> wrote:

On Thu, Jun 27, 2013 at 3:24 PM, Richard Barnes <[email-masked]<mailto:[email-masked]>> wrote:
I think there are two separable issues here, both of which Mike and I probably have divergent opinions on :)
1. Syntax: Should it be possible for the "payload" field JWS to be a JSON string (instead of base64) when the payload is simply a UTF-8 string?

One of the goals of the token was that it not require encoding for it to be transported in a request. IE that it be safe and SIMPLE to include in an HTTP header or in a URL

The answers should be:

How does an implementation determine which parts are separated? Are '.' escaped? Does the implementation need to parse the JSON to determine a valid separator? How does it know the end of the JSON? That sounds really hard to implement, or I am misunderstanding what you are proposing.

-- Dick