Subject: Re: [art] [dispatch] [Secdispatch] Plain text JSON digital signatures

From: Samuel Erdtman <[email-masked]>
Date: Sat, 1 May 2021 23:24:23 +0200

Thanks Carsten,

Your clarifications were great!

See some comments inline.

On Sat, May 1, 2021 at 1:04 AM Carsten Bormann <[email-masked]> wrote:

> Hi Samuel,
>
> > 1. What do you mean with data at rest, data store in database or file?
>
> The point is that you are not signing the data being transferred, but a
> local copy of some (e.g., freshly decoded) data (i.e., at the data model
> level) which is then processed a little (potentially taking out all
> signatures) and then is run through a rather complicated engine to produce
> a signing input that is a deterministic function of the decoded and
> processed data.
>
> There are easier ways to get a signing input from JSON-like data at rest.
> For a (quite workable) strawman: How about doing a CBOR encoding using
> deterministic encoding rules?
>

This is a good point. Still not convinced other solutions would be orders
of magnitude better and the proposed one is easy to implement and easy to
understand (maybe too easy since I had not thought about your alternative
solution).

>
> > Or is it data that does not change? Sorry I do not get it.
> >
> > 2. What is weird with saying "Represented in JSON”?
>
> Your scheme does NOT require (or benefit in any way from) representing the
> data in JSON.
> The data could be transferred in YAML (or CBOR for that matter): as long
> as your local copy of the decoded data (after the little processing) sticks
> inside the confines of the I-JSON data model, you can use your scheme for
> signing.
>

But since JSON according to https://tools.ietf.org/html/rfc7159 is fairly
flexible, would not fitting it into CBOR require similar limitations as
RFC8785 puts on what you can do with the JSON? (i.e. the I-JSON limitations)

>
> > is it that the RFC7493 I-JSON subset is not all that JSON could be? (to
> me this is a reasonable limitation that I in practice never have had to go
> outside)
>
> Well, that has been debated to death, and it is clear that nobody likes
> I-JSON (*), but it is the de-facto boundary within which the actually more
> capable JSON format needs to be used these days.
> (If you need more flexibility, you know where to find CBOR.)
>

So are you saying that CBOR would not impose the same limitations on the
original JSON as RFC8785?

>
> > 3. So I totally get that one does not like XMLDigSig, in my opinion not
> because of the signing procedures but because of the canonicalization
> process. When I looked at it I gave up and created a hardcoded template.
> The difference in canonicalization of JSON (RFC7493 I-JSON subset)
> according to RFC8785 is like night and day compared to XML
> canonicalization. In your comment it seems like you are of the opinion that
> this effort will be as tricky as XMLDigSig. do you think so?
>
> Indeed, the RFC 8785 encoding is (ignoring potential problems on the
> numeric side) simpler than canonicalized XML, even with its weird
> regression to UTF16-land.
>
> Much of the actual problems of XMLDSig weren’t in the canonicalization,
> but in the confusion of how the data at rest was to be processed for
> signing, e.g., what part of the data at rest was contributing to the
> signing input and what the signature on that part then actually meant.  JWS
> is probably flexible enough that a carefully constructed application can
> get all this right, but we are talking about non-trivial specifications
> needed beyond the boring part of generating the byte-string signing input.
>
> > 4. Not sure I agree with “clear text” or “plain text" being bad
> descriptions. Yes what is signed is the RFC8785 transformation of the input
> data, but the signature are then put into your data keeping the data in its
> original “clear text” or “plain text" as opposed to base64-url encoded. I
> guess enveloped JWS is a more accurate name but “clear text” or “plain
> text" is easy to understand.
>
> Well, I understand that the naming you chose is a good strategy for
> selling the scheme.
> It is, however, not describing what is actually going on, and I try to
> minimize the use of misleading terminology.
>

Fair point

>
> Grüße, Carsten
>
> (*) Over at the JSON mailing list, there has been some fresh discussion
> just this week about how to get around some of the implementation
> limitations, or JSON’s (deliberate!) lack of extensibility, or both.
> Archived-At: <
> https://mailarchive.ietf.org/arch/msg/json/BWkSc8JYybzmgLT0Bsmfhiwf7cY>
> and the thread behind that.
>
>

---

From: Samuel Erdtman <[email-masked]>
Date: Sat, 1 May 2021 23:28:27 +0200

Hi Carsten,

One more thing, you wrote "Signing data at rest certainly is a use case
that is worth addressing.". With my new fond insights (thanks) does this
mean that you are in favor of specifying how to do enveloped signatures for
JSON (or at least not against it)?

Cheers
//Samuel

On Sat, May 1, 2021 at 11:24 PM Samuel Erdtman <[email-masked]> wrote:

>> Your scheme does NOT require (or benefit in any way from) representing
>> the data in JSON.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 3 May 2021 17:37:38 +0200

On 2021-05-01, at 23:28, Samuel Erdtman <[email-masked]> wrote:
> 
> Hi Carsten,
> 
> One more thing, you wrote "Signing data at rest certainly is a use case that is worth addressing.". With my new fond insights (thanks) does this mean that you are in favor of specifying how to do enveloped signatures for JSON (or at least not against it)? 

Signing data at rest doesn’t mean that the signature then needs to be put into the signed object.  This is essentially adulterating that object, and is part of the XMLDSig “what was just signed?” practicability issue.
(Such as, does my countersignature include the previous signature or not?
What does it mean to sign a signed object?  Nothing about the existing signature, or does it mean I endorse the existing signature, or does it mean my signature actually is conditional on the validity of that existing signature?)

I’m not sure though I know what an “enveloped signature” is, and whether what I just said above even replies to what I asked.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 3 May 2021 18:06:31 +0200

Hi Samuel,

I haven’t responded to your message yet as it triggers some MacOS mail misbehavior.
Let me try anyway, I hope you can parse the result.

> On 2021-05-01, at 23:24, Samuel Erdtman <[email-masked]> wrote:
> 
>> Thanks Carsten,
>> 
>> The point is that you are not signing the data being transferred, but a local copy of some (e.g., freshly decoded) data (i.e., at the data model level) which is then processed a little (potentially taking out all signatures) and then is run through a rather complicated engine to produce a signing input that is a deterministic function of the decoded and processed data.
>> 
>> For a (quite workable) strawman: How about doing a CBOR encoding using deterministic encoding rules?
> 
> This is a good point. Still not convinced other solutions would be orders of magnitude better and the proposed one is easy to implement and easy to understand (maybe too easy since I had not thought about your alternative solution). 

It is hard to measure interoperability in orders of magnitude…
Doing a CBOR signing input is easy to implement and easy to understand, though.
No UTF-16 needed :-)

>> Your scheme does NOT require (or benefit in any way from) representing the data in JSON.
>> The data could be transferred in YAML (or CBOR for that matter): as long as your local copy of the decoded data (after the little processing) sticks inside the confines of the I-JSON data model, you can use your scheme for signing.
>> 
> But since JSON according to https://tools.ietf.org/html/rfc7159 is fairly flexible, would not fitting it into CBOR require similar limitations as RFC8785 puts on what you can do with the JSON? (i.e. the I-JSON limitations)

It does require some thinking.
https://www.rfc-editor.org/rfc/rfc8949.html#section-6.2 documents what we arrived at when discussing that in the CBOR WG.

(I’m very well aware that many applications would be content with just I-JSON, because they already have made their peace with it to enable JavaScript classic processing.  So I’m just answering the question here.)

>> > is it that the RFC7493 I-JSON subset is not all that JSON could be? (to me this is a reasonable limitation that I in practice never have had to go outside)
>> 
>> Well, that has been debated to death, and it is clear that nobody likes I-JSON (*), but it is the de-facto boundary within which the actually more capable JSON format needs to be used these days.
> So are you saying that CBOR would not impose the same limitations on the original JSON as RFC8785? 

Yes.  Again, please see https://www.rfc-editor.org/rfc/rfc8949.html#section-6.2

Please note that I’m not making these points to push CBOR, even though I’m really convinced CBOR and its “JSON mode” can play a helpful role in addressing the underlying requirements.

Grüße, Carsten

---

From: Samuel Erdtman <[email-masked]>
Date: Thu, 6 May 2021 23:00:31 +0200

Thanks again Carsten for sharing your thoughts

See comment inline

On Mon, May 3, 2021 at 5:37 PM Carsten Bormann <[email-masked]> wrote:

> Signing data at rest doesn’t mean that the signature then needs to be put
> into the signed object.  This is essentially adulterating that object, and
> is part of the XMLDSig “what was just signed?” practicability issue.
>

Agree the signature does not have to go into the signed document, but in my
opinion has its benefits compared to the alternatives. if puting the
signature somewhere else only referencing the signed document it would be
inconvenient to find the signature when needed (yes you could create a
system for that, there likely exists multiple of them all working slightly
differently). The alternative I see would be to create some structure that
would wrap the signed data and the signature but in that way it would not
be much different from the ascii armoring done by normal JWS, I would like
to cater for the use case where you want your doc unchanged apart from the
addition of the signature. (This is my opinion, and I´m okay with not
agreeing)

> What does it mean to sign a signed object?  Nothing about the existing
> signature, or does it mean I endorse the existing signature, or does it
> mean my signature actually is conditional on the validity of that existing
> signature?)
>

Yes there are questions that would need to be answered, but to me the
questions above would be application questions, what is the specific
application trying to achieve, all of the options would be possible to
technically construct.

>
> I’m not sure though I know what an “enveloped signature” is, and whether
> what I just said above even replies to what I asked.
>

With enveloped signature I was referring to when the document that is
signed will contain the signature opposed to enveloping signatures where
the signature encapsulated the document being signed. (maybe not an
established naming)

---

From: Samuel Erdtman <[email-masked]>
Date: Thu, 6 May 2021 23:16:48 +0200

Thanks Carsten,

There was not problem reading the reply :),

To summarize my understanding you have (at least) two issues with the
current doc https://datatracker.ietf.org/doc/draft-jordan-jws-ct/
* It uses RFC8785 to create the signature input which you do not like,
since it limits the expressiveness of JSON (and uses UTF-16), CBOR could be
an alternative (but you are not pushing for it)
* You do not like putting the signature into the signed document because it
can cause all kinds of confusion, with e.g. counter signatures.

Did I capture that correctly?

Cheers
//Samuel

On Mon, May 3, 2021 at 6:06 PM Carsten Bormann <[email-masked]> wrote:

> > This is a good point. Still not convinced other solutions would be
> orders of magnitude better and the proposed one is easy to implement and
> easy to understand (maybe too easy since I had not thought about your
> alternative solution).
>
> >> The data could be transferred in YAML (or CBOR for that matter): as
> long as your local copy of the decoded data (after the little processing)
> sticks inside the confines of the I-JSON data model, you can use your
> scheme for signing.
> >>
> > But since JSON according to https://tools.ietf.org/html/rfc7159 is
> fairly flexible, would not fitting it into CBOR require similar limitations
> as RFC8785 puts on what you can do with the JSON? (i.e. the I-JSON
> limitations)
>
> https://www.rfc-editor.org/rfc/rfc8949.html#section-6.2 documents what we
> arrived at when discussing that in the CBOR WG.
>
> (I’m very well aware that many applications would be content with just
> I-JSON, because they already have made their peace with it to enable
> JavaScript classic processing.  So I’m just answering the question here.)
>
> >> > is it that the RFC7493 I-JSON subset is not all that JSON could be?
> (to me this is a reasonable limitation that I in practice never have had to
> go outside)
> >>
> Yes.  Again, please see
> https://www.rfc-editor.org/rfc/rfc8949.html#section-6.2
>
> Please note that I’m not making these points to push CBOR, even though I’m
> really convinced CBOR and its “JSON mode” can play a helpful role in
> addressing the underlying requirements.
>
> Grüße, Carsten
>
>