Subject: Re: [Cbor] 🔔 WGLC on draft-ietf-cbor-cddl

From: Brian E Carpenter <[email-masked]>
Date: Fri, 2 Mar 2018 08:23:22 +1300

Hi,

As far as I'm concerned this draft is ready. That is a pragmatic view;
it works for my requirements, but there are many features I haven't
used.

Regards
   Brian Carpenter

On 27/02/2018 21:28, Francesca Palombini wrote:
> Dear CBOR WG,
> 
> As discussed during the past IETF meetings, and after the last updates, the CDDL draft is ready for WGLC. 
> 
> https://tools.ietf.org/html/draft-ietf-cbor-cddl-02 
> 
> The WGLC will end by *13th of March*, please make sure to send your comments to the list before then. 
> 
> Best regards,
> Francesca & Joe
> 
>> -----Original Message-----
>> From: CBOR [mailto:[email-masked]] On Behalf Of Carsten Bormann
>> Sent: den 26 februari 2018 21:46
>> To: [email-masked]
>> Subject: [Cbor] Ready for WGLC: draft-ietf-cbor-cddl-02.txt
>>
>> We have submitted draft-ietf-cbor-cddl-02.txt:
>>
>> https://tools.ietf.org/html/draft-ietf-cbor-cddl-02
>>
>> Diff from previous version at:
>>
>> https://tools.ietf.org/rfcdiff?url2=draft-ietf-cbor-cddl-02.txt
>>
>> We believe that this is now ready for WGLC.
>> (If we can start it quickly, the results of the call will be in for discussion in
>> London.)
>>
>> Grüße, Carsten
>>
>> _______________________________________________
>> CBOR mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/cbor
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor
>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Thu, 01 Mar 2018 21:47:37 +0000

I haven't re-reviewed this draft fully, but I think an issue I'd sent
directly to Carsten is still present: if a map key type is specified with
both a cut and an occurrence indicator, and the occurrence indicator is
saturated, do subsequent keys matching the type fall back to later entries?
For example, does:

{
  "a": 3
  "b": "cow"
}

match

{
  ? tstr ^ => int
  any => any
}

? I think it shouldn't, but I don't see a collection of sentences in the
specification that specifies it.

Jeffrey

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 3 Mar 2018 12:06:52 +0400

(typing this from a mobile device)

Hi Jeffrey,

I do believe that the current matching rules indeed saturate the optional group entry, so the cut does not apply to any additional members.  I think that is exactly as it should be. The requirements you seem to be expressing are probably better solved by limiting the wildcard instead. Can you be more specific about your use case?

Sent from mobile

> On 2. Mar 2018, at 01:47, Jeffrey Yasskin <[email-masked]> wrote:
> 
> I haven't re-reviewed this draft fully, but I think an issue I'd sent directly to Carsten is still present: if a map key type is specified with both a cut and an occurrence indicator, and the occurrence indicator is saturated, do subsequent keys matching the type fall back to later entries? For example, does:
> 
> {
>   "a": 3
>   "b": "cow"
> }
> 
> match
> 
> {
>   ? tstr ^ => int
>   any => any
> }
> 
> ? I think it shouldn't, but I don't see a collection of sentences in the specification that specifies it.
> 
> Jeffrey
> 
> 
> 
>> On Tue, Feb 27, 2018 at 12:28 AM Francesca Palombini <[email-masked]> wrote:
>> Dear CBOR WG,
>>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Sat, 03 Mar 2018 16:13:14 +0000

I don't have a particular use case in mind, so if you prefer to ignore the
cut after its occurrences are saturated, that's fine. Which sentences in
the specification specify that?

Thanks,
Jeffrey

On Sat, Mar 3, 2018, 12:07 AM Carsten Bormann <[email-masked]> wrote:

> I do believe that the current matching rules indeed saturate the optional
> group entry, so the cut does not apply to any additional members.  I think
> that is exactly as it should be. The requirements you seem to be expressing
> are probably better solved by limiting the wildcard instead. Can you be
> more specific about your use case?
>
> Sent from mobile
>

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 4 Mar 2018 06:04:29 +0400

I can't point to a specific sentence, but for a cut to be effective, the match needs to occur first, and in this case the whole entry says that can only happen once. 

Sent from mobile

> On 3. Mar 2018, at 20:13, Jeffrey Yasskin <[email-masked]> wrote:
> 
> I don't have a particular use case in mind, so if you prefer to ignore the cut after its occurrences are saturated, that's fine. Which sentences in the specification specify that?
> 
> Thanks,
> Jeffrey
>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Tue, 13 Mar 2018 23:46:47 +0000

I've gotten criticism for using the base CBOR specification in a web
specification because it's not precise enough about when there's a parse
failure (
https://github.com/mozilla/standards-positions/issues/29#issuecomment-334376573),
so I'm extra sensitive to imprecision in CDDL. I wouldn't want an initial
CDDL RFC that we have to change a lot before web folks are comfortable
depending on it.

For occurrences, the specification is in
https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#page-39 (
https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-3.2 does not
contain any statement saying the number of times a group needs to or can
match, except for the special case of "no occurrence indicator"):

> An occurrence indicator modifies the group given to its right by
> requiring the group to match the sequence to be matched exactly for a
> certain number of times (see Section 3.2) in sequence

This specification of occurrence indicators does *not* prevent the group
from matching a subsequence of items; it just says that if the group
matches the wrong number of times, the requirement isn't met. If the
specification intends to describe a more imperative algorithm where groups
stop matching after seeing some sequence elements, it needs to say so, but
unless this is done carefully, it probably leads to dependencies on the
order elements appear in a map.

https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-3.5.3 says

> once the map key matches the entry
> carrying the cut, other potential matches for the key that occur in
> later entries in the group of the map are no longer allowed.

This describes map *keys* matching whole entries ("key + value"s), which
doesn't make a lot of sense. I've been interpreting it as just matching
against the entry's key (but that doesn't work for group entries consisting
of subgroups...), and then it's pretty clear that { "a": 3, "b": 3} has two
entries matching the first group element of { ? tstr ^ => int, any => any
}, and then the occurrence indicator says that match doesn't fit the
requirements.

Jeffrey

On Sat, Mar 3, 2018 at 6:04 PM Carsten Bormann <[email-masked]> wrote:

> I can't point to a specific sentence, but for a cut to be effective, the
> match needs to occur first, and in this case the whole entry says that can
> only happen once.
>
> Sent from mobile
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 14 Mar 2018 07:27:35 +0100

On Mar 14, 2018, at 00:46, Jeffrey Yasskin <[email-masked]> wrote:
> 
>  it's not precise enough about when there's a parse failure (https://github.com/mozilla/standards-positions/issues/29#issuecomment-334376573),

Well, hsivonen is wrong here.  It’s not that “anything goes” in CBOR, and I have no idea how to read Section 3.3 of RFC 7049 (which is linked in the above) that way.  Section 3.3 instead indicates specific error cases.  Yes, it does allow robust parsing, but not without "using a decoder-specific convention that clearly
   indicates there has been a problem”.  If you think there is a need for improving the language here to better represent the intent, I’d sure like to know.

I’ll look at the specification language issues you point out in the CDDL spec later today.

Grüße, Carsten

---

From: Jeffrey Yasskin <[email-masked]>
Date: Wed, 14 Mar 2018 17:08:21 +0000

On Tue, Mar 13, 2018 at 11:27 PM Carsten Bormann <[email-masked]> wrote:

> >  it's not precise enough about when there's a parse failure (
> Well, hsivonen is wrong here.  It’s not that “anything goes” in CBOR, and
> I have no idea how to read Section 3.3 of RFC 7049 (which is linked in the
> above) that way.  Section 3.3 instead indicates specific error cases.  Yes,
> it does allow robust parsing, but not without "using a decoder-specific
> convention that clearly
>    indicates there has been a problem”.  If you think there is a need for
> improving the language here to better represent the intent, I’d sure like
> to know.
>

I filed a similar issue in https://github.com/cbor-wg/CBORbis/issues/3, and
I'm currently working on breaking up the changes in
https://github.com/cbor-wg/CBORbis/compare/master...jyasskin:editorial-improvements
into manageable PRs.

> I’ll look at the specification language issues you point out in the CDDL

Thanks.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 19 Mar 2018 20:29:49 +0000

> For occurrences, the specification is in https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#page-39 (https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-3.2 does not contain any statement saying the number of times a group needs to or can match, except for the special case of “no occurrence indicator"):

The first paragraph of Section 3.2 seems pretty complete to me.  What is missing?

> This specification of occurrence indicators does *not* prevent the group from matching a subsequence of items; it just says that if the group matches the wrong number of times, the requirement isn't met. If the specification intends to describe a more imperative algorithm where groups stop matching after seeing some sequence elements, it needs to say so, but unless this is done carefully, it probably leads to dependencies on the order elements appear in a map.

There is no dependence on the order in which key/value pairs (members) appear in a CBOR map.

> > later entries in the group of the map are no longer allowed.

Right.

> This describes map *keys* matching whole entries (“key + value"s), which doesn't make a lot of sense.

Maybe I’m too tired, but I don’t see that right now.

> I've been interpreting it as just matching against the entry's key (but that doesn't work for group entries consisting of subgroups...), and then it's pretty clear that { "a": 3, "b": 3} has two entries matching the first group element of { ? tstr ^ => int, any => any }, and then the occurrence indicator says that match doesn’t fit the requirements.

? tstr ^ => int, is looking for a member with a text key and an int value.  
Non-deterministically, it matches either member of the above map.
(So the cut has no effect, as there is no failure.)
The occurrence indicator is satisfied.
The other member is then matched by the any => any.

Grüße, Carsten

---

From: Jeffrey Yasskin <[email-masked]>
Date: Tue, 20 Mar 2018 13:22:40 +0000

On Mon, Mar 19, 2018 at 8:29 PM Carsten Bormann <[email-masked]> wrote:

> match, except for the special case of “no occurrence indicator"):
>

Right now, "An optional _occurrence_ indicator can be given in front of a
group entry.  It is either one of the characters '?' (optional), '*' (zero
or more), or '+' (one or more), or is of the form n*m, where n and m are
optional unsigned integers and n is the lower limit (default 0) and m is
the upper limit (default no limit) of occurrences." successfully describes
the meaning of an occurrence indicator, but doesn't tie that to the meaning
of group entries at all. (Saying that they appear next to each other
doesn't specify that their meanings are connected.)

Perhaps, "A group matches a collection of CBOR items if and only if each
group entry can be repeated a number of times allowed by its occurrence
indicator, and then paired 1-1 with a CBOR item (in an array) or key/value
pair of CBOR items (in a map) that matches it." This doesn't have the
semantics you want for cuts, though, so you'll have to write a different
specification.

I suspect this specification also implies an exponential algorithm for
checking map items, but I haven't proven that, and it's not obvious whether
that should disqualify it for the RFC.

> There is no dependence on the order in which key/value pairs (members)

I agree this is a requirement on whatever specification text the RFC
eventually contains.

> Maybe I’m too tired, but I don’t see that right now.

"the map key matches the entry" is impossible because entries include at
least both a key and a value, and if the entry is a group, several keys and
values. Unless you can point to a sentence in the specification that
describes how key items match group entries? You or me having intuition
about it isn't sufficient for an RFC.

> > I've been interpreting it as just matching against the entry's key (but
> that doesn't work for group entries consisting of subgroups...), and then
> it's pretty clear that { "a": 3, "b": 3} has two entries matching the first
> group element of { ? tstr ^ => int, any => any }, and then the occurrence
> indicator says that match doesn’t fit the requirements.
>

The occurrence semantics I wrote above does cause { ? tstr ^ => int, any =>
any } to match { "a": 3, "b": 3} because we rewrite the rule to { tstr ^ =>
int, any => any }, and then the entries match 1-1. For this, the definition
of a map cut would need to explicitly say that once an entry with a cut has
been matched, it doesn't affect further matches.

However, the above group semantics makes a cut on a *'ed entry stop working
entirely:

{*tstr => int, *any => any} matches {"a": "b"} because we rewrite the rule
to {any => any}, which matches. {? a: int, *any => any} also matches {"a":
"q"}.

Jeffrey

Grüße, Carsten
> >> I don't have a particular use case in mind, so if you prefer to ignore
> the cut after its occurrences are saturated, that's fine. Which sentences
> in the specification specify that?
> >>
> >> Thanks,
> >> Jeffrey
> >>
> >> I do believe that the current matching rules indeed saturate the
> optional group entry, so the cut does not apply to any additional members.
> I think that is exactly as it should be. The requirements you seem to be
> expressing are probably better solved by limiting the wildcard instead. Can
> you be more specific about your use case?
> >>
> >> Sent from mobile
> >>
> >>> As discussed during the past IETF meetings, and after the last
> updates, the CDDL draft is ready for WGLC.
> >>>