Subject: [netconf] Require-instance problem [was: RE: WGLC: draft-ietf-netconf-notification-capabilities-11]

From: Balázs Lengyel <[email-masked]>
Date: Fri, 13 Mar 2020 08:55:04 +0000

Hello Mahesh,

Even if there is some problem with ietf-subscribed-notification how does that effect the validity of instance data for ietf-system-capabilities or ietf-notification-capabilities ? I only import a type and a feature from Yangpush (which imports subscribed notification). These seem nothing to do with require-instance.

Aside from my draft:

I do not understand some of libyangs statements:  

“as it turned out (ref <https://tools.ietf.org/html/rfc7950#section-9.9.3> ), you cannot put require-instance into a derived type”. 

Checking ref I did not find why you cannot put require instance into a typedef.

By the way I have bigger issues with YangValidator. When I load ietf-system-capabilities@2020-03-08 into it I get 

RecursionError at /yangvalidator/validator

maximum recursion depth exceeded

Request Method:

POST

Request URL:

http://www.yangvalidator.com/yangvalidator/validator

Django Version:

3.0.4

Exception Type:

RecursionError

Exception Value:

maximum recursion depth exceeded

Exception Location:

/home/yang/yangvalidator/lib/python3.6/site-packages/pyang-2.1.1-py3.6.egg/pyang/statements.py in newf, line 42

Python Executable:

/usr/sbin/uwsgi

Python Version:

3.6.10

Python Path:

['/home/yang/yangvalidator/lib/python3.6/site-packages/pyang-2.1.1-py3.6.egg/pyang/transforms',

Regards Balazs

From: Mahesh Jethanandani <[email-masked]> 
Sent: 2020. március 12., csütörtök 22:28
To: Balázs Lengyel <[email-masked]>
Cc: Kent Watsen <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [netconf] WGLC: draft-ietf-netconf-notification-capabilities-11

[Adding yang-doctors]

Hi Balazs,

On Mar 10, 2020, at 2:06 AM, Balázs Lengyel <[email-masked] <mailto:[email-masked]> > wrote:

Finally, have all the examples in the appendix been validated?

BALAZS: Yes, they have been loaded into a live Confd server.

I bring this up because I ran into issues when validating the https-notif model using ietf-subscribed-notification module, something that you also import. Here is what I see when I try to use yanglint to validate your example. In this case, I named your example as examples-notification-capabilities-1.xml.

bash-3.2$ yanglint -s -i -t auto -p /Volumes/External/git/iana/yang-parameters/ [email-masked] <mailto:[email-masked]>  [email-masked] <mailto:[email-masked]>  examples-notification-capabilities-1.xml 

err : Invalid keyword "require-instance".

err : Module "ietf-subscribed-notifications" parsing failed.

err : Importing "ietf-subscribed-notifications" module into "ietf-yang-push" failed.

err : Module "ietf-yang-push" parsing failed.

err : Importing "ietf-yang-push" module into "ietf-notification-capabilities" failed.

err : Module "ietf-notification-capabilities" parsing failed.

At first I thought the issue was with yanglint, because just like you I used confd and it did not complain. But folks over at libyang tell me that this was discussed as part of their issue  #881 <https://protect2.fireeye.com/v1/url?k=a6420c80-fa9605e6-a6424c1b-8610d8a762ca-d6cddc5edf1151a7&q=1&e=3cd8e8af-8aea-4f96-939b-8e1e455eac74&u=https%3A%2F%2Fgithub.com%2FCESNET%2Flibyang%2Fissues%2F881>  and the conclusion was that you cannot put require-instance in a derived type based on this <https://tools.ietf.org/html/rfc7950#section-9.9.3>  text.

Who is correct in their assertion, libyang or confd/RFC8639?

In case anyone is wondering which ietf-subsciribed-notifications module is being imported, here it is:

bash-3.2$ ls ../iana/yang-parameters/[email-masked] <mailto:iana/yang-parameters/[email-masked]> 

Cheers.

BTW, even if I comment out the ‘require-instance’ statement, yanglint complains about other issues with ietf-subscribed-notifications, which I will bring up in a separate thread.

Mahesh Jethanandani

[email-masked] <mailto:[email-masked]>

---

From: Michal Vaško <[email-masked]>
Date: Fri, 13 Mar 2020 10:04:16 +0100

Hi Balazs,
just to explain libyang behavior, if you look at (pattern <https://tools.ietf.org/html/rfc7950#section-9.4.5> ) definition, there is explicitly mentioned that it can appear in derived types. There is no such wording for (require-instance <https://tools.ietf.org/html/rfc7950#section-9.9.3> ), it mentions only "instance-identifier" and "leafref" types.

Also, libyang does not allow loading invalid schemas so even if you only import it, it will fail.

Regards,
Michal

On Friday, March 13, 2020 09:55 CET, Balázs Lengyel <balazs.lengyel=[email-masked]> wrote: 
 
> bash-3.2$ ls .../iana/yang-parameters/[email-masked] <mailto:iana/yang-parameters/[email-masked]> 
> 
>  
> 
> Cheers.
> 
>  
>

---

From: Balázs Lengyel <[email-masked]>
Date: Fri, 13 Mar 2020 09:40:25 +0000

Hello Michal,
While the Require-instance issue may or may not be a problem, it is not caused by my netconf-notification-capabilities draft. I hope we agree on this.

About the require-instance problem:
Not mentioning derived types for require-instance in 9.9.3 can be intentional or an omission. If I just strictly read the text:
> The "require-instance" statement, which is a substatement to the
>    "type" statement, MAY be present if the type is "instance-identifier"
>    or "leafref".
This says that the require-instance is allowed under type; full stop. Intentions about the pattern of the text are speculation. The text says it MAY be there. So unless an errata is filed, it is allowed. IMHO if that is the intention we would need an explicit sentence like:
"Usage of require-instance is not allowed for derived types."
I don't have a strict view on which was the intention (maybe Martin has) but the text is unambiguous to me.
Regards Balazs

-----Original Message-----
From: Michal Vaško <[email-masked]> 
Sent: 2020. március 13., péntek 10:04
To: Balázs Lengyel <[email-masked]>
Cc: Mahesh Jethanandani <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [netconf] Require-instance problem [was: RE: WGLC: draft-ietf-netconf-notification-capabilities-11]

Hi Balazs,
just to explain libyang behavior, if you look at (pattern <https://tools.ietf.org/html/rfc7950#section-9.4.5> ) definition, there is explicitly mentioned that it can appear in derived types. There is no such wording for (require-instance <https://tools.ietf.org/html/rfc7950#section-9..9.3> ), it mentions only "instance-identifier" and "leafref" types.

Regards,
Michal

On Friday, March 13, 2020 09:55 CET, Balázs Lengyel <balazs.lengyel=[email-masked]> wrote: 
 
> https://protect2.fireeye.com/v1/url?k=6cc7e6c7-3013efa1-6cc7a65c-8610d8a762ca-0a278d65531d4f4f&q=1&e=b7200729-5cfb-4d52-b459-737457b266e0&u=http%3A%2F%2Fwww.yangvalidator.com%2Fyangvalidator%2Fvalidator
> 
> 
> Django Version:
> 
> 3.0.4
> 
> 
> Exception Type:
> 
> RecursionError
> 
> 
> Exception Value:
>

---

From: Radek Krejci <[email-masked]>
Date: Mon, 16 Mar 2020 10:28:07 +0100

Hi Balasz,

Dne 13. 03. 20 v 10:40 Balázs Lengyel napsal(a):
> This says that the require-instance is allowed under type; full stop.

but there is no full stop, the sentence continues, please read it till
its end, it specifies conditions when the substatement can be used. Also
please compare with the text in 9.4.4 (length) or 9.2.4 (range) where
the derived types are explicitly mentioned in case the statement is
meant to be allowed there. The specification is not supposed to
explicitly state where it IS NOT allowed, instead it specifies where it
IS allowed. By your interpretation, require-instance would be allowed
even in integer types since it is not explicitly denied.

Regards,
Radek

>  Intentions about the pattern of the text are speculation. The text says it MAY be there. So unless an errata is filed, it is allowed. IMHO if that is the intention we would need an explicit sentence like:
>> bash-3.2$ ls ..../iana/yang-parameters/[email-masked] <mailto:iana/yang-parameters/[email-masked]> 
>>
>>  
>>
>> Cheers.
>>
>>  
>>
> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Martin Björklund <[email-masked]>
Date: Tue, 17 Mar 2020 17:48:22 +0100 (CET)

Hi,

Some background:

Some types in YANG can be restricted, e.g., an int32 can have a range
restriction, a string can have a length restriction etc.  When a type
A is restricted in this way, the new type B's value space is a subset
of A's value space.

Some types cannot be restricted, i.e., their value space cannot be
altered (e.g., "boolean").

Some other types need substatements to properly define the value
space, e.g., an "enumeration" needs a set of "enum" statements and a
"decimal64" needs a "fraction-digits".

The only way to expand the value space is to use type "union".

So in all cases, the derived type's value space is a subset of its
base type's value space.

Now, for instance-identifier and leafref, the "require-instance" is a
bit weird.  It is called a "restriction" but it isn't really a
restriction, since it doesn't change the value space.  If it is set to
"true" it defines a constraint that must be true in valid data.

Radek Krejci <[email-masked]> wrote:
> Also
> IS allowed.

Agreed.

> By your interpretation, require-instance would be allowed

I don't think this is the logical interpretation.  Suppose we have
two types "foo" and "bar":

   typedef foo {
     type leafref {
       path "...";
     }
   }

   typedef bar {
     type int32;
   }

It makes (some) logical sense to say that "foo" is a "leafref", and
hence it would be legal according to 9.9.3 to use "require-instance" in
"foo".

But it does not make logical sense to say that "bar" is a "leafref",
and hence 9.9.3 doesn't make it legal to use "require-instance" in
"bar".

I don't remember the original intention, but obviously people have
interpreted the text in 9.9.3 in different ways, which indicates that
it is wrong and should be fixed.  However, I don't think either
interpretation is obviously correct.  Thus, I think that we cannot
state that the usage of "require-instance" in
ietf-subscribed-notification is wrong.

/martin

---

From: Balázs Lengyel <[email-masked]>
Date: Tue, 17 Mar 2020 17:06:45 +0000

Hello Radek, Martin,
At this point I cannot imagine a reason why require-instance would be allowed on a locally defined type, but not in a typedef. Do you see any reason for such restriction?
Regards Balazs

-----Original Message-----
From: Martin Björklund <[email-masked]> 
Sent: 2020. március 17., kedd 17:48
To: [email-masked]
Cc: Balázs Lengyel <[email-masked]>; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [netconf] Require-instance problem

Hi,

Some background:

Some types in YANG can be restricted, e.g., an int32 can have a range restriction, a string can have a length restriction etc.  When a type A is restricted in this way, the new type B's value space is a subset of A's value space.

Some types cannot be restricted, i.e., their value space cannot be altered (e.g., "boolean").

Some other types need substatements to properly define the value space, e.g., an "enumeration" needs a set of "enum" statements and a "decimal64" needs a "fraction-digits".

So in all cases, the derived type's value space is a subset of its base type's value space.

Now, for instance-identifier and leafref, the "require-instance" is a bit weird.  It is called a "restriction" but it isn't really a restriction, since it doesn't change the value space.  If it is set to "true" it defines a constraint that must be true in valid data.

Radek Krejci <[email-masked]> wrote:
> explicitly state where it IS NOT allowed, instead it specifies where 
> it IS allowed.

Agreed.

> By your interpretation, require-instance would be allowed even in 
> integer types since it is not explicitly denied.

I don't think this is the logical interpretation.  Suppose we have two types "foo" and "bar":

   typedef foo {
     type leafref {
       path "...";
     }
   }

   typedef bar {
     type int32;
   }

It makes (some) logical sense to say that "foo" is a "leafref", and hence it would be legal according to 9.9.3 to use "require-instance" in "foo".

But it does not make logical sense to say that "bar" is a "leafref", and hence 9.9.3 doesn't make it legal to use "require-instance" in "bar".
BALAZS: 9.9.3 includes "MAY be present if the type is instance-identifier or leafref" 
Which to me clearly states this is not allowed for an integer.

I don't remember the original intention, but obviously people have interpreted the text in 9.9.3 in different ways, which indicates that it is wrong and should be fixed.  However, I don't think either interpretation is obviously correct.  Thus, I think that we cannot state that the usage of "require-instance" in ietf-subscribed-notification is wrong.

/martin

> > Subject: Re: [netconf] Require-instance problem [was: RE: WGLC: 
> > draft-ietf-netconf-notification-capabilities-11]
> >
> > Hi Balazs,
> >> By the way I have bigger issues with YangValidator. When I load 
> >> ietf-system-capabilities@2020-03-08 into it I get
> >>
> >>  
> >>
> >> https://protect2.fireeye.com/v1/url?k=6cc7e6c7-3013efa1-6cc7a65c-86
> >> 10d8a762ca-0a278d65531d4f4f&q=1&e=b7200729-5cfb-4d52-b459-737457b26
> >> 6e0&u=http%3A%2F%2Fwww.yangvalidator.com%2Fyangvalidator%2Fvalidato
> >> r
> >>
> >>
> >> Django Version:
> >>
> >> 3.0.4
> >>
> >>
> >> Exception Type:
> >>
> >> RecursionError
> >>
> >>
> >> Exception Value:
> >>
> >> /home/yang/yangvalidator/lib/python3.6/site-packages/pyang-2.1.1-py
> >> 3.6.egg/pyang/statements.py in newf, line 42
> >>
> >>
> >> Python Executable:
> >>
> >> /usr/sbin/uwsgi
> >>
> >>
> >> Python Version:
> >>
> >> 3.6.10
> >>
> >>
> >> Python Path:
> >>
> >> Subject: Re: [netconf] WGLC: 
> >> draft-ietf-netconf-notification-capabilities-11
> >>
> >>  
> >>
> >> [Adding yang-doctors]
> >>
> >>  
> >>
> >> Hi Balazs,
> >>
> >>  
> >>
> >>  
> >>
> >> bash-3.2$ yanglint -s -i -t auto -p 
> >> /Volumes/External/git/iana/yang-parameters/ 
> >> [email-masked] 
> >> <mailto:[email-masked]>  
> >> [email-masked] 
> >> <mailto:[email-masked]>  
> >> examples-notification-capabilities-1.xml
> >>
> >> bash-3.2$ ls 
> >> ..../iana/yang-parameters/ietf-subscribed-notifications@2019-09-09.
> >> yang 
> >> <mailto:iana/yang-parameters/ietf-subscribed-notifications@2019-09-
> >> 09.yang>
> >>
> >>  
> >>
> >> Cheers.
> >>
> >>  
> >>

---

From: Radek Krejci <[email-masked]>
Date: Tue, 17 Mar 2020 18:07:04 +0100

> ietf-subscribed-notification is wrong.

so, you are saying that despite formulations in 9.9.3 and e.g. 9.2.4 are
different, their meaning is the same? So the "require-instance" can be
used in types derived from instance-identifier and leafref? I'm afraid
that then it is not about different interpretations, but the text simply
misses the part about derived types and errata are required.

Radek

---

From: Benoit Claise <[email-masked]>
Date: Mon, 23 Mar 2020 11:25:58 +0100

Thanks Martin and Radek,

What would the new errata text stipulate?

Regards, Benoit

---

From: Radek Krejci <[email-masked]>
Date: Fri, 27 Mar 2020 11:19:41 +0100

Hi Benoit,
I've just submitted errata (6031) with the text mentioning derived types.

Regards,
Radek