Subject: [Netmod-ver-dt] Version selection

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Tue, 22 Oct 2019 18:46:47 +0000

Hi all,

Here’s take 1 of the client version selection draft update, I still need to add text, review text etc based on what we agree on. Here are some questions and open items, we need to close on some of those before I send next revision. I’ll track issues in github subsequently.

  1.  Use of schema-sets? In the previous version we had config for both NETCONF/RESTCONF using schema-set via port and root-path. We’re now using capability and new RPC/operation for NETCONF, and these work on YANG packages which are in YL (packages draft). I’m proposing that we remove schema-sets for now (for simplification), and have the version selection draft use “top level packages”.
  2.  If we keep schema-sets,  the YANG model has been updated to have a schema-set consist of a list of YANG packages. The question then is whether we want schema-sets to be configurable by the user or not.
  3.  NETCONF experts please take a look at the NETCONF update.
  4.  I’ve used @ as the separator for YANG package and version, e.g. [email-masked]<mailto:[email-masked]>, this is because @  cannot be in the revision-label and looks like it can’t be in the package-name either (pkg-identifier is of type yang:yang-identifier)
  5.  Joe had raised the issue of default package version if none is selected. Any suggestions on how to advertise the default version(s) in capabilities, what about @D after the revision-label e.g. “[email-masked]<mailto:[email-masked]>@D”? Rob, I don’t think your YL extensions has default for YANG package versions right now?

Regards,
Reshad,

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 23 Oct 2019 13:13:27 +0000

Hi Reshad, all,

Personally, I think that the number one requirement here is allow a device to be configured such that the entire device uses a particular schema family at a particular version.  Ideally, I would like the selection of this schema be written into configuration, and to be able to boot from this configuration (which may mean that I want the schema selection YANG module to be able to exist in all schema families).

Regarding schema-sets, the issue that this was trying to solve is that a device may use slightly different schema for different datastores (e.g., perhaps most plausibly, the schema for <operational> may have deviations that don’t apply to the conventional configuration datastores), but the server may require clients to always use consistent schema for the configuration vs operational datastores.

However, RFC 8342 has this text:

   The datastore schema for <operational> MUST be a superset of the
   combined datastore schema used in all configuration datastores,
   except that configuration data nodes supported in a configuration
   datastore MAY be omitted from <operational> if a server is not able
   to accurately report them.

I think that this effectively means that a device must have a single logical schema that is superset of all datastore schema.  Datastore schema are effectively allowed to choose subsets of that schema.

Given this, it is plausible that we don’t need schema-sets (or at least they can be renamed/repurposed), and instead we can rely on a device defining a package definition for the “superset schema”, which will implicitly select the appropriate package to be used for each datastore schema.  E.g. perhaps the package for the superset schema is [email-masked]<mailto:[email-masked]> and then the conventional configuration datastores use the package “[email-masked]” , and the operational state datastore uses “[email-masked]”.  Only the superset packages are advertised in the capabilities.  For devices that are fully compliant with NMDA, then it is plausible that they will use exactly the same schema (and hence package) for all configuration datastores and operational.

I think that this will cover the mainline case, where a client only wants to use one schema for all connections, and they just want to be able to choose the family and version of the schema that they are using to control the device.

The second use case, is where a vendor may want to allow different clients to interact with different schema families, or different schema versions at the same time (e.g. on separate connections).  I see that for this mechanism the RPC is probably the best way to go for NETCONF, and the path is the best way to go for RESTCONF.

Based on the text about, my suggestions are:

  1.  Define “datastore superset schema” as the schema that is a superset of all datastore schema (yes, we may need better names here).
  2.  Define “datastore superset package” as the associated YANG package that defines the “datastore superset schema”.
  3.  Perhaps rename schema-sets (or at least the per datastore parts of them) (see below)
  4.  Update the ietf-schema-version-selection YANG model:
     *   Perhaps rename “schema-sets” to “schema-superset”:
        *   Change to be read-only
        *   Do we need to advertise the restconf root path, or can then just be done via using “<package-name>@<version>” in the path?
        *   Probably change the list to be keyed by (package name, package version)
           *   The package here is both the superset package and also the default package/version used for each datastore unless a more explicit entry is listed in the datastore list.
     *   Keep the configurable “default-schema-set” leaf, but perhaps rename it.

Thoughts?  You may want to discuss some of this first before jumping into making changes?

Thanks,
Rob

From: Netmod-ver-dt <[email-masked]> On Behalf Of Reshad Rahman (rrahman)
Sent: 22 October 2019 19:47
To: [email-masked]
Subject: [Netmod-ver-dt] Version selection

Hi all,

Regards,
Reshad,

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Wed, 23 Oct 2019 20:41:19 +0000

On Oct 23, 2019, at 09:13, Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>> wrote:

Hi Reshad, all,

I agree with this.  It seems this will be what a number of vendors may opt for in lieu of per-session selection.

Joe

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Wed, 23 Oct 2019 20:50:51 +0000

1.  NETCONF experts please take a look at the NETCONF update.

Not saying I’m an expert, but the RPC looks funny to me.  Why can’t the package by CDATA?  How will you define an element for each and every package?  That is, should the RPC look more like:

<rpc message-id="101"
             xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
          <select-package>
            <yang-package-version>
              [email-masked]
            </yang-package-version>
          </select-package>
        </rpc>

Also, I don’t see the RPC defined in the included YANG module.  Are you waiting for feedback before doing that?

One other thing I noticed is that you say the following in Section 5.1.2:

Used by a client to select a YANG package version which
   has been advertised by the server via the mechanism described above
   in Section 5.1.1.  The YANG package version is selected for the
   lifetime of the NETCONF session, unless a different version of the
   YANG package is subsequently selected via this operation.

Then later:

Negative response: An <rpc-error> element is included in the <rpc-
   reply> if the request cannot be completed for any reason.  A <select-
   package> operation can fail for a number of reasons, such as the YANG
   package version is not supported by the server, or a different
   version of the YANG package has already been selected by another
   client.

How can a different client affect the current client’s session?  Meaning, if you and I are two clients both operation on the same server and you first set your package version, and then I set mine, why would mine fail simply because you set yours?

Now, it may be that only one version of a package can be used at a time across the entire server, but then that would break the per-session semantics.

  1.
  2.  I’ve used @ as the separator for YANG package and version, e.g. [email-masked]<mailto:[email-masked]>, this is because @  cannot be in the revision-label and looks like it can’t be in the package-name either (pkg-identifier is of type yang:yang-identifier)

I’m fine with this notation.  I assume that whatever is in the “version” field of the package header would be used here?

  1.
  2.  Joe had raised the issue of default package version if none is selected. Any suggestions on how to advertise the default version(s) in capabilities, what about @D after the revision-label e.g. “[email-masked]<mailto:[email-masked]>@D”? Rob, I don’t think your YL extensions has default for YANG package versions right now?

Make it bold in the capabilities list? :-D

Could another delimiter be used after the version like ‘;’ and then you can have:

?list=[email-masked],[email-masked];default

Right now, we’ll have to make sure ‘,’ doesn’t exist in a revision-label.

Joe

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Wed, 23 Oct 2019 22:24:33 +0000

Hi Rob,

Thanks for the detailed response, inline. Looks like we’ll be discussing these points during the meeting.

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wednesday, October 23, 2019 at 9:13 AM
To: "Reshad Rahman (rrahman)" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: RE: Version selection

Hi Reshad, all,

Personally, I think that the number one requirement here is allow a device to be configured such that the entire device uses a particular schema family at a particular version.
<RR> So basically what default-schema-set does (but probably with something else than schema-sets)? I agree.
<RR> One schema family at a particular version or multiple schema families each at their version? E.g. let’s say we have  2 schema families (vendor and IETF), so  e.g. top-level schemas are example-ietf-routing-top and example-vendor-xxx-top, are you suggesting we need to select one schema family or the other but not both? Some implementations may support 1 schema family at a time, others may concurrently support multiple schema families.
<RR> Also, when you say “configured”,  so we have config for default schema/version(s) and that can be overridden per “session” via the NETCONF/RESTCONF mechanisms? This is the equivalent of default-schema-set then?

Ideally, I would like the selection of this schema be written into configuration, and to be able to boot from this configuration (which may mean that I want the schema selection YANG module to be able to exist in all schema families).
<RR> If this config is for default behaviour, I agree.
<RR> I had thought about the schema selection YANG module needing to be in all schema families, that could be problematic. Instead of that, can we have a schema-family just for YANG version selection?

I think that this effectively means that a device must have a single logical schema that is superset of all datastore schema.  Datastore schema are effectively allowed to choose subsets of that schema.
<RR> Is the superset schema the combination of all schema families or 1 superset schema per schema family? If it’s the latter (which seems to be the case from your example below), how does the superset package differ from top-level package for a schema family? Or is it the same thing?

Given this, it is plausible that we don’t need schema-sets (or at least they can be renamed/repurposed), and instead we can rely on a device defining a package definition for the “superset schema”, which will implicitly select the appropriate package to be used for each datastore schema.  E.g. perhaps the package for the superset schema is [email-masked]<mailto:[email-masked]> and then the conventional configuration datastores use the package “[email-masked]” , and the operational state datastore uses “[email-masked]”.  Only the superset packages are advertised in the capabilities.  For devices that are fully compliant with NMDA, then it is plausible that they will use exactly the same schema (and hence package) for all configuration datastores and operational.
<RR> I agree that only the superset packages (which I understand to be same as top level packages) should be advertised in capabilities, that was the plan.

I think that this will cover the mainline case, where a client only wants to use one schema for all connections, and they just want to be able to choose the family and version of the schema that they are using to control the device.
<RR> So the superset package+version is the default schema/version pair then?

<RR> Let’s discuss in the meeting.

Regards,
Reshad.

Thanks,
Rob

Hi all,

Regards,
Reshad,

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Wed, 23 Oct 2019 22:25:16 +0000

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Wednesday, October 23, 2019 at 4:41 PM
To: "Rob Wilton (rwilton)" <[email-masked]>
Cc: "Reshad Rahman (rrahman)" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Netmod-ver-dt] Version selection

Hi Reshad, all,

<RR> The reason to have per session selection is that various clients may be able to handle different versions. I agree that the config for the default behaviour above is needed, that’s what the default-schema-set is meant to be. Anyway I’ve just replied to Rob’s email, so let’s continue discussing that point there.

Regards,
Reshad.

Joe

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Wed, 23 Oct 2019 22:32:14 +0000

Thanks Joe.

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Wednesday, October 23, 2019 at 4:50 PM
To: "Reshad Rahman (rrahman)" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [Netmod-ver-dt] Version selection

Not saying I’m an expert, but the RPC looks funny to me.  Why can’t the package by CDATA?  How will you define an element for each and every package?  That is, should the RPC look more like:
<RR> Good point, right now it’s only for 1 top-level package. And yes what you have below is better than what I sent.

<rpc message-id="101"

Also, I don’t see the RPC defined in the included YANG module.  Are you waiting for feedback before doing that?
<RR>Yes and I ran out of time ☺

Then later:

Now, it may be that only one version of a package can be used at a time across the entire server, but then that would break the per-session semantics.
<RR> Yes this is for the case where either a) package version can be selected once and after that is sticky OR b) different versions cannot be supported concurrently.

  1.

I’m fine with this notation.  I assume that whatever is in the “version” field of the package header would be used here?
<RR> Yes, this should be mentioned in the draft.

  1.

<RR> Works for me.

Regards,
Reshad.

Joe

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 24 Oct 2019 10:30:16 +0000

Hi Reshad,

Hum, it seems that my comments may have caused some confusion …

Please see inline …

From: Reshad Rahman (rrahman) <[email-masked]>
Sent: 23 October 2019 23:25
To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
Subject: Re: Version selection

Hi Rob,

From: "Rob Wilton (rwilton)" <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, October 23, 2019 at 9:13 AM
To: "Reshad Rahman (rrahman)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: Version selection

Hi Reshad, all,

Personally, I think that the number one requirement here is allow a device to be configured such that the entire device uses a particular schema family at a particular version.
[RW]
Note, I’m not saying that the current model doesn’t already cover this, only trying to emphasize that this is the requirement that I think is most important, most likely to be used, and hence want to ensure that the solution covers this in an easy way.

<RR> So basically what default-schema-set does (but probably with something else than schema-sets)? I agree.
[RW]
Yes, I agree that this is something like “default-schema-set”.

<RR> One schema family at a particular version or multiple schema families each at their version? E.g. let’s say we have  2 schema families (vendor and IETF), so  e.g. top-level schemas are example-ietf-routing-top and example-vendor-xxx-top, are you suggesting we need to select one schema family or the other but not both? Some implementations may support 1 schema family at a time, others may concurrently support multiple schema families.
[RW]
I think that the ideal goal is that clients will use one version of one schema family to configure their device.

Pragmatically, it is plausible that some features may only exist in some schema, so in the short/medium term I think that there is a need to allow multiple schema families to be used in the same session (and probably also during boot).

However, I don’t think that we need to allow clients to support multiple versions of a schema family in a particular session.

Also, even if a device supports multiple schema families for configuration, it is plausible that a client might want to restrict

[RW]
Yes.

Ideally, I would like the selection of this schema be written into configuration, and to be able to boot from this configuration (which may mean that I want the schema selection YANG module to be able to exist in all schema families).
<RR> If this config is for default behaviour, I agree.
<RR> I had thought about the schema selection YANG module needing to be in all schema families, that could be problematic. Instead of that, can we have a schema-family just for YANG version selection?
[RW]
Yes, having a separate schema family might be a good solution here.  Effectively it is a bit like meta configuration anyway.

I think that this effectively means that a device must have a single logical schema that is superset of all datastore schema.  Datastore schema are effectively allowed to choose subsets of that schema.
<RR> Is the superset schema the combination of all schema families or 1 superset schema per schema family? If it’s the latter (which seems to be the case from your example below), how does the superset package differ from top-level package for a schema family? Or is it the same thing?
[RW]
I think that it is latter, i.e. one superset schema per schema family.

I think that a superset schema probably ends up being the same as a top-level package.

Packages just define a schema.  That schema might only apply to a particular datastore, or that schema might apply to the device, effectively constraining what the per-datastore schema can look like.

Normally, I think that a client wants to choose a package that applies to all datastores, and hence they want to choose a superset schema package.  Of course, it could also be the case that all datastores use the same schema.

Maybe we also want to allow a client to specify a schema only for one datastore, in which case I think that they would be wanting to choose a package that relates to a datastore schema rather than a device schema.  (E.g. a client might want to read a subset of the operational data using an OpenConfig schema.  Whether this makes and sense, and whether this is feasible is unclear to me).  Potentially, we should defer this consideration at this time, and just focus on choosing superset schema.

Given this, it is plausible that we don’t need schema-sets (or at least they can be renamed/repurposed), and instead we can rely on a device defining a package definition for the “superset schema”, which will implicitly select the appropriate package to be used for each datastore schema.  E.g. perhaps the package for the superset schema is [email-masked]<mailto:[email-masked]> and then the conventional configuration datastores use the package “[email-masked]<mailto:[email-masked]>” , and the operational state datastore uses “[email-masked]<mailto:[email-masked]>”.  Only the superset packages are advertised in the capabilities.  For devices that are fully compliant with NMDA, then it is plausible that they will use exactly the same schema (and hence package) for all configuration datastores and operational.
<RR> I agree that only the superset packages (which I understand to be same as top level packages) should be advertised in capabilities, that was the plan.
[RW]
OK.

I think that this will cover the mainline case, where a client only wants to use one schema for all connections, and they just want to be able to choose the family and version of the schema that they are using to control the device.
<RR> So the superset package+version is the default schema/version pair then?
[RW]
Actually, I’m going to row back on this point.  I suspect that it might be one version per supported schema family.  So clients may want to choose which schema families that want to use, and also which versions of those schema families they wish to use.

(i)     Define “datastore superset schema” as the schema that is a superset of all datastore schema (yes, we may need better names here).

(ii)   Define “datastore superset package” as the associated YANG package that defines the “datastore superset schema”.

(iii)  Perhaps rename schema-sets (or at least the per datastore parts of them) (see below)

(iv)  Update the ietf-schema-version-selection YANG model:

o   Perhaps rename “schema-sets” to “schema-superset”:

§  Change to be read-only

§  Do we need to advertise the restconf root path, or can then just be done via using “<package-name>@<version>” in the path?

§  Probably change the list to be keyed by (package name, package version)

·       The package here is both the superset package and also the default package/version used for each datastore unless a more explicit entry is listed in the datastore list.

o   Keep the configurable “default-schema-set” leaf, but perhaps rename it.

[RW]
Ack.

Thanks,
Rob

Regards,
Reshad.

Thanks,
Rob

From: Netmod-ver-dt <[email-masked]<mailto:[email-masked]>> On Behalf Of Reshad Rahman (rrahman)
Sent: 22 October 2019 19:47
To: [email-masked]<mailto:[email-masked]>
Subject: [Netmod-ver-dt] Version selection

Hi all,

Regards,
Reshad,