Subject: [Netmod-ver-dt] Updated YANG module for version selection

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Mon, 2 Dec 2019 15:42:01 +0000

Hi,

Based on the discussion on Thursday, I said that I would try and write some YANG to cover what we discussed.  I've now had a go at this.  All of the YANG modules are pushed the DT repository, at the path yang-ver-selection/tmp/.

1) At the same time, I have also updated the packages YANG modules a bit (but not updated the draft yet).

  *   I've somewhat aligned the structured between the instance data (e.g. no longer using module-sets in the YL definition).
  *   I've tried to use <pkg-name>@<pkg-version> to identify the package (e.g. a list key in the YL definition, but I'm not sure that this makes sense in the file (which doesn't have a list of modules).  It would be good to discuss/get thoughts about this.  Should we allow a revision-date as a valid pkg-version identifier?

2) For the YANG version selection, my thought process is along these lines:

Packages list (in YL):
- Flat list of packages, where each package represents a schema (which could be local, could be incomplete)
- Each package has a 1:1 mapping of what is on the device vs instance-data file.
- Each package has a 1:1 relationship to datastores.
- Packages can be defined by third party entities (e.g. IETF), and are immutable definitions (for a particular revision)
- Should 'YANG packages' just be called 'YANG schema'?  Really a package is a definition of a (partial) schema.
selectable-packages:
- List of packages that can be selected into schema.
- Also documents relationship to datastore schema
custom-package:
- Allows clients to configure a selectable package.
 - Only configured, appears in both the selectable-packages and packages list.
- Need to decide how much of a package definition can be configured (not modules/sub-modules, not conflicting packages, but probably more than what I allow currently).

default-schema:
- which 'selectable package' represents the default schema.

secondary-schema:
- which 'selectable packages' represents the secondary schemas.

Here is the current YL schema:

~/netmod-wg/yang-ver-dt/yang-ver-selection/tmp$ pyang -f tree ietf-schema-selection.yang
ietf-schema-selection.yang:7: warning: imported module "ietf-inet-types" not used
module: ietf-schema-selection
  +--rw schema-selection
     +--rw default-schema?       -> /schema-selection/selectable-package/identifier {default-schema}?
     +--rw secondary-schema*     -> /schema-selection/selectable-package/identifier {secondary-schema}?
     +--rw custom-package* [identifier] {custom-package}?
     |  +--rw identifier          pkg:pkg-identifier
     |  +--rw description?        string
     |  +--rw included-package*   -> /schema-selection/selectable-package/identifier
     +--ro selectable-package* [identifier]
        +--ro identifier            -> /yanglib:yang-library/yl-pkg:package/identifier
        +--ro datastores* [datastore]
        |  +--ro datastore    ds:datastore-ref
        |  +--ro package?     -> /yanglib:yang-library/yl-pkg:package/identifier
        +--ro required-package*     -> /schema-selection/selectable-package/identifier
        +--ro combinable-package*   -> /schema-selection/selectable-package/identifier
        +--ro compatible-package*   -> /schema-selection/selectable-package/identifier

I've omitted the RPC and RESTCONF stuff for the moment.  I was hoping that the default-schema and secondary-schema could be advertised using the package identifier in NETCONF capabilities and in the RESTCONF path (but would also need to think about encoding).

3) I think that the use of features gives a lot of (perhaps too much?) flexibility on how schema selection could be supported by devices.  E.g. modes of operation for a server:
(1) Doesn't support schema version selection at all (YANG schema selection module isn't implemented).
(2) Server has a defined list of available schema.  Clients can choose a single schema from the list as the default schema. (Enabled features: default-schema)
(3) Server has a defined list of available schema.  Clients cannot change the default schema, but can select additional secondary schema from the list. (Enabled features: secondary-schema)
(4) Server has a defined list of available schema.  Clients can choose default and secondary schema from the list. (Enabled features: default-schema, secondary-schema)
(5) For options (2), (3) and (4), the server can also optionally support the client configuring the schema package definitions by combining compatible selectable packages into a new client defined package definition. (Enabled features: custom-package and at least one of default-schema and secondary-schema).

4) For reference.  Updated tree output for packages (instance data file that hasn't changed much, and then YL augmentations that has changed more):

module: ietf-yang-package
  +--rw package
     +--rw name                  pkg-name
     +--rw version               pkg-version
     +--rw timestamp?            yang:date-and-time
     +--rw organization?         string
     +--rw contact?              string
     +--rw description?          string
     +--rw reference?            string
     +--rw location*             inet:uri
     +--rw complete?             boolean
     +--rw local?                boolean
     +--rw previous-version?     pkg-version
     +--rw nbc-changes?          boolean
     +--rw tag*                  tags:tag
     +--rw mandatory-feature*    scoped-feature
     +--rw included-package* [identifer]
     |  +--rw identifier          pkg-identifier
     |  +--rw replaces-version*   pkg-version
     |  +--rw nbc-modified?       boolean
     |  +--rw location*           inet:uri
     |  +--rw checksum?           pkg-types:sha-256-hash
     +--rw module* [name]
     |  +--rw name                 yang:yang-identifier
     |  +--rw revision?            rev:revision-date-or-label
     |  +--rw replaces-revision*   rev:revision-date-or-label
     |  +--rw namespace?           inet:uri
     |  +--rw location*            inet:uri
     |  +--rw checksum?            pkg-types:sha-256-hash
     |  +--rw submodule* [name]
     |     +--rw name        yang:yang-identifier
     |     +--rw revision    yang:revision-identifier
     |     +--rw location*   inet:uri
     |     +--rw checksum?   pkg-types:sha-256-hash
     +--rw import-only-module* [name revision]
        +--rw name                 yang:yang-identifier
        +--rw revision             rev:revision-date-or-label
        +--rw replaces-revision*   rev:revision-date-or-label
        +--rw namespace?           inet:uri
        +--rw location*            inet:uri
        +--rw checksum?            pkg-types:sha-256-hash
        +--rw submodule* [name]
           +--rw name        yang:yang-identifier
           +--rw revision    yang:revision-identifier
           +--rw location*   inet:uri
           +--rw checksum?   pkg-types:sha-256-hash

module: ietf-yl-packages
  augment /yanglib:yang-library:
    +--ro package* [identifier]
       +--ro identifier            pkg-identifier
       +--ro timestamp?            yang:date-and-time
       +--ro organization?         string
       +--ro contact?              string
       +--ro description?          string
       +--ro reference?            string
       +--ro location*             inet:uri
       +--ro complete?             boolean
       +--ro local?                boolean
       +--ro previous-version?     pkg-version
       +--ro nbc-changes?          boolean
       +--ro tag*                  tags:tag
       +--ro mandatory-feature*    scoped-feature
       +--ro included-package* [identifer]
       |  +--ro identifier          pkg-identifier
       |  +--ro replaces-version*   pkg-version
       |  +--ro nbc-modified?       boolean
       |  +--ro location*           inet:uri
       |  +--ro checksum?           pkg-types:sha-256-hash
       +--ro module* [name]
       |  +--ro name                 yang:yang-identifier
       |  +--ro revision?            rev:revision-date-or-label
       |  +--ro replaces-revision*   rev:revision-date-or-label
       |  +--ro namespace?           inet:uri
       |  +--ro location*            inet:uri
       |  +--ro checksum?            pkg-types:sha-256-hash
       |  +--ro submodule* [name]
       |     +--ro name        yang:yang-identifier
       |     +--ro revision    yang:revision-identifier
       |     +--ro location*   inet:uri
       |     +--ro checksum?   pkg-types:sha-256-hash
       +--ro import-only-module* [name revision]
          +--ro name                 yang:yang-identifier
          +--ro revision             rev:revision-date-or-label
          +--ro replaces-revision*   rev:revision-date-or-label
          +--ro namespace?           inet:uri
          +--ro location*            inet:uri
          +--ro checksum?            pkg-types:sha-256-hash
          +--ro submodule* [name]
             +--ro name        yang:yang-identifier
             +--ro revision    yang:revision-identifier
             +--ro location*   inet:uri
             +--ro checksum?   pkg-types:sha-256-hash
  augment /yanglib:yang-library/yanglib:schema:
    +--ro package
       +--ro identifier?                   -> /yanglib:yang-library/package/identifier
       +--ro checksum?                     pkg-types:sha-256-hash
       +--ro supported-optional-feature*   pkg-types:scoped-feature

Comments/thoughts welcome by email, or can wait until Thursday.

Thanks,
Rob

Subject: [Netmod-ver-dt] [netmod-wg/yang-ver-dt] 5c6b7a: Some updated YANG modules based on Thursdays meeting
From: Robert Wilton <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Mon, 2 Dec 2019 15:15:38 +0000

  Branch: refs/heads/develop
  Home:   https://github.com/netmod-wg/yang-ver-dt
  Commit: 5c6b7a954a03603cb6a664ac92640f75e60548b4
      https://github.com/netmod-wg/yang-ver-dt/commit/5c6b7a954a03603cb6a664ac92640f75e60548b4
  Author: rwilton <[email-masked]>
  Date:   2019-12-02 (Mon, 02 Dec 2019)

  Changed paths:
    A yang-ver-selection/tmp/[email-masked]
    A yang-ver-selection/tmp/ietf-schema-selection.yang
    A yang-ver-selection/tmp/ietf-yang-package-types.yang
    A yang-ver-selection/tmp/ietf-yang-package.yang
    A yang-ver-selection/tmp/ietf-yang-revisions.yang
    A yang-ver-selection/tmp/[email-masked]
    A yang-ver-selection/tmp/ietf-yl-packages.yang

  Log Message:
  -----------
  Some updated YANG modules based on Thursdays meeting

_______________________________________________
Netmod-ver-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod-ver-dt

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Thu, 5 Dec 2019 16:19:08 +0000

Thx Rob.
See below for some initial thoughts. I need to digest this some more and I'll try to give some further feedback.
Jason

From: Netmod-ver-dt <[email-masked]> On Behalf Of Rob Wilton (rwilton)
Sent: Monday, December 2, 2019 10:42 AM
To: [email-masked]
Subject: [Netmod-ver-dt] Updated YANG module for version selection

Hi,

  *   I've somewhat aligned the structured between the instance data (e.g. no longer using module-sets in the YL definition).
  *   I've tried to use <pkg-name>@<pkg-version> to identify the package (e.g. a list key in the YL definition, but I'm not sure that this makes sense in the file (which doesn't have a list of modules)..  It would be good to discuss/get thoughts about this.  Should we allow a revision-date as a valid pkg-version identifier?

Packages list (in YL):
- Packages can be defined by third party entities (e..g. IETF), and are immutable definitions (for a particular revision)

selectable-packages:

custom-package:

default-schema:
- which 'selectable package' represents the default schema.
[>>JTS: ] I assume that reading this leaf from the 'running' DS would *not* return a value unless it had actually been configured right?  But reading it from the operational DS might return some default default-schema in use by a server if nothing is configured?

secondary-schema:

Here is the current YL schema:[>>JTS: ] Did you mean "current selection schema" or perhaps "current YP schema" ?

[>>JTS: ] You asked in another email "For custom packages, what fields should we allow to be configured (e.g. just a minimal subset, or everything that could be valid)?  Should clients be allowed to select features?"
I don't love the idea of a *client* selecting features. Features are for a server to advertise what it supports. Clients can then use the associated leafs, etc or not if they wish.

[>>JTS: ] You asked in another email "Is using <pkg-name@version> okay as a schema identifier in RESTCONF?  Would the name need to be encoded beforehand?  Or do we want to force the client to configure a simpler name?"
Do we need to *force* the client to configure a simpler name, or just *allow* a client to configure a simpler name? Your "custom-package" concept allows this (it then becomes part of the selectable-package list right?).

3) I think that the use of features gives a lot of (perhaps too much?) flexibility on how schema selection could be supported by devices.  E.g. modes of operation for a server:
(1) Doesn't support schema version selection at all (YANG schema selection module isn't implemented).
(2) Server has a defined list of available schema.  Clients can choose a single schema from the list as the default schema.. (Enabled features: default-schema)
(3) Server has a defined list of available schema.  Clients cannot change the default schema, but can select additional secondary schema from the list. (Enabled features: secondary-schema)
(4) Server has a defined list of available schema.  Clients can choose default and secondary schema from the list. (Enabled features: default-schema, secondary-schema)
(5) For options (2), (3) and (4), the server can also optionally support the client configuring the schema package definitions by combining compatible selectable packages into a new client defined package definition. (Enabled features: custom-package and at least one of default-schema and secondary-schema).
[>>JTS: ] There is a lot of flexibility there, but it isn't obvious that we can remove much of it. There will be significant variation in how much of this will be supported by different servers.
If a server only supports one schema at a time in the entire network element (from a list of N possible schemas in the 'selectable-package' list), then I suppose a client has two options:
1) select the schema it wants in the schema selection RPC,
*or*
2) configure the default-schema  (but maybe that is a chicken-and-egg problem because to configure *anything* you need to already have selected a schema that includes the ietf-schema-selection module)

4) For reference.  Updated tree output for packages (instance data file that hasn't changed much, and then YL augmentations that has changed more):
[>>JTS: ] I'm probably just forgetting something fundamental, but why is the tree below all "rw" and not "ro" ?  (other than maybe the 'tag' leaf)

module: ietf-yl-packages

[>>JTS: ] You asked in another email "Is a revision date a valid version identifier for a package?"

Long term it is probably better to just have one option -> version (not date-or-version).

Thanks,
Rob

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 6 Dec 2019 14:30:39 +0000

Apologies for missing yesterdays meeting at short notice.

Thanks for the comments, please see [RW] inline ...

From: Netmod-ver-dt <[email-masked]> On Behalf Of Sterne, Jason (Nokia - CA/Ottawa)
Sent: 05 December 2019 16:19
To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
Subject: Re: [Netmod-ver-dt] Updated YANG module for version selection

Thx Rob.

From: Netmod-ver-dt <[email-masked]<mailto:[email-masked]>> On Behalf Of Rob Wilton (rwilton)
Sent: Monday, December 2, 2019 10:42 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [Netmod-ver-dt] Updated YANG module for version selection

Hi,

-        I've somewhat aligned the structured between the instance data (e.g. no longer using module-sets in the YL definition).

-        I've tried to use <pkg-name>@<pkg-version> to identify the package (e.g. a list key in the YL definition, but I'm not sure that this makes sense in the file (which doesn't have a list of modules)..  It would be good to discuss/get thoughts about this.  Should we allow a revision-date as a valid pkg-version identifier?

Packages list (in YL):

selectable-packages:

custom-package:

default-schema:

[RW]
Yes, exactly right.

<running> only shows what is configured, but <operational> would show the default-schema in use (if the ietf-schema-selection module was implemented).  An implementation could choose to only support the "default-schema" in <operational> and not <running>, if they wished.

secondary-schema:

Here is the current YL schema:[>>JTS: ] Did you mean "current selection schema" or perhaps "current YP schema" ?
[RW]
Sorry, I meant the tree output of the schema selection module that I had knocked up backed on last week's discussion.

[>>JTS: ] You asked in another email "For custom packages, what fields should we allow to be configured (e.g. just a minimal subset, or everything that could be valid)?  Should clients be allowed to select features?"
I don't love the idea of a *client* selecting features. Features are for a server to advertise what it supports. Clients can then use the associated leafs, etc or not if they wish.
[RW]
I think that I agree with you regarding features ...  Or at the very least, they should not be considered at this time because it increases complexity, and we are trying to simplify.

[>>JTS: ] You asked in another email "Is using <pkg-name@version> okay as a schema identifier in RESTCONF?  Would the name need to be encoded beforehand?  Or do we want to force the client to configure a simpler name?"
Do we need to *force* the client to configure a simpler name, or just *allow* a client to configure a simpler name? Your "custom-package" concept allows this (it then becomes part of the selectable-package list right?).
[RW]
For the simple case, I am trying to avoid the requirement of clients *having* to configure custom packages.  I think that it is quite plausible that some implementations won't allow clients to configure custom packages at all, and will just force them to select between some predefined schema (e.g. perhaps OpenConfig instead of the vendor native schema).

In the case that the device does allow custom packages to be configured, then I agree that a valid use of them could be to give the schema a short/nice name.  But do we possibly want to enforce that the custom package identifier must have a version assigned to it for consistency?  In which case we would still be back to always having '@' in the package identifier anyway.

Perhaps it is OK to just say that the package identifier is always URI encoded, which would mean that a package of [email-masked]<mailto:[email-masked]> would end up being "basic-router-pkg%401.2.3" in the URI.  Would this be okay, or too ugly?

(1) Doesn't support schema version selection at all (YANG schema selection module isn't implemented)..
[RW]
Yes, I think that one idea, that probably needs to be explored further, is that the ietf-schema-selection module (and any dependencies) might need to exist across the different schema families.

4) For reference.  Updated tree output for packages (instance data file that hasn't changed much, and then YL augmentations that has changed more):
[>>JTS: ] I'm probably just forgetting something fundamental, but why is the tree below all "rw" and not "ro" ?  (other than maybe the 'tag' leaf)
[RW]
Sorry, it is effectively read-only.  The base version of pyang doesn't fully support the "structure" element, so I hacked it into a container to generate the tree output.

module: ietf-yl-packages

[RW]
Currently I think that pkg-version uses the type revision-label, which is defined as any string except for a revision-date.  So, I think that the question is really whether there is a good reason to explicitly disallow revision dates here.

Thanks,
Rob

Thanks,
Rob