Subject: [Netmod-ver-dt] YANG packages draft - now ready for review please

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Mon, 7 Oct 2019 13:58:05 +0000

Hi,

I think that this document has had sufficient updates that it is ready for review.

The latest version is available at per the commit information above, or attached.

I appreciate that it is quite short notice, but if it is possible for folks to review and provide comments before Thursday's meeting that probably be helpful.

Thanks,
Rob

Subject: [Netmod-ver-dt] [rgwilton/YANG-Packages-Draft] f37798: Further markups for DT review
From: Robert Wilton <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Mon, 7 Oct 2019 13:54:03 +0000

  Branch: refs/heads/master
  Home:   https://github.com/rgwilton/YANG-Packages-Draft
  Commit: f37798f3ff28eb88a20eacdb978d38e4f25906e4
      https://github.com/rgwilton/YANG-Packages-Draft/commit/f37798f3ff28eb88a20eacdb978d38e4f25906e4
  Author: rwilton <[email-masked]>
  Date:   2019-10-07 (Mon, 07 Oct 2019)

  Changed paths:
    M draft-rwilton-netmod-yang-packages.txt
    M draft-rwilton-netmod-yang-packages.xml
    A ietf-yang-inst-data-pkg.yang
    M ietf-yang-package-types.yang
    M ietf-yang-package.yang
    M ietf-yl-packages.yang
    A tmp/[email-masked]

  Log Message:
  -----------
  Further markups for DT review

_______________________________________________
Netmod-ver-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod-ver-dt

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Tue, 8 Oct 2019 17:16:51 +0000

Hi Rob,

Last round I think you just posted this to the main NETMOD list and I added my comments to the list. Just checking that you did indeed want to review this internally to the DT first.

Jason

From: Netmod-ver-dt <[email-masked]> On Behalf Of Rob Wilton (rwilton)
Sent: Monday, October 7, 2019 9:58 AM
To: [email-masked]
Subject: [Netmod-ver-dt] YANG packages draft - now ready for review please

Hi,

Thanks,
Rob

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 9 Oct 2019 08:09:10 +0000

Hi Jason,

Yes, review comments from the DT list are welcome.

The idea is that the packages draft is logically part of the 5 drafts that address the versioning work, and as such come under the umbrella of the design team.  I've not changed the name (or github location for the draft) for convenience.

Thanks,
Rob

From: Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>
Sent: 08 October 2019 18:17
To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
Subject: RE: YANG packages draft - now ready for review please

Hi Rob,

Jason

From: Netmod-ver-dt <[email-masked]<mailto:[email-masked]>> On Behalf Of Rob Wilton (rwilton)
Sent: Monday, October 7, 2019 9:58 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [Netmod-ver-dt] YANG packages draft - now ready for review please

Hi,

Thanks,
Rob

---

From: Balázs Lengyel <[email-masked]>
Date: Thu, 10 Oct 2019 16:18:40 +0000

Hello,

Some comments:

General)

Run the spellchecker

I want the instance-data and run-time data to be the same. IMHO just expand
yang-library and use that as package instance data. The fact that run-time
and implementation-tie data is similar is PORTANT!  Some data duplication of
readOnly data is much better than having different YAMs online and offline.

Actually all the additional data added to yanglib can also be available in a
yang-instance-data-file so now we have two instance data file formats. VERY
BAD :-(

Sorry I didn't have time to check the YANG modules in detail.

CH.4)

What does "set modifications" mean. Is just the word "of" missing or is this
a concept?

Ch.5)Can there be any mandatory deviations?  My guess would be yes. If yes,
are they identified by a deviation file or something more fine grained? 

Ch 5.1 para 3) I am not sure what is a package instance? What identifies a
package and what identifies a package instance? Do we need it in
terminology? 

5.2.1.1) 

Is there a difference between a package importing or including something.
As importing means something specific in YANG importing import-only modules
and importing non-import-only modules is confusing for me.  I am confused
between this mix of import/include, you use both.

If I import a package does it mean I implement its YAMs?

As deviations are basically modules why do we need to mention para 5
deviations separately. Can you have deviations on a package or only the
usual YAM deviations?

5.2.1.3)It is not defined which data is considered "meta" data.

5.3.1) What is the relation between the top level package-checksum and 
/yanglib:yang-library/yanglib:content-id? Do they both cover the same
information?  Describe it in the text.

5.5.2)  How would a local package be used?

6) Why do you recommend JSON? Why is it important to recommend one format
over the other? Maybe for IETF packages we should recommend one format, but
otherwise IMHO let them use what they like.

The 2 name variant look the same to me. Did you mean .xml for the second
one?

The rules about package names should be in a separate chapter. They are
valid outside instance data a too.

IMHO it should be allowed to add more text to the description field. The
desc field should START with ...

Deviations are missing for modules and import-only-modules. They should be
here.

7.1) What if only part of a module set is included in a package? As I see it
the current yl-packages YAM can't handle that.

7.2)  Why don't we allow BC deviations. E.g. Ericsson and Tailf often adds
some yang extension statements. That is not allowed here.

You have the text: "deviations   to the original package". This seems to
indicate that the deviations are on package level. IMHO they are on the
modules inside the package, realized by adding additional modules. Or is it
possible to change some package properties e.g. location ?

10) module ietf-yang-package should define simple YANG data, not a
structure. 

Structure is only needed if you want to extend the definition of the
instance data itself like in module ietf-yang-inst-data-pkg.

Is there really a need for a timestamp instead of a simple date for
packages?

Why is checksum not part of yang-pkg-common-leafs? It is always there when
the grouping is used.

In Yanglib the package checksum states: 

      leaf checksum {

        type pkg-types:sha-256-hash;

        description

          "The SHA-256 hash calculated on the textual package

           definition, represented as a hexadecimal string.";

      }

What is the textual representation of the package. Is it the representation
according to the instance-data file? If yes, this must be stated.

11) We should develop a standard security text about instance-data files.
In notification capabilities I use:

      <t>When that data is in file format, data should be protected against 

        modification or unauthorized access using normal file handling
mechanisms.

      </t>

Actually all the additional data added to yanglib can also be available in a
yang-instance-data-file.

Is the namespace really needed for these modules/import-only-modules? I
don't object, I just don't see the need.

12) Only 3 of the 4 YAMs are present.

Regards Balazs

From: Netmod-ver-dt <[email-masked]> On Behalf Of Rob Wilton
(rwilton)
Sent: 2019. október 7., hétfő 15:58
To: [email-masked]
Subject: [Netmod-ver-dt] YANG packages draft - now ready for review please

Hi,

I think that this document has had sufficient updates that it is ready for
review.

The latest version is available at per the commit information above, or
attached.

I appreciate that it is quite short notice, but if it is possible for folks
to review and provide comments before Thursday's meeting that probably be
helpful.

Thanks,

Rob

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 15 Oct 2019 15:31:20 +0000

Hi Balazs,

Thanks for the review.  Please see comments inline.  Hopefully we can manage to discuss some of these on Thursday’s meeting.

From: Balázs Lengyel <[email-masked]>
Sent: 10 October 2019 17:19
To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
Subject: RE: YANG packages draft - now ready for review please

Hello,
Some comments:
General)
Run the spellchecker
[RW] OK 😉

I want the instance-data and run-time data to be the same. IMHO just expand yang-library and use that as package instance data. The fact that run-time and implementation-tie data is similar is PORTANT!  Some data duplication of readOnly data is much better than having different YAMs online and offline.
Actually all the additional data added to yanglib can also be available in a yang-instance-data-file so now we have two instance data file formats. VERY BAD :-(
[RW]
I understand the desire for these to use the same format, but I don’t think that extending YANG library can really work because it would effectively fundamentally change the definition of YANG library from being a flat list of modules to a hierarchical structure in YANG packages.  I.e. I think that it would require a new non-backwards compatible version 3 of YANG library.

The alternative solution here would be for YANG packages not to try and reuse the module information in YANG library when constructing packages on the router.  Instead, the same definition could be used both in instance data files and also on the device.

I think that this issue is important enough that we should debate it in the next DT meeting, but probably this one requires input and discussion from the wider working group.

Sorry I didn’t have time to check the YANG modules in detail.

CH.4)
What does “set modifications” mean. Is just the word “of” missing or is this a concept?
[RW]
“Set of modifications”.  Fixed.

Ch.5)Can there be any mandatory deviations?  My guess would be yes. If yes, are they identified by a deviation file or something more fine grained?
[RW]
Deviations are defined by adding addition modules to the package that define the deviations.  They are effectively mandatory like everything else in the package (except for unlisted features).

Ch 5.1 para 3) I am not sure what is a package instance? What identifies a package and what identifies a package instance? Do we need it in terminology?
[RW]
Yes, it sounds like we need additional terminology.

By “package instance” I mean a particular version of a package.  Whereas a package more generally might be used to refer to all possible versions of a package with the same name.

So, a package is defined by having a name that acts as a unique identifier.
A package instance is defined by a instance data file containing a package definition, and is uniquely identified by the (name, version) tuple, and can be verified by its checksum.

5.2.1.1)
Is there a difference between a package importing or including something.
[RW]
No, I should choose one term and stick to it.  Probably include is better to avoid confusion.

  As importing means something specific in YANG importing import-only modules and importing non-import-only modules is confusing for me.  I am confused between this mix of import/include, you use both.
If I import a package does it mean I implement its YAMs?
[RW]
The intention is that if you “import/include a package” then all modules in the “module” list are implemented, and all modules in the “import-only-module” list are import only.

As deviations are basically modules why do we need to mention para 5 deviations separately. Can you have deviations on a package or only the usual YAM deviations?
[RW]
The deviations are only the usage YAM deviations.  I.e. they are acting on modules that are part of the module list, on any included package module list.

5.2.1.3)It is not defined which data is considered “meta” data.
[RW]
OK, this should be clarified.

5.3.1) What is the relation between the top level package-checksum and
/yanglib:yang-library/yanglib:content-id? Do they both cover the same information?  Describe it in the text.
[RW]
They are unrelated.
YANG library content-id is local to the device, and generated in a device specific manner.  I.e. clients cannot use the content-id to avoid downloading and checking YANG library from a device.

The checksum is guaranteed to be the same for the same package, and hence, if the client already knows the content of <pkg-name>@<version>, and the checksum matches then it doesn’t need to re-download the content from the device since it knows that it already matches.

5.5.2)  How would a local package be used?
[RW]
If a device has a patch installed that modified the schema slightly, then it would need a different package definition that is defined as <base-pkg> + <hot-fix-pkg>*.  This package definition may depend on what hot fixes have been installed on the device and hence can only really be defined locally to the device (but the package definitions for the base-pkg and each of the hot fixes can be globally unique).

6) Why do you recommend JSON? Why is it important to recommend one format over the other? Maybe for IETF packages we should recommend one format, but otherwise IMHO let them use what they like.
[RW]
I was recommending JSON because I think that it is more readable for humans.  It perhaps also helps for the package checksums if they are mostly defined in the same encoding.  But I don’t feel too strongly on this.

The 2 name variant look the same to me. Did you mean .xml for the second one?
[RW]
Yes.

The rules about package names should be in a separate chapter. They are valid outside instance data a too.
IMHO it should be allowed to add more text to the description field. The desc field should START with ...
[RW]
OK, we should probably discuss this one.  Note that there is already a description field with the package definition.

Deviations are missing for modules and import-only-modules. They should be here.
[RW]
They are not required, as in the information is redundant.  I would have preferred if they hadn’t been included in YANG library.  E.g. if a module deviates another module in YANG library, but the deviation isn’t listed in YANG library then what does that mean?

Also, packages are trying to allow conformance to be defined/checked at the package level rather than the individual module level.

7.1) What if only part of a module set is included in a package? As I see it the current yl-packages YAM can’t handle that.
[RW]
I don’t think that should be allowed.  In that case, a separate module-set can be defined, or the original module-set could be split into two separate module sets.

7.2)  Why don’t we allow BC deviations. E.g. Ericsson and Tailf often adds some yang extension statements. That is not allowed here.
[RW]
That is a mistake.  That should be allowed.

You have the text: “deviations   to the original package”. This seems to indicate that the deviations are on package level. IMHO they are on the modules inside the package, realized by adding additional modules. Or is it possible to change some package properties e.g. location ?
[RW]
I need to tweak the text.  Deviations are exactly as before, i.e. they act on paths defined by the modules.

10) module ietf-yang-package should define simple YANG data, not a structure.
Structure is only needed if you want to extend the definition of the instance data itself like in module ietf-yang-inst-data-pkg.
[RW]
We should discuss this one.  At the moment, it uses YANG structure because it is currently only used to define a file format.  The intention is that the file package definition wouldn’t be used on a device, which is why I made it a structure rather than regular YANG.

Is there really a need for a timestamp instead of a simple date for packages?
[RW]
This was suggested in one of the meetings.  Note that the timestamp (or date if we used that instead) doesn’t uniquely identify a package version.  Perhaps a date is sufficient.  My concern is that if you have automated tooling then it is quite plausible that you could end up with more than one package with the same date, whether that is a problem is a different question.

Why is checksum not part of yang-pkg-common-leafs? It is always there when the grouping is used.
[RW]
A package file instance doesn’t define a checksum because it cannot be specified (since the checksum value would need to be part of the checksum calculation).

In Yanglib the package checksum states:
      leaf checksum {
        type pkg-types:sha-256-hash;
        description
          "The SHA-256 hash calculated on the textual package
           definition, represented as a hexadecimal string.";
      }
What is the textual representation of the package. Is it the representation according to the instance-data file? If yes, this must be stated.
[RW]
Yes, it is the instance data file.  I think that this is stated somewhere, but I will clarify it.

11) We should develop a standard security text about instance-data files.  In notification capabilities I use:
      <t>When that data is in file format, data should be protected against
        modification or unauthorized access using normal file handling mechanisms.
      </t>
[RW]
I agree.

Actually all the additional data added to yanglib can also be available in a yang-instance-data-file.
[RW]
I presume that is true of any YANG module?  The intention is that the augmentations to YANG library are not intended to be used in YANG instance data files.

Is the namespace really needed for these modules/import-only-modules? I don’t object, I just don’t see the need.
[RW]
I don’t think that is needed, but it is there in case someone wants to include the namespace information (e.g. it is also available in YANG library).

In fact, I’m not even convinced that a URI is generally required, at least not for standardized YANG modules.  The name, revision, checksum should be sufficient, on the assumption that tooling should be able to figure out where to fetch standard YANG models from.  So I was thinking of removing the URIs from the examples, at least for published standard YANG models.

12) Only 3 of the 4 YAMs are present.
[RW]
I think that they are all there (or at least they are in my XML) 😉

Thanks again for the detailed review.  Obvious some issues that we need to work through.  I’ve make the appropriate fixes above, and then track the other things that you have raised as open issues.

Thanks,
Rob

Regards Balazs

Hi,

I appreciate that it is quite short notice, but if it is possible for folks to review and provide comments before Thursday’s meeting that probably be helpful.

Thanks,
Rob