Subject: Re: [Netmod-ver-dt] Versioning and features

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Thu, 13 Sep 2018 13:43:17 +0000

Hi Rob,

I'm not sure I'd consider your example as a non-backwards-compatible change (non-BC change) - at least not in the same "class" as the issues we've been discussing. 

The server in release X+1 would not advertise featurette1 so the client is fully aware that the nodes controlled by featurette1 are not available.  I think the advertisement (or not) of features in NETCONF/YANG is a bit orthogonal to these discussions about non-BC changes in the YANG.

Jason

> -----Original Message-----
> From: Netmod-ver-dt <[email-masked]> On Behalf Of
> Juergen Schoenwaelder
> Sent: Monday, July 23, 2018 12:07 PM
> To: Robert Wilton <rwilton=[email-masked]>
> Cc: [email-masked]
> Subject: Re: [Netmod-ver-dt] Versioning and features
> 
> On Mon, Jul 23, 2018 at 04:52:54PM +0100, Robert Wilton wrote:
> > In some of the versioning discussions at IETF, there is the idea of using
> > features to allow backwards compatible changes to a module that might
> not be
> > supported in an older software release.
> >
> > However, it occurred to me that feature statements can be used to break
> the
> > backwards compatible versioning contract.
> >
> > E.g. consider a module that has some configuration enabled under an
> > if-feature "featurette1" statement.
> >
> > If in release X the YANG feature "featurette1" is enabled, but in release X
> > + 1, the YANG feature "featurette1" is not enabled.  For all intents and
> > purposes, a client that is using "featurette1" and moving from release X to
> > release X + 1 would see this an non-backwards compatible change to the
> > configuration model.
> >
> > Similar considerations apply to deviations as well.
> 
> I do not know what "feature enabled in YANG" means.
> 
> A feature is supported by an _implementation_ or it is not. A client
> requiring a certain feature to be supported in order to function
> requires this feature to be supported to function. If an
> implementation removes support for a feature, well the application
> will detect this and stop working.
> 
> Or do you talk about removing a feature from a YANG module? Well this
> makes conditionally mandatory definitions mandatory, this can't break
> interoperability (except that a client checking for the presence of a
> feature would not see this announced anymore).
> 
> I think section 11 of RFC 7950 is fine wrt features. If your intention
> was to say that if we allow uncontrolled changes to features we can
> mess things up, then yes this is likely true.
> 
> /js
> 
> --
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>
> 
> _______________________________________________
> Netmod-ver-dt mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod-ver-dt

---

From: Robert Wilton <[email-masked]>
Date: Thu, 13 Sep 2018 14:57:14 +0100

Hi Jason,

As a client when upgrading a server from version X to version Y, I would 
like to know whether my existing client coded to version X will (i) most 
very likely work unchanged, (ii) almost certainly require some changes 
to interact with the server running version Y.

To answer this question, I need to know not just how the modules have 
changed, but also whether there are any changes in features that are 
enabled on the device between version X and version Y, and also if there 
are any changed in deviations.

Of course, to truly know whether it is going to work, requires testing 
against the new version.

But I think that the next best step is to construct the full schema of 
the device in version X (inc deviations and features), and construct the 
full schema of the device in version Y (inc deviations and features).  
Then compare these two schema trees (optimized to only be the nodes that 
I actually configure, query, or register for notifications of) to 
determine for each node whether (i) it has not been changed at all, (ii) 
it has been changed in a backwards compatible way, (iii) it has been 
changed in a non backwards compatible way (which could include not 
enabling a default in version Y).

This may sound quite complex, but I suspect that it is only solution 
that truly solves the problem the client upgrade problem.  OK, there are 
some alternatives:  one alternatives is to require that all changes are 
backwards-compatible, another is to force all servers to support all 
older revisions of modules (so that the client can continue to interact 
using version X).

Thanks,
Rob

On 13/09/2018 14:43, Sterne, Jason (Nokia - CA/Ottawa) wrote:
> .

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 13 Sep 2018 18:47:11 +0200

On Thu, Sep 13, 2018 at 02:57:14PM +0100, Robert Wilton wrote:
> 
> This may sound quite complex, but I suspect that it is only solution that
> truly solves the problem the client upgrade problem.  OK, there are some
> alternatives:  one alternatives is to require that all changes are
> backwards-compatible, another is to force all servers to support all older
> revisions of modules (so that the client can continue to interact using
> version X).
>

Remove 'all' from the last sentence. A server only needs to support
old modules it wants to provide backward compatibility for. The point
is that the server says "I support versions A,B,C" and clients that
are written against versions A or B or C will thus work without having
to do a full analysis of schema trees. And a client that was written
against D will detect easily that it has likely a problem.

I guess I do not like at all how this is phrased. I suggest "another
is to have servers announce the versions of the modules they support
(so that the client can written for version X can easily determine
whether it will work with the versions supported by a given server).

/js

---

From: Robert Wilton <[email-masked]>
Date: Thu, 13 Sep 2018 18:07:32 +0100

On 13/09/2018 17:47, Juergen Schoenwaelder wrote:
> to do a full analysis of schema trees.
But this only works if there are also no changes in deviations, or 
features being removed between releases.

Or consider a client that currently coded to with some Cisco router, and 
I want to know whether the same config will work with another vendors 
device.  I think that the only way to make this determination is to 
consider module versions, deviations and features.  (Basically, a dump 
of YANG library).

>   And a client that was written
Sorry, I was writing my reply quickly - I wasn't trying to be 
particularly dismissive of this suggestion.

But, I think that having a device supporting a client choosing an 
arbitrary older version of a YANG module (from a set of supported 
versions) is going to be too hard to implement due to the potential 
relationship between different versions of different YANG modules. E.g. 
a client might ask to use "base_infra module v1" along with "advanced 
feature module v2" which actually requires "base_infra v4 or v5".  I 
suspect that it would be only sane if the server could advertise 
particular sets of modules that work together, but then I don't know how 
that would work if one particular protocol daemon had been upgraded.  
I.e. I think that this solution gets very complex very quickly.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 13 Sep 2018 21:11:59 +0200

On Thu, Sep 13, 2018 at 06:07:32PM +0100, Robert Wilton wrote:
> But this only works if there are also no changes in deviations, or features
> being removed between releases.

Yes, like today.
 
> Or consider a client that currently coded to with some Cisco router, and I
> want to know whether the same config will work with another vendors device. 
> I think that the only way to make this determination is to consider module
> versions, deviations and features.  (Basically, a dump of YANG library).

You missed the point. My comment was triggered by 'force all servers
to support all older revisions of modules' which I do not agree with.

Frankly, there are three types of clients:

a) dump clients that blindly use a certain path regardless which
   server they hit (simple scripts written in hurry); these clients
   won't check versions, they likely don't check deviations, and most
   likely do not check features. They surely won't check if a path
   suddenly means something different.

b) naive clients that may do a simple version check but not more (they
   often are an evolution of type a) clients once they start to fail
   badly)

c) smart clients that go have the capability to go through the effort
   to carefully analyse the announced schema, features and deviations
   and then try to do the right thing.

Perhaps it helps if we are clear about which clients we are targeting
with this work.

> Sorry, I was writing my reply quickly - I wasn't trying to be particularly
> dismissive of this suggestion.
> 
> But, I think that having a device supporting a client choosing an arbitrary
> older version of a YANG module (from a set of supported versions) is going
> to be too hard to implement due to the potential relationship between
> different versions of different YANG modules. 

Wrong again. The client is not "choosing an arbitrary older
version". The client is written against a specific version (or a set
of versions) and it wants to check whether the server has a version
that matches the client's expectations. The key is to signal in a
simple way "we do not get along, lets better stop".

> E.g. a client might ask to use
> "base_infra module v1" along with "advanced feature module v2" which
> actually requires "base_infra v4 or v5".  I suspect that it would be only
> sane if the server could advertise particular sets of modules that work
> together, but then I don't know how that would work if one particular
> protocol daemon had been upgraded.  I.e. I think that this solution gets
> very complex very quickly.

That is the nature of it. Versioning is hard. I agree that meaningful
packages help to reduce the pain.

/js

---

From: Robert Wilton <[email-masked]>
Date: Fri, 14 Sep 2018 11:19:08 +0100

>> being removed between releases.
I don't think that YANG guarantees this at all.  Different releases of 
software are at liberty to enable different features and to implement 
different modules (and hence deviations).  Hence to determine whether a 
client could migrate from one software release to another requires that 
it takes into account implemented module and their versions, features 
and deviations.  Just considering module versions alone only covers part 
of the story.

E.g. one of Chris Hopp's suggestions for vendors managing YANG was:

1. Version the YANG entirely independently from device software release 
trains.
2. Always add new code (major or minor) to the latest version of the 
schema (even if it is required in an older release of the device software).
3. Always put new changes under if-feature so that module can be used in 
older releases with unsupported features not turned on (e.g. allowing 
new functionality to be added to older releases through careful control 
of features).
4. Over time, once a feature is supported in all releases the if-feature 
statement can be removed.

With this scheme, it seems very likely that a feature would end up being 
enabled in an software release A.B, but is not enabled in release 
A+1.C.  Although, I have to confess that I wouldn't really like to see 
features used in this way.

Separately, for deviations, I can easily imagine that they could vary 
depending on what type of linecard is interested, and what hardware 
capabilities it has.  Actually, it is arguably worse that this in that 
the schema may not be changed, instead the configuration gets rejected 
at runtime because of dynamically imposed constraints that are not 
expressed in the YANG schema.

So, as I see it, classical Semver ensures that a client can go from 
version X.A to version X.B, where B > A.  But it makes no guarantees 
about going from X.A to Y.C, where Y > X,  and A and C could be 
anything.  And any change in features or deviations can break everything.

I'm trying to look for a solution that allows a client to determine 
whether it can move from using one arbitrary release to another 
arbitrary release (different modules, module versions, deviations, 
features).  Because ultimately, I think that the backwards compatible 
check needs to be done against the constructed schema trees.  This 
doesn't mean that this needs to be done at run time, vendors could 
publish the YANG library information independently of the release.

> Perhaps it helps if we are clear about which clients we are targeting
I'm not sure.  I would guess that I have most interest in (c).  I.e. my 
particular interest is in clients that work in a robust and reliable way 
with the servers that they manage, even if the clients or servers fail 
and reboot, or connectivity is lost at an inconvenient time, etc.

If I was writing a client, I would aim to write to my own operator 
specific schema (based on OC or IETF) that is independent of the 
devices, and then map that schema to what the underlying 
devices/versions support.  Where the devices faithfully implement the 
same base schema the mapping is trivial, where the don't some mapping 
will be required.  My aim here would be to abstract away the operator 
manageability from variations in differences between vendors and vendor 
software releases.

However having said that, my suggestion of being able to perform a 
comparison between the constructed schema seems like an off the box 
thing and could work equally well for any of the clients above.

> simple way "we do not get along, lets better stop".
 From a clients perspective it is not choosing an arbitrary older 
version, but from a server's perspective, that could be interacting with 
many different clients, then effectively each client could be choosing 
an different arbitrary set of modules.

My concern is not the query about whether the modules are compatible if 
the server only supports a single revision of each module, although then 
I would think that the client could just check YANG library to see if it 
is compatible.

My concern is if there is an expectation that a server will be able to 
interact with clients using different versions of modules.  I think that 
the only plausible way that this can work would be if the server can 
choose sets of consistent modules that the client can interact with, and 
the client chooses from these sets.  I don't think it works if a client 
can just choose which version of each module it wants to interact to the 
server with.

Note that I also think that my suggestion of comparing schema trees also 
solves this problem. But what I am suggesting achieves this in a way 
that also handles features and deviations, and can be done off the box.  
It has the added benefit that it also works if one of the modules is 
going across a major version, where semantic versioning can't help.  Not 
only does it tell whether or not the software is compatible, it can also 
list the specific paths that are incompatible (which can be filtered by 
the paths that the client is actually interested in).

I'm not saying that we shouldn't have some sort of module level version 
number, but I am saying that I don't think that it truly solves the core 
problem here.

> That is the nature of it. Versioning is hard.
I completely agree that this is a hard problem to solve.

>   I agree that meaningful
Yes.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 14 Sep 2018 12:51:41 +0200

On Fri, Sep 14, 2018 at 11:19:08AM +0100, Robert Wilton wrote:
> client could migrate from one software release to another requires that it
> takes into account implemented module and their versions, features and
> deviations.  Just considering module versions alone only covers part of the
> story.

What I was saying is that this is what we have today. Good clients
should check features they depend on and good clients should check
deviations affecting them.
 
> 2. Always add new code (major or minor) to the latest version of the schema
> (even if it is required in an older release of the device software).
> older releases with unsupported features not turned on (e.g. allowing new
> functionality to be added to older releases through careful control of
> features).
> enabled in an software release A.B, but is not enabled in release A+1.C. 
> Although, I have to confess that I wouldn't really like to see features used
> in this way.

The feature removal step above is broken. Clients checking the
presence of a certain feature will freak out when the feature
disappears.

> capabilities it has.  Actually, it is arguably worse that this in that the
> schema may not be changed, instead the configuration gets rejected at
> runtime because of dynamically imposed constraints that are not expressed in
> the YANG schema.

Well, there is never a guarantee you can derive from the schema that a
certain edit will succeed. Hence, an implementation may just not go
and analyse deviations and simply try as it needs to be prepared to
handle errors anyway. Checking the server implements a matching
version and the necessary features a client needs may practically be
sufficient.

> So, as I see it, classical Semver ensures that a client can go from version
> X.A to version X.B, where B > A.  But it makes no guarantees about going
> from X.A to Y.C, where Y > X,  and A and C could be anything.  And any
> change in features or deviations can break everything.
> 
> I'm trying to look for a solution that allows a client to determine whether
> it can move from using one arbitrary release to another arbitrary release
> (different modules, module versions, deviations, features).  Because
> ultimately, I think that the backwards compatible check needs to be done
> against the constructed schema trees.  This doesn't mean that this needs to
> be done at run time, vendors could publish the YANG library information
> independently of the release.

If the client has to construct and analyse the semantics of an entire
schema tree, you do not need any version numbers anymore. The problem
is that (a) few clients will do this and (b) determining semantic
changes likely ends up in an unsolvable problem and you will need lots
of schema annotations. This sounds like a solution that may exist in
theory, I fail to see that this will work in practice. None of the
software I have ever used was able to automatically adapt to API
changes.

> with the servers that they manage, even if the clients or servers fail and
> reboot, or connectivity is lost at an inconvenient time, etc.
> 
> If I was writing a client, I would aim to write to my own operator specific
> schema (based on OC or IETF) that is independent of the devices, and then
> map that schema to what the underlying devices/versions support.  Where the
> devices faithfully implement the same base schema the mapping is trivial,
> where the don't some mapping will be required.  My aim here would be to
> abstract away the operator manageability from variations in differences
> between vendors and vendor software releases.
> 
> comparison between the constructed schema seems like an off the box thing
> and could work equally well for any of the clients above.

I continue to believe that there will be very different clients and we
should be clear for which clients we provide a solution. A hyper smart
AI driven client that can automatically adapt to API changes does not
need version numbers and versioning rules. I believe we are done if
this is our target.

> From a clients perspective it is not choosing an arbitrary older version,
> but from a server's perspective, that could be interacting with many
> different clients, then effectively each client could be choosing an
> different arbitrary set of modules.
> 
> My concern is not the query about whether the modules are compatible if the
> server only supports a single revision of each module, although then I would
> think that the client could just check YANG library to see if it is
> compatible.
> 
> interact with clients using different versions of modules.  I think that the
> only plausible way that this can work would be if the server can choose sets
> of consistent modules that the client can interact with, and the client
> chooses from these sets.  I don't think it works if a client can just choose
> which version of each module it wants to interact to the server with.

This is a business decision. Next version of MacOS will not run 32-bit
applications anymore. It will not support older graphic cards anymore.
This is a business decision. Cisco may drop support for certain YANG
modules that stop certain clients to function. So be it.

> solves this problem. But what I am suggesting achieves this in a way that
> also handles features and deviations, and can be done off the box.  It has
> the added benefit that it also works if one of the modules is going across a
> major version, where semantic versioning can't help.  Not only does it tell
> whether or not the software is compatible, it can also list the specific
> paths that are incompatible (which can be filtered by the paths that the
> client is actually interested in).
> 

Your proposal needs a proof that this is feasible in general (which I
doubt) and if it works, it may only be a solution for type c) clients.
I understand you do not care about type a) and b) clients, but these
are the ones that actually need versioning rules.

/js

---

From: Robert Wilton <[email-masked]>
Date: Thu, 20 Sep 2018 14:53:09 +0100

Hi Juergen,

Sorry for the late reply, it has been a busy week.

> deviations affecting them.
OK, but I don't this this solves the problem that I'm hoping to solve 
which is:

I have a client using software version A (with modules at particular 
versions, features, and deviations).  I would like to know whether I can 
move to software version B (with different modules at particular 
versions, features, and deviations).  I.e. I would like to know whether 
anything is going to break, and if so what it will be.  I'm particularly 
considering trying to achieve this offline (assuming the YANG library 
information is available), but obviously it could also be done when 
connecting to a device.

One variant of this is to assume that a A and B are different devices by 
different vendors.
Other variant is that this is two software revisions of the same device 
by one vendor.  I think that this is an open question here as to whether 
it is safe to assume a feature will never be removed, and deviations 
won't change.  Note - I don't think that this assumption is safe.

> presence of a certain feature will freak out when the feature
Putting this in temporal example:

Software release 1.0 was released on Jan 1st 2017.
At a customer's request, a new YANG feature "foo" is added to software 
1.1 in Sept 2017.
Software release 2.0 was released on Feb 1st 2018.  The YANG for 2.0 
includes YANG feature "foo" but the code doesn't support it yet so it 
isn't enabled.
Software release 2.1, released on 1st June 2018 enables feature "foo".

So, if a customer tries to upgrade from 1.1 to 2.0 they will see that 
feature "foo" disappears, but they could upgrade from version 1.1 to 
version 2.1.

I would like to be able for a client to easily determine whether they 
can upgrade from release A to release B, and if so what the impact would be.

> software I have ever used was able to automatically adapt to API
I'm thinking more of a generic offline tool (e.g. pyang + offline YANG 
library data) rather than an online script.

E.g. I'm thinking that at the time that an ISP is considering a new 
software version, being able to use the tooling to determine which 
scripts will need changes, and which scripts will continue to work as is.

In terms of the problem:
  - I'm assuming that we do have a module level semantic version that 
indicated non-backwards-compatible changes.
  - In addition to that I'm thinking of an annotation to "description", 
"must", "when" that references to the same module level semantic version 
whenever any non-backwards compatible changes are made.

I'm definitely not proposing lots of extra per data node annotations, 
only where they are actually necessary.   All other non backwards 
compatible changes can be spotted via tooling following the RFC 7950 
module update rules.

> need version numbers and versioning rules. I believe we are done if
OK, I think that there are two problems being considered here:
(1) Can we determine whether a client will break (either ahead of time, 
or when it connects).
(2) Can we handle older clients without them being impacted.

> are the ones that actually need versioning rules.
The tooling that I was considering would/could be an offline thing. I 
would think that it could be applicable to all client types, if the 
person managing those scripts know which versions they are intended to 
target.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Sep 2018 18:16:27 +0200

On Thu, Sep 20, 2018 at 02:53:09PM +0100, Robert Wilton wrote:
> move to software version B (with different modules at particular versions,
> features, and deviations).  I.e. I would like to know whether anything is
> going to break, and if so what it will be.  I'm particularly considering
> trying to achieve this offline (assuming the YANG library information is
> available), but obviously it could also be done when connecting to a device.
> 
> Other variant is that this is two software revisions of the same device by
> one vendor.  I think that this is an open question here as to whether it is
> safe to assume a feature will never be removed, and deviations won't
> change.  Note - I don't think that this assumption is safe.

[...]

> At a customer's request, a new YANG feature "foo" is added to software 1.1
> in Sept 2017.
> includes YANG feature "foo" but the code doesn't support it yet so it isn't
> enabled.
> feature "foo" disappears, but they could upgrade from version 1.1 to version
> 2.1.
> 
> I would like to be able for a client to easily determine whether they can
> upgrade from release A to release B, and if so what the impact would be.

With release, I assume you mean software artefacts that incorporate
YANG models but many other things. What is meaningful for us here is
that you have multiple schemas and you want to check which ones
satisfy the requirements of a client. Regarding features, if the
client needs feature foo, then only schemas with feature "foo" enabled
qualify. If the client has further version requirements, then the YANG
modules should match these version requirements. In general, it is
probably safe to assume that clients often use only subsets of a
schema. If that subset is known, then you could dig deeper - it is a
question of how granular the version check is going to be.

And operational people likely often think in the version numbers of
the software artefacts ("os releases") and not in terms of yang schema
and collections of versions YANG modules. Back in SNMP land, it was
quite common I think that clients simply checked the "os release" and
then decided - if they checked things at all.

This is why I think it is important that we keep in mind for which
kind of clients we produce a solution. It might be that a super smart
solution won't need version numbers and ignorant clients won't care
anyway. So version numbers likely have the value for the more regular
clients that are willing to do a reasonable effort to check version
compatibility - but not a great effort that requires to analyze
multiple schemas.

> E.g. I'm thinking that at the time that an ISP is considering a new software
> version, being able to use the tooling to determine which scripts will need
> changes, and which scripts will continue to work as is.

Sounds good but requires knowledge about what the scripts actually
need in order to function. Scripts tend to grow, they tend to have
conditional code (if I talk to A I use model A, if I talk to B I use
model B) and so on. So somehow you need a formalization of the subset
of a schema needed by a script to run your new tools.
 
> I'm definitely not proposing lots of extra per data node annotations, only
> where they are actually necessary.   All other non backwards compatible
> changes can be spotted via tooling following the RFC 7950 module update
> rules.

Well, you may change pattern such that they look substantially
different but still have the same effect. So it is going to be some
effort for a program to decide whether two definitions are the same or
not.

> (1) Can we determine whether a client will break (either ahead of time, or
> when it connects).

Yes. And at connect time, the check must be reasonably efficient to
code, execute and provide a reasonably robust answer. For the ahead of
time check, you have less constraints but you likely need some
formalization of the schema needed by a client to work and over time
this may be a collection of different versioned schemas.

> (2) Can we handle older clients without them being impacted.

Unless someone has a clever idea, there is likely no other way as
keeping old versions supported on the server until there is no
business case anymore to do so.

> The tooling that I was considering would/could be an offline thing. I would
> think that it could be applicable to all client types, if the person
> managing those scripts know which versions they are intended to target.

Yes, an offline tool may have more time and resources, but even then
there will likely be limits. But if you manage to handle 80% or 90%,
the tool likely has already significant practical value.

/js

---

From: Robert Wilton <[email-masked]>
Date: Thu, 20 Sep 2018 18:09:54 +0100

> then decided - if they checked things at all.
Yes.  For what I propose to work, the vendor would need to publish some 
sort of information equivalent to YANG library for each release artifact.

> compatibility - but not a great effort that requires to analyze
OK, I agree.

I guess that I'm thinking of different layers/sophistication of solution.

For regular clients, a module level version may be sufficient.  A module 
level version number of its own still has some value, and can help 
identify problems (but only at a coarse level).

The per data-node tree comparison that I describe would be an enhanced 
version of the solution above and beyond the module level version, but 
to get its greatest value I think that it would need the additional 
annotations to cover the cases that where a reasonable tool cannot spot 
it.  I also don't think that this necessarily needs to be solved at the 
same time (but I would probably push harder for it if we end up doing a 
major revision of YANG).

> of a schema needed by a script to run your new tools.
Yes that would be the ideal.  The more tightly the set of impacted nodes 
can be defined the more useful the tool becomes, in that it would give 
less false positives.

> effort for a program to decide whether two definitions are the same or
OK, so it looks like "pattern" would also need to be added to the list 
of stmts that can also be annotated.  Hopefully there are not too many 
more.  I would not want to force a tool to have to do really complicated 
comparisons to determine whether it is a non-backwards-compatible 
change, but I'm thinking that a tool can check the more basic ones.

Another possible variant might to not do any extra per datanode 
annotations at all, and instead the tool always flags anything that 
might be a non-backwards-compatible change.  Or perhaps the annotation 
is used to mark that it is a backward compatible change (again only for 
description, must, when, pattern, etc), with the default, perhaps safer, 
assumption that it is not backwards compatible.

> business case anymore to do so.
I agree, but at the moment, I'm not aware of any vendors doing this (at 
least for the native YANG models).  As far as I am aware that for a 
given release of the software they just support a single revision of a 
new module, and the burden is put on the client to cope with this.  
Where possible we try and the changes backwards compatible, but there 
are many cases where we break this, mostly when hardening YANG 
statements covering new functionality.

Even if we define a mechanism that allows a server to publish other 
versions that are supported it is still unclear to me whether this part 
of the spec would get implemented.  I think that we probably need to try 
and keep this part as simple as possible to have any chance of getting 
adoption.

> the tool likely has already significant practical value.
Yes, I would think so.

Particularly if vendors do not support older versions, then a good tool 
telling them how they may be impacted is likely to be beneficial.  Cisco 
already uses a script + pyang to achieve some of this, but IIRC the 
output that it generates is quite long between two releases, I think 
that there are too many false positives, and probably it needs to be 
restricted to the subset of the schema that is being used to be really 
useful.  The tool is also slow to run (30+ minutes), but I'm not 
convinced that has to be the case.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Sep 2018 20:27:56 +0200

On Thu, Sep 20, 2018 at 06:09:54PM +0100, Robert Wilton wrote:
> version of the solution above and beyond the module level version, but to
> get its greatest value I think that it would need the additional annotations
> to cover the cases that where a reasonable tool cannot spot it.  I also
> don't think that this necessarily needs to be solved at the same time (but I
> would probably push harder for it if we end up doing a major revision of
> YANG).
>

In SMI land, we had tools like smidiff that report changes that break
the backwards compatibility rules are errors or warnings. I assume
there are similar YANG diff tools these days. Now you need just to
take this an restrict it to the stuff used by a given client. For such
a tool, what you really want is an annotation that indicates that
something is backwards compatible even though the definition changed
so that tools stop producing false positives.

> least for the native YANG models).  As far as I am aware that for a given
> release of the software they just support a single revision of a new module,
> and the burden is put on the client to cope with this.  Where possible we
> try and the changes backwards compatible, but there are many cases where we
> break this, mostly when hardening YANG statements covering new
> functionality.

In SMI land, vendors did very well support old modules. Perhaps in
YANG this will be different if vendors are consistent in not making
backwards compatibility promises and the market is OK with that. BTW,
do you support openconfig and IETF and <you name it> overlapping
models? It the same thing, no?

> Even if we define a mechanism that allows a server to publish other versions
> that are supported it is still unclear to me whether this part of the spec
> would get implemented.  I think that we probably need to try and keep this
> part as simple as possible to have any chance of getting adoption.

Well, market forces will decided what you do.

> already uses a script + pyang to achieve some of this, but IIRC the output
> that it generates is quite long between two releases, I think that there are
> too many false positives, and probably it needs to be restricted to the
> subset of the schema that is being used to be really useful.  The tool is
> also slow to run (30+ minutes), but I'm not convinced that has to be the
> case.

It likely should go much faster but at the end the runtime does not
matter that much since you do not do this constantly I suppose.

/js