Subject: [Netmod-ver-dt] Is semantic versioning sufficient?

From: Robert Wilton <[email-masked]>
Date: Tue, 16 Oct 2018 16:13:58 +0100

One of the questions that I've been trying to understand is whether 
standard semantic versioning is sufficient, so I decided to consider a 
scenario where YANG modules are versioned (like an API) entirely 
independently of any release.  Each S/W release then chooses a 
particular module version to ship with.

So, following semver rules (major.minor.patch), then a module could have 
a version history that looks like this:

Module versions: V1.0.0- V1.0.1 - V1.1.0 - V1.2.0 - V1.2.1 - V2.0.0 - 
V2.0.1 - V2.1.0

Software releases (with some arbitrary numbering scheme) might ship 
software release versions as thus:
S/W release R1   ships with module version V1.0.0
S/W release R1.1 ships withmodule version V1.0.1
S/W release R2   ships withmodule version V1.2.0
S/W release R3   ships withmodule version V1.2.1
S/W release R4   ships withmodule version V2.0.0
S/W release R4.1 ships withmodule version V2.0.1
S/w release R5   ships withmodule version V2.1.0

The key questions for me are: What changes MUST we support for one of 
the existing software releases (i.e. at any branch point):
1) Do we have to allow for bc bug fixes in older releases?
2) Do we have to allow for nbc bug fixes in older releases?
3) Do we have to allow for bc enhancements in older releases?
4) Do we have to allow for nbc enhancements in older releases?

My person view is that (1) is REQUIRED. (2) and (3) are nice to haves 
rather than must haves, and probably (4) should generally be avoided.

Alas, basic semver does not allow for any of these!  For some releases 
it might be possible to do 1 or 3, but nothing is guaranteed, it just 
depends on which version of the module happened to ship with that 
release.  The expectation with semver is that it should always be 
possible to force the customer to upgrade to the latest software.  But I 
don't think that is realistic for us.

All of the other solutions that we have been discussing allow for more 
flexibility of options 1 to 4, but at the expense that some of the 
benefits of semantic versioning are lost, or additional complexity is added.

If the requirement that we have to support is (1) and optionally (2), 
then another scheme that is a variant of the semver rules is as follows:

"major.minor.patch(b?)"
major <- MUST be updated for any nbc enhancement, MAY be updated for nbc 
bug fix.
minor <- MUST be updated for bc enhancement, MAY be updated for bc bug fix
patch <- either minor or patch MUST be updated for a bc bug fix.
          either major or patch + "(b)" MUST be updated for an nbc bug fix.
"(b)", for "breaking", is appended to the patch version number if the 
bug fix, or any previous patch bug fix, is an nbc change.

Notes:
- patch is only used for bug fixes of older releases, i.e.:
  - where possible, if a bc bugfix is made (e.g. head of the release 
train) then updating minor is better than updating patch.
  - where possible, if an nbc bugfix is made (e.g. head of the release 
train) then updating major is better than updating patch + "(b)".
  - Cosmetic bug fixes (i.e. those with no semantic change) are not 
distinguished separately and are handled as bc bug fixes.

The only property of the "semantic versioning" scheme that is lost with 
this new scheme is that it is not possible to know when going from A.B.C 
to A.D.E is safe (where C > 0, and D > B).  But I don't think that any 
simple versioning scheme is able of preserving this property, i.e. a 
linear sequence of backwards compatible updates at the same time as 
branching of historic revisions.

Considering the releases above, then the only problem case, where it is 
harder to allow a bug fix is R2 that is on v1.2.0 because R3 is on 
v1.2.1.  But since v1.2.1 is only a bug fix release of v1.2.0 then it 
seems reasonable to put the bug fix on v1.2.2 and upgrade from v1.2.0 to 
v1.2.2.

Thoughts, does this latest scheme have any legs?

Thanks,
Rob

---

From: Qin Wu <[email-masked]>
Date: Wed, 17 Oct 2018 08:27:08 +0000

In our implementation, 1) and 3) are respected, 2) and 4) nbc cases should be avoided.

-Qin
-----邮件原件-----
发件人: Netmod-ver-dt [mailto:[email-masked]] 代表 Robert Wilton
发送时间: 2018年10月16日 23:14
收件人: [email-masked]
主题: [Netmod-ver-dt] Is semantic versioning sufficient?

One of the questions that I've been trying to understand is whether standard semantic versioning is sufficient, so I decided to consider a scenario where YANG modules are versioned (like an API) entirely independently of any release.  Each S/W release then chooses a particular module version to ship with.

So, following semver rules (major.minor.patch), then a module could have a version history that looks like this:

Software releases (with some arbitrary numbering scheme) might ship software release versions as thus:
S/W release R1   ships with module version V1.0.0 S/W release R1.1 ships withmodule version V1.0.1 S/W release R2   ships withmodule version V1.2.0 S/W release R3   ships withmodule version V1.2.1 S/W release R4   ships withmodule version V2.0.0 S/W release R4.1 ships withmodule version V2.0.1 S/w release R5   ships withmodule version V2.1.0

The key questions for me are: What changes MUST we support for one of the existing software releases (i.e. at any branch point):

"major.minor.patch(b?)"

Notes:

Thanks,
Rob

_______________________________________________
Netmod-ver-dt mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod-ver-dt

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 08:48:30 +0000

Hello Rob,

Before I really respond to your proposal I would like some
discussion on the definition of a bug-fix. IMO as long as we do
not know what a bugfix is, what we are speaking about, your
proposal is hard to evaluate. In an earlier email I proposed:

Bug-fix: Any change that does not change the intended and
described working of the interface. Bug fixes are always
considered bc changes. Bug-fixes include:

Updates to the description statement if they don't change
the semantics of the schema-node

Correction type updates to the schema definition if the
original intent was clearly described in the YANG model at
least in the description statement. These types of bug-fixes
are allowed to be formally (accorfding to rfc7950 chapter
11) nbc.

Examples bugfixes  include:

Updating a pattern statement for an IpV4 address that
allows addresses starting with 355

Updating a leafref path that points to a different
object then what is defined in a description

Updating a description statement that doesn't correctly
define the datatype used, like wrong textual definition of a
distinguished name defined in an other RFC.

The implementation behind the model usually (but not always)
rejects incorrect values even if the faulty YANG model allows
it.

The possiblity of bugfixes MUST not be abused to hide
functional updates by labeling them bug fixes. If an
implementation fails to fullfill the contract defined by the
model that in itself does not mean the model is faulty (so
should be fixed), it just means the implementation deviates
from the model.

Cosmetic change: (I don't know if we need the term at all):
Update to the description statement if it does not change the
semantics of the schema-node like correcting spelling mistakes.

Do you/we agree on this? If not please propose an alternative.

The above definition includes the statement that "Bug fixes are
considered bc changes." So there is no such thing as an nbc bugfix
anymore, so your proposal and one of our requirements becomes
moot. As I understood Jurgen was also arguing for a similar
definition when he stated that a regexp correction for an IP
address was backwards compatible. (I hope I didn't misunderstand
him.)

This definition separates the formal compatibility and the real
compatibility, which is a new concept, but I argue it is an
already existing concept implicitly used when you say: bugfix.
IMHO the only thing that separates a bugfix from any other change
is that somehow it is already stated that the model should work
differently (like the RFC for an IPAddress). This existing
description is what should make clients prepared for the change
even if it is formally nbc.

Anyway a definition is really needed !!!

regards Balazs

On 2018. 10. 16. 17:13, Robert Wilton
wrote:

One of
the questions that I've been trying to understand is whether
standard semantic versioning is sufficient, so I decided to
consider a scenario where YANG modules are versioned (like an API)
entirely independently of any release.  Each S/W release then
chooses a particular module version to ship with.

So, following semver rules (major.minor.patch), then a module
could have a version history that looks like this:

Module versions: V1.0.0- V1.0.1 - V1.1.0 - V1.2.0 - V1.2.1 -
V2.0.0 - V2.0.1 - V2.1.0

Software releases (with some arbitrary numbering scheme) might
ship software release versions as thus:

The key questions for me are: What changes MUST we support for one
of the existing software releases (i.e. at any branch point):

My person view is that (1) is REQUIRED. (2) and (3) are nice to
haves rather than must haves, and probably (4) should generally be
avoided.

Alas, basic semver does not allow for any of these!  For some
releases it might be possible to do 1 or 3, but nothing is
guaranteed, it just depends on which version of the module
happened to ship with that release.  The expectation with semver
is that it should always be possible to force the customer to
upgrade to the latest software.  But I don't think that is
realistic for us.

All of the other solutions that we have been discussing allow for
more flexibility of options 1 to 4, but at the expense that some
of the benefits of semantic versioning are lost, or additional
complexity is added.

If the requirement that we have to support is (1) and optionally
(2), then another scheme that is a variant of the semver rules is
as follows:

"major.minor.patch(b?)"

major <- MUST be updated for any nbc enhancement, MAY be
updated for nbc bug fix.

minor <- MUST be updated for bc enhancement, MAY be updated for
bc bug fix

patch <- either minor or patch MUST be updated for a bc bug
fix.

either major or patch + "(b)" MUST be updated for an nbc
bug fix.

"(b)", for "breaking", is appended to the patch version number if
the bug fix, or any previous patch bug fix, is an nbc change.

Notes:

- where possible, if a bc bugfix is made (e.g. head of the
release train) then updating minor is better than updating patch.

- where possible, if an nbc bugfix is made (e.g. head of the
release train) then updating major is better than updating patch +
"(b)".

The only property of the "semantic versioning" scheme that is lost
with this new scheme is that it is not possible to know when going
from A.B.C to A.D.E is safe (where C > 0, and D > B).  But I
don't think that any simple versioning scheme is able of
preserving this property, i.e. a linear sequence of backwards
compatible updates at the same time as branching of historic
revisions.

Considering the releases above, then the only problem case, where
it is harder to allow a bug fix is R2 that is on v1.2.0 because R3
is on v1.2.1.  But since v1.2.1 is only a bug fix release of
v1.2.0 then it seems reasonable to put the bug fix on v1.2.2 and
upgrade from v1.2.0 to v1.2.2.

Thanks,

Rob

[email-masked]

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 08:50:50 +0000

We allow 1) and sometimes (rarely) allow 3) and avoid 2 and 4).

regards Balazs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 11:05:46 +0200

On Wed, Oct 17, 2018 at 08:48:30AM +0000, Balázs Lengyel wrote:
>    Hello Rob,
> 
>    Before I really respond to your proposal I would like some discussion on
>    the definition of a bug-fix. IMO as long as we do not know what a bugfix
>    is, what we are speaking about, your proposal is hard to evaluate. In an
>    earlier email I proposed:

I agree.
 
>      Bug-fix: Any change that does not change the intended and described
>      working of the interface. Bug fixes are always considered bc changes.
>      Bug-fixes include:
> 
>        * Updates to the description statement if they don't change the
>          semantics of the schema-node
>        * Correction type updates to the schema definition if the original
>          intent was clearly described in the YANG model at least in the
>          description statement. These types of bug-fixes are allowed to be
>          formally (accorfding to rfc7950 chapter 11) nbc.
> 
>        * Examples bugfixes� include:
> 
>          � � � Updating a pattern statement for an IpV4 address that allows
>          addresses starting with 355
>          � � � Updating a leafref path that points to a different object then
>          what is defined in a description
>          � � � Updating a description statement that doesn't correctly define
>          the datatype used, like wrong textual definition of a distinguished
>          name defined in an other RFC.

Not sure I understand the third example.
 
>      The implementation behind the model usually (but not always) rejects
>      incorrect values even if the faulty YANG model allows it.
>
>      The possiblity of bugfixes MUST not be abused to hide functional updates
>      by labeling them bug fixes. If an implementation fails to fullfill the
>      contract defined by the model that in itself does not mean the model is
>      faulty (so should be fixed), it just means the implementation deviates
>      from the model.
> 
>      Cosmetic change: (I don't know if we need the term at all): Update to
>      the description statement if it does not change the semantics of the
>      schema-node like correcting spelling mistakes.

If we define this, then I prefer 'editorial change' instead of
'cosmetic change'. But given the first bullet above, this is already
covered as a bug fix (that is, if we define 'editorial change' as
another category, we should remove the first bullet from the
definition of 'bug fix').
 
>    The above definition includes the statement that "Bug fixes are considered
>    bc changes." So there is no such thing as an nbc bugfix anymore, so your
>    proposal and one of our requirements becomes moot. As I understood Jurgen
>    was also arguing for a similar definition when he stated that a regexp
>    correction for an IP address was backwards compatible. (I hope I didn't
>    misunderstand him.)

Yes, I am fine with this. Perhaps add an explicit statement that if
the original intent is not clear, then the original definition is
simply ambiguous and this can't be resolved by a bug fix.

>    Anyway a definition is really needed !!!

I agree that a definition is needed. And I agree that 'nbc bugfixes'
should not exist. An 'nbc bugfixes' means that the intent of the
original definition has not been clear and if you a change to fix
this, we are not talking about a bugfix anymore.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 10:13:28 +0100

Hi Balazs,

My definition of a bug fix is broader than what you describe, and I 
would allow for non backwards compatible changes.  Generally, the non 
backwards compatible changes that I would allow are those where I think 
that relatively few clients are likely to be impacted, and/or the cost 
of fixing should be minimal.

I strongly agree that nbc bug fixes are not ideal, and that we should 
try and minimize them, but sometimes we ship code with bugs in it, and 
the choice is between fixing it properly, or trying to work around the 
bug in a backwards compatible way.  In many cases, I think that fixing 
it properly is the correct choice.

To that end, I think that I would perhaps suggest something like the 
following definition:

Bug fix: Any change to a published YANG model with the aim of solely 
fixing incorrectly specified functionality, without introducing new 
functionality.

Along with what you describe below, I think that bug fixes can include:
  - fixing an incorrect type (e.g. uint32 had been used instead of uint64),
  - fixing a typo in an identity or enumeration name (e.g. 'ivp4' 
instead of 'ipv4'),
  - adding additional constraints on the configuration that may not have 
specified previously,
  - modifying the description, in a way that excludes values that 
previously been unintentionally allowed,
  - making container, leaf or list names consistent.

Thanks,
Rob

On 17/10/2018 09:48, Balázs Lengyel wrote:
>
> Hello Rob,
>
> Before I really respond to your proposal I would like some discussion 
> on the definition of a bug-fix. IMO as long as we do not know what a 
> bugfix is, what we are speaking about, your proposal is hard to 
> evaluate. In an earlier email I proposed:
>
>>   * Examples bugfixes  include:
>>
>>         Updating a pattern statement for an IpV4 address that allows
>>         Updating a leafref path that points to a different object
>>     then what is defined in a description
>> The possiblity of bugfixes MUST not be abused to hide functional 
>> updates by labeling them bug fixes. If an implementation fails to 
>> fullfill the contract defined by the model that in itself does not 
>> mean the model is faulty (so should be fixed), it just means the 
>> implementation deviates from the model.
>>
> anymore, so your proposal and one of our requirements becomes moot. As 
> I understood Jurgen was also arguing for a similar definition when he 
> stated that a regexp correction for an IP address was backwards 
> compatible. (I hope I didn't misunderstand him.)
>
> compatibility, which is a new concept, but I argue it is an already 
> existing concept implicitly used when you say: bugfix. IMHO the only 
> thing that separates a bugfix from any other change is that somehow it 
> is already stated that the model should work differently (like the RFC 
> for an IPAddress). This existing description is what should make 
> clients prepared for the change even if it is formally nbc.
>
> On 2018. 10. 16. 17:13, Robert Wilton wrote:
>> standard semantic versioning is sufficient, so I decided to consider 
>> a scenario where YANG modules are versioned (like an API) entirely 
>> So, following semver rules (major.minor.patch), then a module could 
>> have a version history that looks like this:
>>
>> guaranteed, it just depends on which version of the module happened 
>> to ship with that release.  The expectation with semver is that it 
>> should always be possible to force the customer to upgrade to the 
>> latest software.  But I don't think that is realistic for us.
>>
>> more flexibility of options 1 to 4, but at the expense that some of 
>> the benefits of semantic versioning are lost, or additional 
>> complexity is added.
>>
>>
>> major <- MUST be updated for any nbc enhancement, MAY be updated for 
>> nbc bug fix.
>> from A.B.C to A.D.E is safe (where C > 0, and D > B).  But I don't 
>> think that any simple versioning scheme is able of preserving this 
>> property, i.e. a linear sequence of backwards compatible updates at 
>> the same time as branching of historic revisions.
>>
>> Considering the releases above, then the only problem case, where it 
>> is harder to allow a bug fix is R2 that is on v1.2.0 because R3 is on 
>> seems reasonable to put the bug fix on v1.2.2 and upgrade from v1.2.0 
>> to v1.2.2.
>>
>>
> Mobile: +36-70-330-7909              email:[email-masked]  
>
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 11:19:15 +0200

On Wed, Oct 17, 2018 at 10:13:28AM +0100, Robert Wilton wrote:
> Hi Balazs,
> 
> My definition of a bug fix is broader than what you describe, and I would
> allow for non backwards compatible changes.  Generally, the non backwards
> compatible changes that I would allow are those where I think that
> relatively few clients are likely to be impacted, and/or the cost of fixing
> should be minimal.
> 
> I strongly agree that nbc bug fixes are not ideal, and that we should try
> and minimize them, but sometimes we ship code with bugs in it, and the
> choice is between fixing it properly, or trying to work around the bug in a
> backwards compatible way.  In many cases, I think that fixing it properly is
> the correct choice.
> 
> Bug fix: Any change to a published YANG model with the aim of solely fixing
> incorrectly specified functionality, without introducing new functionality.
> 
>  - fixing a typo in an identity or enumeration name (e.g. 'ivp4' instead of
> 'ipv4'),
>  - modifying the description, in a way that excludes values that previously
> been unintentionally allowed,

Perhaps this works if you only care about vendor modules that are only
implemented on one vendor's gear but clearly this creates trouble in
standards where you have multiple independent implementations. Why
would have a problem with treating an 'nbc bugfix' as a regular 'nbc
change'? You still have the choice to fix things properly.

/js

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 10:42:22 +0100

On 17/10/2018 10:19, Juergen Schoenwaelder wrote:
> standards where you have multiple independent implementations.
That is OK.

I think that we can separately define:
  - what a bug fix is
  - what can be classified as a backwards compatible bug fix
  - what can be classified as a backwards incompatible bug fix.

IETF could state that only backwards compatible bug fixes are allowed in 
standards models.  But equally it might be worth looking back at what 
fixes went into the L3SM YANG module.  I'm pretty sure that some of 
those were not backwards compatible.

>   Why
OK, so I think that this is the crux of the problem.

I like the information of what semver gives you (i.e. using the version 
numbers allows you to detect whether updating between two modules will 
be backwards compatible or not).  I particularly like the aspects that 
this should be machine readable.

However, I can't see any reasonable solution that completely preserves 
semver semantics, but at the same time allows a non backwards compatible 
change on a module version that has been released to the wild.  For some 
schemes it might be acceptable to always put any non backwards 
compatible fixes on to the head of the branch, but I'm not convinced 
that is true in all scenarios.  I'm not even convinced that it is a 
great choice for say the OpenConfig models.  For example if we, as a 
vendor, have implemented modules version 3.0.0 and find an issue, and 
the only solution is to update to version 6.0.0 (along with all the 
dependencies), where the nbc fix is included then that might just be too 
much churn/work.  What we want is 3.0.0 (or something close to that) 
with the fix applied to it.

Hence the latest solution that I proposed yesterday:
  - follow semver rules at the head of the branch
  - but allow for backwards compatible bug fixes to released code 
(semver does not allow this because it uses the same minor version 
number for both backwards compatible enhancements and backwards 
compatible bug fixes).
  - also allow for non backwards compatible bug fixes, but at the cost 
that it "poisons" the branch, and all subsequent fixes on that branch 
are subsequently classified as backwards incompatible.

Thanks,
Rob

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 09:46:29 +0000

Hello Rob,

I am with Jurgen on this. What is the difference between an nbc bugfix 
and a normal nbc change.
In my view it is that somehow the original intent was already 
defined/documented in a manner
that was visible to the client: either in the description or using a 
reference to an external standard.

Taking your example changing a type from int32 to uint64. Why is this a 
bugfix? It might be needed,
it might be a real bug. However if this is only documented in 
cisco-secret-study.ppt or your head or
it is found out during Ericsson's implementation, that uint64 is needed, 
the client has no way to
know this. I argue this is a simple nbc change that the client is 
totally unprepared for.

So it is not the type of the change that I am arguing about, but the 
condition that it must already
be documented in the original model.

regards Balazs

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 11:13:21 +0100

Hi Balazs,

> I am with Jurgen on this. What is the difference between an nbc bugfix 
One is introducing new functionality, the other is fixing existing 
functionality.

> reference to an external standard.
These descriptions are sometimes vague, ambiguous, or wrong.

>
> Taking your example changing a type from int32 to uint64. Why is this 
> a bugfix? It might be needed,
> it is found out during Ericsson's implementation, that uint64 is 
> needed, the client has no way to
OK, say I have a container with 32 uint64 counters, and someone augments 
that container with a uint32 counter.  Assume that in all cases, the 
counters count a subset of line rate packets, and the engineer adding 
the 32 bit counter wasn't aware at how quickly that counter could wrap 
on a 100G link.  It is fairly obvious to me that defining this as a 32 
bit counter was a mistake, a bug, and if it had been caught at review 
they would have changed it to use a 64 bit counter instead.

Or perhaps they used int64 rather than uint64 ... the same idea 
applies.  RFC 7950 is quite clear here, changing the type is an nbc change.

> be documented in the original model.
So the associated description could be "The count of 802.1Q tagged 
packets".  Certainly, I wouldn't expect the type to be embedded in the 
original description.

Thanks,
Rob

>>> would
>>> fixing
>>> instead of

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 12:50:42 +0200

On Wed, Oct 17, 2018 at 11:13:21AM +0100, Robert Wilton wrote:
> counters count a subset of line rate packets, and the engineer adding the 32
> bit counter wasn't aware at how quickly that counter could wrap on a 100G
> link.  It is fairly obvious to me that defining this as a 32 bit counter was
> a mistake, a bug, and if it had been caught at review they would have
> changed it to use a 64 bit counter instead.

Fixing broken design is not a bug fix.

/js

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 11:09:13 +0000

Hello,

While you (correctly) called my criteria "original intent was already 
defined/documented" and the resulting descriptions somewhat vague, 
dispensing with such criteria is worse.

How should a client know that the 32 bit counter was put there by 
ericsson/cisco  on purpose? How can we assume that the client will be 
prepared for such a change? It is only documented in 
cisco-secret-study.ppt. Or should a client review the vendor models and 
second guess what is there? Doesn't work. How would you draw the border 
between changing a counter from int32 to uint64 and changing e.g. ipv4 
to ipv6 or even ipv10. It should have been obvious that we run out of 
IpAddress space.

If the intent was already documented e.g. Ipv4Address with a wrong 
pattern we can assume the client knows that address 355.0.0.1 is 
illegal.We can assume the client is prepared for this formally nbc 
change. IMHO that's the crucial point: can we assume the client already 
knows what should have been in the Yang Module (YAM)?

regards Balazs

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 12:23:58 +0100

> Fixing broken design is not a bug fix.
I think that our definitions here differ.

Specifications have mistakes in them (including IETF's RFCs, or we would 
not have errata, which are 'meant to fix "bugs" in the specification').
Designs have mistakes in them.
Implementations have mistakes in them.

I don't really care what we call them, but our requirement is to fix the 
mistakes in the specification (i.e. YANG module).  I'm not sure that 
saying that it is only OK to fix a subset of mistakes in specifications 
is particularly helpful.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 12:40:38 +0100

On 17/10/2018 12:09, Balázs Lengyel wrote:
> ericsson/cisco  on purpose?
It doesn't.

> How can we assume that the client will be prepared for such a change?
We can't.

Hence my proposal:
  - If the nbc fix is being made to the latest code then you might go 
from 2.3.1 to 3.0.0, indicating that it is a nbc change.
  - However if the nbc fix was to 1.2.1 then you would go to 1.2.2(b) to 
indicate that a nbc bug fix has been made.

So, the client gets to decide whether they are broken.

> It is only documented in cisco-secret-study.ppt. Or should a client 
> review the vendor models and second guess what is there? Doesn't work. 
> How would you draw the border between changing a counter from int32 to 
> uint64 and changing e.g. ipv4 to ipv6 or even ipv10. It should have 
> been obvious that we run out of IpAddress space.
I think that we just apply common sense and judgement.  E.g. 
https://www.ietf.org/blog/iesg-processing-rfc-errata-ietf-stream/ gives 
guidance on how to handle bug fixes to RFCs.

> change. IMHO that's the crucial point: can we assume the client 
> already knows what should have been in the Yang Module (YAM)?
This does not mean that the client will not break due to the change.

If the server accepted the IP address "355.0.0.1" but did just ignored 
the config, and then changed its behaviour to either start rejecting the 
configuration, or to return a different error, then it is possible that 
clients that previously worked (e.g. if the bad IP address was not used 
in any way) will start to break (principally because other configuration 
committed at the same time would also be rejected).

If we have a scheme that allows for nbc bug fixes to older releases then 
I will argue that a stricter interpretation of nbc rules is better, 
because it puts the power to decide whether a change is breaking into 
the hands of the client.

If the scheme does not allow for nbc bug fixes to older releases then a 
loose interpretation of the nbc rules might be regard to fudge around 
the fact that we don't allow nbc bug fixes.

However, the point is slightly moot in that the semver scheme described 
on semver.org does not allow for bc bugfixes to an older release, unless 
the implementation also picks up any bc enhancements that may have been 
made to that branch.  That is partly what my solution was trying to fix, 
but I decided to extend it to also allow for nbc bugfixes.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 13:43:39 +0200

On Wed, Oct 17, 2018 at 12:23:58PM +0100, Robert Wilton wrote:
> Specifications have mistakes in them (including IETF's RFCs, or we would not
> have errata, which are 'meant to fix "bugs" in the specification').
> mistakes in the specification (i.e. YANG module).  I'm not sure that saying
> that it is only OK to fix a subset of mistakes in specifications is
> particularly helpful.
>

There are restrictions on what you can do with an errata and when you
need a new RFC. There will be restrictions of what is a bugfix (that
bypasses the versioning system) and what requires a new version.

/js

---

From: Joe Clarke <[email-masked]>
Date: Wed, 17 Oct 2018 08:52:02 -0400

On 10/17/18 04:48, Balázs Lengyel wrote:
>>         Updating a leafref path that points to a different object then
>>         Updating a description statement that doesn't correctly define
>>     the datatype used, like wrong textual definition of a
> anymore, so your proposal and one of our requirements becomes moot. As I
> understood Jurgen was also arguing for a similar definition when he
> for an IPAddress). This existing description is what should make clients
> prepared for the change even if it is formally nbc.

I agree with this definition.  I like Jürgen's idea of editorial change
vs. cosmetic.

When I was reading Rob's email last night, I was wrestling with this.
These bugfix examples cover a good cross-section of what may be
expected, and they could reasonably be considered BC given this definition.

In light of that, if changes are kept to BC only, then you don't need
the 'b' construct, which I feel is problematic for a couple of reasons:

* 'b' typically denotes beta when applied to versions
* It doesn't compare well.  The idea of a version is that you can
compare to a different version.  If a version is marked 'b', is it NBC
compared to one previous?  What about two previous where that version
also has a 'b'?

We could keep the semver semantics, and have a clear definition of bugfix.

Joe

>> possible to force the customer to upgrade to the latest software.  But
>> I don't think that is realistic for us.
>>

---

From: Joe Clarke <[email-masked]>
Date: Wed, 17 Oct 2018 09:00:35 -0400

On 10/17/18 05:13, Robert Wilton wrote:
>  - fixing a typo in an identity or enumeration name (e.g. 'ivp4' instead
> of 'ipv4'),

These are bug fixes to be sure.  When I look to what would happen if
this were an API, I'd probably see deprecation and proper
changes+obsolescence happening in the next major release.

Remember, we're only talking about NBC bug fixes in older trains.  Would
it not be reasonable (and for the latter case it almost certainly would
be) to offer guidance that these changes must be made in NBC ways in
older trains.  In the latest train, you can bump MAJOR and make the
fixes in place.

Joe

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 13:07:39 +0000

On 2018. 10. 17. 15:00, Joe Clarke
wrote:

Hi Balazs,

Joe

BALAZS: Yes it would be possible, I don't have
real problems with it, but at this point IMHO I think it would be
better to use the same method for all bugfixes. What is the
advantage of this last proposal?

Thanks,
Rob

Hello Rob,

regards Balazs

"major.minor.patch(b?)"
major <- MUST be updated for any nbc enhancement, MAY be updated for
nbc bug fix.
minor <- MUST be updated for bc enhancement, MAY be updated for bc
bug fix
patch <- either minor or patch MUST be updated for a bc bug fix.
either major or patch + "(b)" MUST be updated for an nbc bug
fix.
"(b)", for "breaking", is appended to the patch version number if the
bug fix, or any previous patch bug fix, is an nbc change.

Notes:

Thanks,
Rob

---

From: Joe Clarke <[email-masked]>
Date: Wed, 17 Oct 2018 09:15:44 -0400

On 10/17/18 09:07, Balázs Lengyel wrote:
> 
> On 2018. 10. 17. 15:00, Joe Clarke wrote:
> BALAZS: Yes it would be possible, I don't have real problems with it,
> but at this point IMHO I think it would be better to use the same method
> for all bugfixes. What is the advantage of this last proposal?

I'm trying to rationalize how many NBC bug fixes will be required in
older trains of modules.  It seems like this will be a small set.  In
that case, I'm trying to avoid this 'b' construct that Rob raised.

I'm still rolling things around in my head.  Rob's examples above don't
really meet your definition of bug fix (for all of them at least).  And
it might be hard to look at a potential fix and decide if it's an
allowed NBC bug fix (without a MAJOR bump).

What do you think of Rob's examples above?  Would you think they fit
your idea of a bug fix that could go in without a MAJOR version bump?

Joe

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 14:18:55 +0100

> bypasses the versioning system) and what requires a new version.
OK, but that does not mean that the bug itself is not backwards 
compatible.  It means that nbc bug fixes can only be fixed in new RFCs.

Consider the discussion regarding origin on top level np containers in 
the NMDA draft.  Is that a bug?  Is your proposed rewording of the text 
a bug fix?  Certainly, it is not backwards compatible in that a client 
might have been relying on the origin always existing at the top level ...

Thanks,
Rob

---

From: Balázs Lengyel <[email-masked]>
Date: Wed, 17 Oct 2018 13:26:46 +0000

If the intent was already documented than the change should be allowed 
and considered a bugfix.  E.g. Ipv4Address with a wrong pattern we can 
assume the client knows that address 355.0.0.1 is illegal. We can assume 
the client is prepared for this formally nbc change. IMHO that's the 
crucial point: can we assume the client already knows what should have 
been in the Yang Module (YAM)?

So assuming the intent was already clear you can do anything. :-) Now I 
have a hard time thinking up a way to document that  the whole 
containment hierarchy should be rearranged but "theoretically" it is 
possible ;-)

regards Balazs

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 14:50:54 +0100

> * 'b' typically denotes beta when applied to versions
OK, I'm not wedded to the letter 'b', another alternative I was 
considering was 'p' for "poisoned" because all subsequent bug fixes for 
that major.minor.X branch must always be marked 'p'.

> compared to one previous?
Yes, it doesn't compare as well.  I.e. it is only a partial ordering.  
This is actually true whether or not it has an extra ('b') for breaking 
change.

So A.B.C to A.B.D, where D > C is a backwards compatible change.
A.B.C to A.B.D(b), is a non backwards compatible change.
A.B.C(b) to A.B.D(b), may be a non backwards compatible change.
A.B.C to A.D.E, where C > 0 and B > D may be non backwards compatible 
change.
A.B.C to D.E.F, where D > A is a non backwards compatible change.

>    What about two previous where that version
> also has a 'b'?
>
Do you mean the standard semver semantics, or the modified ones that I 
proposed but without the 'b'?

Because unfortunately vanilla semver does not always allow bug fixes to 
released code.  Sometimes you might be lucky, other times, not.

E.g. consider my example again, but without release 4.1.

Now a customer has deployed R4 and they report a bug in YANG model 
version V2.0.0 that they would like fixed in R4.1 (this is a reasonably 
common scenario for us).  Assume for the moment that a backwards 
compatible bug fix is available.

You can't create V2.1.0 because that already exists, so you would have 
to create V2.2.0 instead, but that already contains new functionality at 
V2.1.0.

In short, standard semver mixes nbc bug fixes and nbc new functionality 
into the major version number, and mixes bc bug fixes and bc 
functionality into the minor version number.

Hence, if follow the standard semver rules that to provide a nbc bug fix 
then customers have to be willing to migrate to the latest major 
version, potentially along with updating other dependent models, new 
hardware, 6+ months delay etc.

And similar principles apply to bc bug fixes as well.  Unless the vendor 
is willing to backport the new functionality introduced in V2.1.0 to 
release R4.1, then they will have to force customers to upgrade to R5.1 
or later (with the same issues, costs, delays, etc).

Going to "release.major.minor.patch" solves the backwards compatible 
problem, but at the expense that you break the main benefit of semver.

Or everything can just ship using 0.A.B and then ignore the semver 
rules, but I also don't see how that really helps clients.

The core of the problem is mixing both new functionality and bug fixes 
into the same version numbers, this gives you linearity but cannot 
support branching.  Conversely, if you allow some branching (e.g. my 
major.minor.bugfix) scheme then it supports branching but loses some of 
the semver benefits.

Another solution is to put the branches into the version number, so if I 
want to put a bc bugfix on V "2.1.3" then the version with the bc fix is 
"2.1.3/1.1.0", but I've presumed that this scheme would be regarded as 
being too complex ...  And it still only gives you a partial ordering, 
i.e. you do not know if going from "2.1.3/1.1.0" to "2.2.0" is a 
backwards compatible change or not.

Thanks,
Rob

> .

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 16:33:25 +0200

On Wed, Oct 17, 2018 at 02:18:55PM +0100, Robert Wilton wrote:
> OK, but that does not mean that the bug itself is not backwards compatible. 
> It means that nbc bug fixes can only be fixed in new RFCs.
> 
> Consider the discussion regarding origin on top level np containers in the
> NMDA draft.  Is that a bug?  Is your proposed rewording of the text a bug
> fix?  Certainly, it is not backwards compatible in that a client might have
> been relying on the origin always existing at the top level ...
>

Yes, and the issue has been raised. I do not know what a client would
do with looking at the origin at the top level but this is a different
discussion and perhaps this can only be fixed by having an update and
not via an errata.

/js

---

From: Robert Wilton <[email-masked]>
Date: Wed, 17 Oct 2018 15:48:16 +0100

> discussion and perhaps this can only be fixed by having an update and
I guess that the key point that I'm trying to make is that:
  - generically bugs can affect specifications.
  - the fixes for those bugs may require non backwards compatible changes.
  - hence non backwards compatible bug fixes exist.

So, I don't agree with any definition that says bug fixes to YANG 
modules are always backwards compatible.  I think that I've already 
provided some reasonable counter examples.

So, I think that what Balazs is actually defining is the subset of bug 
fixes to YANG modules that can be regarded as backwards compatible.

Then I think that it is a separate discussion point as to whether or not 
non-backwards-compatible bug fixes should be allowed for released code.  
>From Cisco's IOS XR perspective, I think that this is a requirement for 
the versioning scheme to be usable on vendor YANG models.  We have to be 
able to support and provide fixes for the code that we have sold to 
customers, saying that there are some classes of bugs that we can't fix 
due to version number restricts is not going to be sufficient.

Thanks,
Rob

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Wed, 17 Oct 2018 14:53:42 +0000

+1

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 17 Oct 2018 17:09:49 +0200

On Wed, Oct 17, 2018 at 03:48:16PM +0100, Robert Wilton wrote:
> So, I don't agree with any definition that says bug fixes to YANG modules
> are always backwards compatible.  I think that I've already provided some
> reasonable counter examples.

A specification error for me is a specification error and not a bug.
We seem to differ what we call a bug and hence our conclusions differ
widely.

> So, I think that what Balazs is actually defining is the subset of bug fixes
> to YANG modules that can be regarded as backwards compatible.

The history have been issues where there was an apparent discrepancy
between say a description clause and say a type definition and whether
RFC 7950 rules would allow us to go and fix the issue. I believe if
the intent was clear, we can step over RFC 7950 rules. I do not think
that design errors can be treated in the same way. So yes, I am making
a distinction here that you do not make.

> From Cisco's IOS XR perspective, I think that this is a requirement for the
> versioning scheme to be usable on vendor YANG models.  We have to be able to
> support and provide fixes for the code that we have sold to customers,
> saying that there are some classes of bugs that we can't fix due to version
> number restricts is not going to be sufficient.

If you need NBC changes on all release trains, you need them. If your
versioning system can't handle them, the versioning system may be not
strong enough to express the things you need.

/js

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Wed, 17 Oct 2018 20:53:58 +0000

Hi guys,

It is likely to be an endless argument to define what is a bug fix vs an enhancement (and even if we agree amongst ourselves, we'll get it all debated again by any new person who reads our drafts). I'd really suggest we stay away from those terms and just focus on NBC vs BC.

Maybe making NBC changes on existing older streams is "bad practice" but we should define a versioning scheme that can indicate when that occurs. We try to avoid it in Nokia but I just can't see it being possible to say it will never happen.

Jason

> -----Original Message-----
> From: Netmod-ver-dt <[email-masked]> On Behalf Of Juergen
> Schoenwaelder
> Sent: Wednesday, October 17, 2018 11:10 AM
> To: Robert Wilton <[email-masked]>
> Cc: [email-masked]; Balázs Lengyel <[email-masked]>
> Subject: Re: [Netmod-ver-dt] Is semantic versioning sufficient? - What is a
> bugfix ?
>