Subject: [i2rs] [[email-masked]: I-D Action: draft-haas-i2rs-ephemeral-state-reqs-00.txt]

From: Jeffrey Haas <[email-masked]>
Date: Tue, 26 May 2015 15:40:42 -0400

Working Group,

The following document is a straw-man document attempting to document
changes necessary to implement I2RS's ephemeral state needs.

This document has already gotten some discussion off-list with several of
the usual contributors to the netmod and netconf Working Groups as an
attempt to gain some early traction in the discussion.  While that
discussion has been energetic, it hasn't achieved any further consensus from
the contents posted herein.  

I would like to request that further discussion related to this draft take
place on the mailing list.

This draft will be a topic for tomorrow's virtual interim.  My apologies for
not posting this more promptly.

-- Jeff

----- Forwarded message from [email-masked] -----

Date: Tue, 26 May 2015 12:14:57 -0700
From: [email-masked]
To: [email-masked]
Subject: I-D Action: draft-haas-i2rs-ephemeral-state-reqs-00.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : I2RS Ephemeral State Requirements
        Author          : Jeffrey Haas
	Filename        : draft-haas-i2rs-ephemeral-state-reqs-00.txt
	Pages           : 9
	Date            : 2015-05-26

Abstract:
   This document covers requests to the netmod and netconf Working
   Groups for functionality to support the ephemeral state requirements
   to implement the I2RS architecture.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-haas-i2rs-ephemeral-state-reqs/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-haas-i2rs-ephemeral-state-reqs-00

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

----- End forwarded message -----

---

From: Kent Watsen <[email-masked]>
Date: Wed, 27 May 2015 03:22:32 +0000

I likely won't be able to make tomorrow's interim meeting.  Hopefully this
helps.

In the off-list discussion Jeff mentioned, it was stated that using
distinct datastores avoids architectural problems and special purpose
extensions.  This is a perspective I'm coming to have as well.

Section 7.1 of draft-haas-i2rs-ephemeral-state-reqs-00 describes
disadvantages to having multiple datastores, namely issues around
instance-identifiers and must/when expressions. My opinion is that since
I2RS is defining a new concept, it can also define the necessary semantics
to enable what's needed.  While it's true that normal datastores can't
reference each other, these are not normal datastores.   To underscore
this, if I2RS selects RESTCONF, NETCONF's datastores are no longer
relevant.  In fact, the term "datastore" may be tripping us up, perhaps we
call it an "overlay" ;)

So Section 7.2 of draft-haas-i2rs-ephemeral-state-reqs-00 describes
disadvantages to the overlay approach, namely complexity and consistency
issues.   Neither of which I fully understand.   The overlay approach was
discussed on list last September and again in October.  It actually seemed
to have a lot of support, albeit some minor issues, but they don't seemed
nearly as bad as issues discussed in other proposals to date.

My proposal is to revisit the overlay approach again, defining the
necessary rules as needed.  Taking a step into design, I propose not just
one overlay, but N overlays for N priorities as follows:

  - each client has an assigned priority
  - each overlay has an assigned priority
  - clients can read lower priority overlays, but not higher priority
overlays
  - each client writes into the overlay of matching priority (no
exceptions)

  - the system calculates the effective configuration using this algorithm:

    initialize accumulation buffer with NV-config
    for priority from lowest to highest:
      merge into accumulation buffer overlay[priority]
      assert conceptual `validate` on accumulation buffer succeeds

Where the merge operation includes the following rules:
  - merging a scalar value overwrites lower-priority values
  - merging an ordered-by user list entry causes the entry goes to
    The beginning of the list (this assumes an ordering semantic, a
    YANG annotation may be needed to indicate directionality)
  - it is not possible to simply delete a lower-priority value

And client interactions with an overlay includes the following rules:
  - any update (including on NV config) that might cause the
    conceptual `validate` on the accumulation buffer to fail has
    the effect of necessary config to be first *copied* into the
    higher-priority overlay.

The primary advantages to having multiple overlays are
  - priority doesn't need to be stored per leaf
  - issues around how lower priority client updates interact
    with higher priority config disappear.

One problem with the approach of having multiple overlays (as oppose to
just one) is that it has the consequence of re-exposing a lower-priority
value when a higher-priority value is removed...unless we insist that
writing higher-priority value wipes out matching lower-priority value (not
including NV-config, of course).   What's unnerving about re-exposing a
lower-priority ephemeral value is that one might argue why the system
didn't instead restore an ephemeral value from a client with the same
priority instead.  I assume that two clients of equal priority would have
their own way of resolving it, or else I2RS disallows clients having equal
priorities...

Thanks,
Kent

On 5/26/15, 12:40 PM, "Jeffrey Haas" <[email-masked]> wrote:

>i2rs mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/i2rs

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 May 2015 23:40:06 -0400

We discussed saving and re-exposing lower priority operations when 
higher priority ones were removed.  The working group decided that was a 
bad idea, as it has a tendency to produce unintended consequences.

Also, all direct over-writes among I2RS operations are considered 
errors.  They will need to produce notifications of these errors.  And 
then the higher priority one is applied as a means of producing 
deterministic results, not because it is reliably right.

Also, if an I2RS client has read permission on I2RS data, it reads the 
current value.  No matter what priority the value was set using.

So I would be really unhappy with trying to model this as an overlay per 
priority.  It seems to produce even more complexity, more unexpected 
results, and does not help us.

If I2RS data can be modeled as a single datastore with read-through to 
the underlying operational state, and no possibility of performing a 
commit then it can probably be made to work.

Yours,
Joel

---

From: Kent Watsen <[email-masked]>
Date: Wed, 27 May 2015 16:52:22 +0000

>bad idea, as it has a tendency to produce unintended consequences.

Right, which is why I wrote that writing higher-priority value could
wipeout lower-priority values, not simply shadow them.

>deterministic results, not because it is reliably right.

I didn't mention notifications, but I think what you write is consistent
with my proposal.

>current value.  No matter what priority the value was set using.

There is the difference between config and op-state.  I didn't mention
op-state but I agree with your statement.

>results, and does not help us.

To me it seems easier with more predictable results.  That said, multiple
overlays could be characterized as just an implementation detail, as it
doesn't impact an I2RS Client's interactions with an I2RS Agent.

>commit then it can probably be made to work.

Right, the overlay model gives us this.  We seem to agree, only I'm not
calling it a "datastore" for reasons described in my previous message.

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2015 11:08:52 -0700

> relevant.  In fact, the term "datastore" may be tripping us up, perhaps we

I think it is important that tools do not have to hack the path-expr
in the Location URI of I2RS data when it is created.

   /restconf/data/foo/bar=1?datastore=datastore-name

This approach allows the URI for the resource to be the same
in all datastores, without the need to parse and patch the URI
to access each datastore.

Overlay implies patch or delta from the running config.

>   - each client writes into the overlay of matching priority (no

I agree with Joel that the requirements as written exclude this approach.

IMO it would be better to understand how running-config
and ephemeral-config interact to form operational state.
The derivative case are these 2 panes.

It makes a big difference whether the ephemeral pane contains a patch
or a copy of the running config.

module foo {
   leaf enable-foo {
      description "Enables some routing feature";
      type boolean;
      default true;
   }

   container foo {
      config false;
      i2rs:ephemeral true;
      when "/enable-foo";
       ...
   }
}

Does the 'enable-foo' leaf exist in the ephemeral datastore?

There seems to be 2 approaches:

   1) overlay
      - the when-stmt context for ephemeral is ephemeral first,
        and then config
      - any data in the running config but not in the overlay is visible

    2) mirror
       - ephemeral starts as copy of running
       - data changed in running may or may not be automatically
        visible in ephemeral (could be operator selected)
      - when-stmt applies to the ephemeral datastore only

What happens if the 'enable-foo' leaf gets deleted from the ephemeral
datastore? According to YANG, the default of 'true' would be in effect.
But this is not the desired behavior for an overlay model.
The value in the running config is supposed to go into affect,
not the default value in the ephemeral config.

It seems that a client needs to include all the data related to an edit
so it will be tagged with the correct priority.  E.g., the caline needs to
set 'enable-foo' even if it has the desired value already, to make sure
a lower priority client cannot change it.

Andy