Subject: [Netconf] datastore vs. database

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Aug 2010 13:06:08 +0200

Hi,

the term 'datastore' has been changed to 'database' in the latest
with-defaults ID and I believe this is wrong. As far as I can tell,
datastore or configuration datastore is the term we use in 4741 and
YANG.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Aug 2010 13:18:49 +0200 (CEST)

> datastore or configuration datastore is the term we use in 4741 and

+1.  I think Tom's comment was that the term "configuration datastore"
should be used, which would be fine with me.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Aug 2010 13:51:09 +0200

> should be used, which would be fine with me.

If we talk about the configuration datastore. Where does state data
reside?

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Aug 2010 14:02:33 +0200 (CEST)

> If we talk about the configuration datastore. Where does state data

It is separate, as defined in 4741.  There are three configuration
datastores defined, and state data.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Aug 2010 14:12:47 +0200

> datastores defined, and state data.

So this place where state data resides is also a datastore? If I write
datastore I mean both state datastore and configuration datastore?

Perhaps we need to add a terminology section to 4741bis to make sure
we understand our terms.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Aug 2010 14:16:55 +0200 (CEST)

> datastore I mean both state datastore and configuration datastore?

I do not think so.  I think 'datastore' is used as a shorthand for
'configuration datastore' in a few places in 4741.

> Perhaps we need to add a terminology section to 4741bis to make sure

Yes, and s/database/datastore/ in a couple of places.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Aug 2010 14:17:11 +0200

> If we talk about the configuration datastore. Where does state data

YANG uses "data tree" for instantiated configuration + state data,
perhaps this could be adopted universally.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Aug 2010 14:35:57 +0200

> 'configuration datastore' in a few places in 4741.

We should then avoid the shorthand and be explicit.
 
> Yes, and s/database/datastore/ in a couple of places.

Yep

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Aug 2010 08:19:43 -0700

> perhaps this could be adopted universally.

Maybe you are right.
YANG clearly allows configuration and non-configuration data nodes
to exist as sibling nodes at any level of the conceptual data tree.
NETCONF seems to ignore this fact, even though the content layer
is supposed to be independent of the other layers.

The term 'configuration datastore' is really just a conceptual subset
of the data tree, to refer to the config=true nodes.  Oops -- it is a
conceptual tree only if the <nc:config> node is treated as the root,
even though non-config nodes use this as the root as well.  The <get>
operation is somewhat broken because it returns config=true nodes
under <nc:data> root.

BTW, NETCONF/YANG does not allow for dual-role (config and operational state),
such as a <duplex-mode> leaf that would magically change its value from 'auto'
to 'full' on its own.  There is no text that would suggest this is supported,
therefore a data model designer MUST choose 2 YANG leafs.  This is just a
bad SMIv2 practice that should not apply to YANG.

>> /js

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Aug 2010 11:13:46 -0700

> If we talk about the configuration datastore. Where does state data

Let's try to resolve this once and for all.
The NETCONF content layer is somewhat confusing wrt/ the <get> operation.

YANG clearly allows any number of config=false subtrees at any point
in the conceptual data tree.  NETCONF must support this data modeling
capability.

A <get-config> is clear -- it identifies all the top-level config=true
subtrees internally associated with the requested <source>, and gathers
them into a <nc:config> container.  Any config=false descendant subtrees
are excluded from the response.

A <get> is also clear.  It returns all the running config=true nodes plus
all the top-level config=false nodes and any descendant config=false nodes
within a config=true list or container.  It gathers all these nodes
into a <nc:data> container.

The 'datastore' terminology is not clear because the architecture is not clear.
Why do the conceptual data tree operations work this way?
If the <get> did not include config=true nodes then could any
descendant config=false nodes ever be accessed?

If <get> just returned config=false leaf/leaf-lists/anyxml, and any container/list
ancestor nodes as needed, then the <config> vs. <data> conceptual container
in the <rpc-reply> would make more sense to me.

But that would be a big change in base:1.1.
And the terminology issue still needs to be solved.
Should I do 's/database/datastore/' and call it done,
or do something else in the with-defaults draft?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Aug 2010 21:54:41 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> them into a <nc:config> container.

No, the reply to a <get-config> is sent in <nc:data>.

> Any config=false descendant subtrees

Maybe this could be made to work, but that's not how it currently
works, and I see no reason to do any changes here...

> Should I do 's/database/datastore/' and call it done,

Yes.  Make sure the term is 'configuration datastore'.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Aug 2010 13:36:55 -0700

> Yes.  Make sure the term is 'configuration datastore'.

but that excludes config=false nodes, so I think I like data tree better.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Aug 2010 23:51:24 +0200

> but that excludes config=false nodes, so I think I like data tree better.

My understanding so far was that the configuration datastore is an
abstrace place to store configuration. The format of the storage is
irrelevant.

The data tree is defined as "the instantiated tree of configuration
and state data on a device." This definitions says it is a tree. My
interpretation was that this refers to the XML tree, that is the
serialization of data from a datastore into an XML infoset. That is,
in the case of processing a <get> operation:

      config            state
    datastore         datastore
	|		  |
	V                 |
      data <--------------+
      tree
        |
        v
       XML
  serialization

Sure, one can discuss whether state resides in a datastore or is just
there, or whether even a different term is needed. If we would call it
state datastore, we could refer to any datastore by the term
datastore. Anyway, the point I am trying to make is that for me there
is a difference between a datastore and a data tree. So depending on
the context, the right term should be picked.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Aug 2010 18:01:03 -0700

> the context, the right term should be picked.

I will use the generic 'datastore' -- back to the way it was before
this change.  If the sentence refers to only config=true or config=false
nodes, then configuration or state will be prepended to the term.

Your picture does not account for config=false nodes within config=true
ancestors.

   container foo {
     list bar { config false; ... }
   }

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Aug 2010 19:52:50 -0700

> abstrace place to store configuration. The format of the storage is

OK -- also, the context (input, output, notification, datastore) is
irrelevant.  The use-case for a default for config=false or output,
or notification is simple -- to save bandwidth.  If there is a default
value for a status leaf (quite likely), then why waste bytes on the wire
reporting it?

So what term applies to all the 'places' a default can be applied,
where a default is the value that is used when the leaf is not present.
There should be no need to have different meanings of a default
for different NETCONF content.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Aug 2010 09:06:23 +0200

>      list bar { config false; ... }

Not sure I can follow. If you take my picture, then these nesting
issues are relevant only for the data tree, not for the datastores. A
config false piece will never be in an configuration datastore. So if
a document discusses these nesting issues, it is likely that data tree
is the term it should be using for discussing the issue.

/js

---

From: "t.petch" <[email-masked]>
Date: Fri, 13 Aug 2010 11:24:20 +0200

----- Original Message -----
From: "Juergen Schoenwaelder" <[email-masked]>
To: "Andy Bierman" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, August 12, 2010 11:51 PM
Subject: Re: [Netconf] datastore vs. database

> the context, the right term should be picked.

The other point is that data tree is defined by YANG and not by Netconf.

It is ok for me to have differences between the two if one is needed - for
example, YANG explicitly says that the schema tree is a tree and so the data
tree must be a tree as well.  Netconf I do not think does so, although the
filtering mechanisms come close to implying a tree structure, so if Netconf is
to cater for non-tree data, then data tree would be wrong.

As you have rightly said, my original comment was on the mismatch between the
I-Ds, that datastore is not defined.  Netconf talks of 'configuration and state
data' or 'configuration datastore'.  'State datastore' works for me but I don't
have strong feelings as long as it all hangs together, which for me at present
it does not.

The other related issue is the YANG defines 'data node' as part of the schema
tree and so is the wrong term for whatever it is that has or has not a default
value in the configuration datastore.  Again, Netconf could redefine data node
for its purposes (but I would rather it did not).

Tom Petch

> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Aug 2010 12:49:26 +0200

On Fri, Aug 13, 2010 at 11:24:20AM +0200, t.petch wrote:
 
> It is ok for me to have differences between the two if one is needed
> - for example, YANG explicitly says that the schema tree is a tree
> and so the data tree must be a tree as well.  Netconf I do not think
> does so, although the filtering mechanisms come close to implying a
> tree structure, so if Netconf is to cater for non-tree data, then
> data tree would be wrong.

NETCONF says the content payload is XML and thus the content payload
has a tree structure. That, of course, does not imply that the
datastore necessarily has a tree structure. In other words, the
requirement that the data tree is a tree is hard wired in NETCONF and
not falling out of YANG.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Aug 2010 09:09:27 -0700

> requirement that the data tree is a tree is hard wired in NETCONF and

I changed the term 'database' to 'configuration datastore' or 'datastore'.

I understand that the terminology needs to be as consistent as possible,
but we should be more concerned with client/server interactions.

The YANG spec clearly allows the default-stmt to apply to any context
in which a data-def-stmt is allowed:
  1 - rpc/input
  2 - rpc/output
  3 - datastore (data tree root)
  4 - notification/event-type

The document is heavily biased towards the configuration subset of (3),
but any missing leaf can have a default-stmt applied in its absence.

One of the biggest complaints I keep hearing about NETCONF is that XML
sucks because it is too verbose ( --> slow).  Pruning all 'normal' status
leafs from a notification or rpc/output addresses this problem.

The draft does not cover the server behavior for (2) or (4).
Does a client assume it is the basic-mode behavior?  Or assume there
is no predictable behavior and just add in the default if the leaf is missing?

Andy