Subject: [NGO] reusable types and groupings

From: Andy Bierman <[email-masked]>
Date: Wed, 13 Feb 2008 06:02:00 -0800

Hi,

There are some important CANMOD requirements that are not
agreed upon, including deep keys, and perhaps reusable complex
structures.

There is a difference between a reusable container on the wire
and a conceptual set of sibling nodes defined in an off-line document.
A container can be processed by shared code that knows
the syntax and semantics of that container.
(Any C programmer knows the difference between 'typedef struct foo_t'
and '#define FOO_ITEMS'.)

The IETF needs the ability to define standard NETCONF models
which include reusable containers (conceptual and on-the-wire),
and the ability to use leafs from those containers in the key for a list.
The examples that keep coming up are quire relevant to networking,
like address/zone index, address/port, etc.

The YANG draft specifies that instances of lists are reordered on-the-wire
so all the leaf nodes are encoded first (instead of where they belong
based on document order), para 2 and 3 below:

    7.8.5.  XML Encoding Rules

       A list is encoded as a series of XML elements, one for each entry in
       the list.  Each element's name is the list's identifier, and its XML
       namespace is the module's XML namespace.

       The list's key nodes are encoded as subelements to the list's
       identifier element, in the same order as they are defined within the
       key statement.

       The rest of the list's child nodes are encoded as subelements to the
       list element, after the keys, in the same order as they are defined
       within the list statement.

There is no explanation in the draft why this done, or considered a feature.
It would obviously never work with deep keys, since removing them
from the sub-tree changes their identity (/name is not the same as /foo/name).

Even if the same grouping is used in 2 different lists, or
in a container, it won't be a reusable container on the wire at all.

    grouping stats_entry {
       leaf src_addr { type smi:InetAddress; }
       leaf dst_addr { type smi:InetAddress; }
       leaf octets { type smi:Counter32; }
       leaf packets { type smi:Counter32; }
       leaf last_active_time { type smi:DateAndTime' }
    }

    container A {
      uses stats_entry;
    }

    list B {
      key "src_addr dst_addr";
      uses stats_entry;
    }

    list C {
      key "dst_addr src_addr";
      uses stats_entry;
    }

    list D {
      key "last_active_time";
      uses stats_entry;
    }

Only 'container A' and 'list B' will be encoded on the wire in document order.
Even worse, there is no way to model the real intent, which
is to have one real stats list, and different ways of sorting the entries.

Andy

_______________________________________________
NGO mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ngo

---

From: "David Harrington" <[email-masked]>
Date: Wed, 13 Feb 2008 14:30:15 -0500

Hi,

Like Andy, I'm a bit concerned about the reordering requirements. I
have three concerns.

1) In RFC3535, operators said they don't want data arbitrarily
reordered:

   10. It is highly desirable that text processing tools such as diff,
       and version management tools such as RCS or CVS, can be used to
       process configurations, which implies that devices should not
       arbitrarily reorder data such as access control lists.

YANG supports "system ordering", which seems counter to the
requirements expressed by operators. 

7.7.4.1.  ordered-by system

   The entries in the list are sorted according to an unspecified
order.
   Thus an implementation is free to sort the entries in the most
   appropriate order.  An implementation SHOULD use the same order for
   the same data, regardless of how the data were created.  Using a
   deterministic order will makes comparisons possible using simple
   tools like "diff".

   This is the default order.

This seems totally counter to the operators' requests.
A deterministic order for making comparisons should be defined by a
vendor-independent interoperable standard that specifies the order,
not a "system" order defined by each vendor.  

2) Simplicity

YANG supports user-ordered lists:

	In the other style of lists, the order of list entries matters
for
   the implementation of the list's configuration and the user is
   responsible for ordering the entries, while the device maintains
that
   order.  YANG calls this style of list "user ordered" and they are
   indicated with the statement "ordered-by user".

   For example, the order in which firewall filters entries are
applied
   to incoming traffic may affect how that traffic is filtered.  The
   user would need to decide if the filter entry that discards all TCP
   traffic should be applied before or after the filter entry that
   allows all traffic from trusted interfaces.  The choice of order
   would be crucial.

   In an "ordered-by user" leaf-list, the attributes "insert" and
   "value" in the YANG namespace (Section 5.4.1) can be used to
control
   where in the leaf-list the entry is inserted.  These can be used
   during "create" operations to insert a new leaf-list entry, or
during
   "merge" or "replace" operations to insert a new leaf-list entry or
   move an existing one.

This strikes me as being too oriented toward a specific implementation
strategy, and I have a concern that a managed node needs to keep a lot
of meta-information about the sort order to be able to return data in
a user-specified order based on a user-ordered linked-list approach. I
think this adds much complexity on the device with little benefit,
compared to defining the data order as part of a standard data model.

Is this something that operators have requested? I didn't see this
requirement in RFC3535.

3) Keep the agent simple

Traditionally, the IETF Management Framework has kept complexity off
the managed device. Where complex functionality is needed, such as
reordering data or deriving values from other values, it is done by
the NMS, not by the managed device. I want my routers routing and my
switches switching, not spending a lot of time reordering buffers of
data (or calculating standard deviations, or mean values, or
whatever).  Sorting the data into whatever order I want depends on the
the specific task I am working on, off the device. I can have my NMS
application on my PC reorder the data, where I typically do not have
network-operations-critical processing occurring, and where I
typically have lots of resources available.

dbh

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Andy Bierman
> Sent: Wednesday, February 13, 2008 9:02 AM
> To: yang; NETCONF Goes On
> Subject: [NGO] reusable types and groupings
> 
> Hi,
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 13 Feb 2008 20:37:35 +0100

> requirements expressed by operators. 

It simply depends on the data. For some data, people are used that the
system puts it into a canonical order, regardless in which order it
was created. A typical example is an IP forwarding table which is
typically ordered by the prefixes.
 
> not a "system" order defined by each vendor.  

This ordered-by system tells us simply that the order is maintained by
the system. It is up to the writer of the description clause to
specify what the order is if there is a commonly used and agreeable
order for things (like in the IP forwarding table case).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
NGO mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ngo

---

From: Andy Bierman <[email-masked]>
Date: Wed, 13 Feb 2008 12:12:18 -0800

> Like Andy, I'm a bit concerned about the reordering requirements. I

This issue with the XML encoding of 'list' is different than the 'insert'
and 'key' XML attribute extensions to the NETCONF protocol.

Those extensions to the <edit-config> operation support the use case
in which the DM writer does not want to use a key, but still
keep a specific sort order, provided by the user.  Data structures
like access control lists could be modeled with 'line numbers'
as the index for each rule, or simply managed by the user instead
without numbering each rule.  This is a more elegant solution
than the SNMP 'arbitrary INTEGER index', and ever requires renumbering
the rules.

The purpose of moving the key leafs from their document order
is to favor agent implementations that do not want to buffer
the incoming NETCONF PDU.  By providing all the naming information
in the first N leafs, the agent can direct the request to
a sub-agent more efficiently, similar to an AGENTX master agent
that may look into a varbind (which always contains all the
naming information for each leaf -- very redundant encoding but
useful for sub-agent dispatching).

IMO, NETCONF <edit-config> PDUs represent such a small percentage
of the bandwidth and memory resources needed by an agent, and
the 'special case' impact on manager applications is not worth
the memory resources on the agent needed to receive an entire
<edit-config> PDU before processing it completely.

The 'key leaf within a container' issue could be considered separate,
but it is related to this PDU buffering issue.

I would rather have a blatant CLR that says key leafs MUST be
the first nodes defined in a list, rather than some weird XML
encoding rules just for YANG lists.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 13 Feb 2008 15:56:48 -0500

>requirements expressed by operators. 

"ordered-by system" means the system defines the rules for ordering
(such as alphabetically), not that the order is arbitrary.  As such,
the order should be consistent as config is fetched over time,
enhancing the ability of text-based tools to work with configurations.

>compared to defining the data order as part of a standard data model.

It has less to do with the implementation than the nature of the
data being modeled.  If I'm modeling login names, the order is
unimportant, but if I'm modeling actions performed inside a firewall
filter, the order of those actions affects the ability to perform
the required filtering.

>I want my routers routing and my
>the specific task I am working on, off the device.

In YANG, the order is either important to the data, in which case
the user/application is responsible for dictating the order, or it
is not important, in which case the system is allowed to sort it
in a natural order.  The order will often be detailed in the
description statement of the list, but will not be arbitrary, random,
or based on nonsense such as standard deviations or mean values.
If your application requires such specific sorting, you are free
to perform this sort after (or as) the data is downloaded from the
device.

Thanks,
 Phil

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 14 Feb 2008 09:48:04 +0100

If we would allow putting the key inside the grouping that would avoid the reordering problem 
as the keys would have a fixed place.

I had a very similar problem recently. I want to use the containers/groupings as managed 
objects (just like the Siemens/Nokia guys). However I am not allowed to put the "key" statement 
inside a grouping which is a problem. It is clearly part of an objects definition/sematics to 
identify which piece of data is the key.

Balazs

>        leaf last_active_time { type smi:DateAndTime' }
...

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Feb 2008 12:35:03 -0800

Hi -

> From: "Balazs Lengyel" <[email-masked]>
> To: "Andy Bierman" <[email-masked]>
> Cc: "yang" <[email-masked]>; "NETCONF Goes On" <[email-masked]>
> Sent: Thursday, February 14, 2008 12:48 AM
> Subject: Re: [NGO] reusable types and groupings
>
...

Agree.  In the CMIP/GDMO world the identification of the "key" took place
in a bit of definition called a "name binding" which was separate from the
class definition.  A single class could employ different name bindings in
different contexts.  In my opinion, this was just plain over-engineered.
I can't recall a single case where it was actually useful to keep the name
binding separate from the class definition.

Back to the "ordering" part of this discussion, I agree strongly with Andy
that it's highly desirable for order to be completely deterministic.  However,
there are a few cases that are interesting to work through in the context of
whatever solution you favor:
   (1) modeling information where there is no semantic value to the order
        (a list of user names, for example) (Long-time SNMP folks put
        a key on it anyway, so get, get-next, etc. will work)
   (2) modeling information where there *is* semantic value to the order,
        but not necessarily a "natural" key by which things might be ordered.
        (access control rules, for example) (Note that long-time SNMP folks
        will unconsciously solve this by adding another key.)
   (3) insertions and deletions from an order due to evolution of a model -
         this is an interaction with how your approach handles versioning of
         models, and interactions between clients and servers employing
         different versions of "the same" model.

Randy

---

From: David Partain <[email-masked]>
Date: Tue, 26 Feb 2008 17:02:52 +0100

Hi all,

Please forgive the cross-posting.  Please strip ngo from any public responses 
and we'll take followup on the YANG list.

Since many of those who are interested in the modeling for NETCONF
question in general and YANG in particular, the YANG gang thinks it'd
be a good idea to spend some time doing some YANG-related work during
the upcoming IETF.

Some ideas:

1. A tutorial on YANG with the latest updates since the last IETF?
2. A modeling session?  It might be that people (like IPFIX and DNS
have already started doing) want to use YANG for modeling their
configuration problem....
3. A CLR sweep?  Go through YANG to get rid of unnecessary crappy little 
rules...
4. Discussion of specific technical questions?  (we'll use
yang-central.org to build the list of questions, in that case)  Could
be things like deep keys, actions/operations, etc..

Our ideas may be great, but it'd be far better if those who want to
come tell us what they want the meeting to be about!  As such, if
you'd like to be there, please respond directly to me and tell me that
you'd like to come and what you'd like the meeting to address in some
kind of prioritized list.

We don't know yet when this will be.  Best guess at this point is
Monday evening.  No, this is _not_ official IETF business.  It's just
those of us who are interested getting together to work because we're all in 
the same place.

Cheers,

David