Subject: partial locking

From: "David Harrington" <[email-masked]>
Date: Thu, 30 Aug 2007 11:56:20 -0400

Hi,

For partial locking, since the lock applies not only to Netconf but to
other NM interfaces as well, it is imperative that the managed entity
understand just what **instrumentation** is being partially locked,
not just what Netconf data model subset is being partially locked. 

In SNMP, we use strings to identify a "naming scope" - a community or
a context, and it is imperative that the agent understand what data is
in the community or the context. It is the implementation of the
managed entity that determines just what distinct communities or
contexts exist within the managed entity, and an agent knows about the
data subsets, even if there are dynamic hardware changes that affect
the subsets. The manager can learn what managed objects are in a
community or context by querying the agent. I think the same
agent-centric determination of named data partitions will be required
if the partial locking will apply to multiple NM interfaces that use
different data modeling interfaces. 

In SNMP and/or Netconf, the manager/operatorcan also dynamically
specify subtrees of data or "view families" or XPath expressions. I
think we need to identify whether and how this type of dynamic subset
of data might be locked by partial locking, whether such
dynamically-specified locks apply across multiple NM interfaces, and
how the managed entity knows which non-Netconf data to partially lock.

David Harrington
[email-masked]
[email-masked]

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Aug 2007 16:33:13 -0700

> how the managed entity knows which non-Netconf data to partially lock.

I'm not sure there is any use case to share write access to
the configuration database between SNMP and NETCONF.
Is anybody doing this already?

Is anybody implementing SNMP context or community string based 'partitions'
within the NETCONF protocol?

There is no non-NETCONF data to lock.
There are non-NETCONF locks on NETCONF data, and
global locks of this type are probably good enough.

> [email-masked]

Andy

---

From: Eliot Lear <[email-masked]>
Date: Fri, 31 Aug 2007 11:43:02 +0200

Should NETCONF impose a write-lock on the data then it shouldn't be 
writable to other applications, be that SNMP, or for that matter the 
CLI.  An error should be returned.  In other words, the lock interface 
needs to be below the external protocol interface.  This can prove 
Challenging in some implementations, but it's still the right thing to 
do.  I'm a little leery (perhaps evena  little LEARy) about attempting 
to codify coherency of this form within the IETF, but I could be 
convinced.  Customers, on the other hand, can codify anything they want 
in their RFPs...

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Fri, 31 Aug 2007 12:50:40 +0200

Eliot,

Just to make sure that I understand the requirement here. Do you assume
that the data is writeable only by NETCONF? What if another protocol may
change the same set of data or part of it? 

Dan

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Eliot Lear
> Sent: Friday, August 31, 2007 12:43 PM
> To: Andy Bierman
> Cc: David Harrington; 'Netconf (E-mail)'
> Subject: Re: partial locking
> 
> 
>  Should NETCONF impose a write-lock on the data then it 
> shouldn't be writable to other applications, be that SNMP, or 
> for that matter the CLI.  An error should be returned.  In 
> other words, the lock interface needs to be below the 
> external protocol interface.  This can prove Challenging in 
> some implementations, but it's still the right thing to do.  
> I'm a little leery (perhaps evena  little LEARy) about 
> attempting to codify coherency of this form within the IETF, 
> but I could be convinced.  Customers, on the other hand, can 
> codify anything they want in their RFPs...
> 
> --
> to unsubscribe send a message to [email-masked] 
> with the word 'unsubscribe' in a single line as the message text body.

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Fri, 31 Aug 2007 13:06:35 +0200

> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Friday, August 31, 2007 2:33 AM
> To: David Harrington
> Cc: 'Netconf (E-mail)'
> There are non-NETCONF locks on NETCONF data, and global locks 
> of this type are probably good enough.
> 

I am not sure that I understand the concepts of NETCONF data and
non-NETCONF data. We deal with network devices that are being configured
nowadays using different protocols - I know for example uses of CLI and
SNMP and some forms of secure file transfer and telephony provisioning
protocols in any combination - and now we want to add NETCONF as another
configuration protocol. Unless we assume that what is being configured
by NETCONF is something completely new or that we introduce a
requirement that I was not aware about that what is configured by
NETCONF is not configured by other protocols it seems to me that there
the use case of NETCONF sharing write access with other protocols is
obvious. Am I missing something? 

Dan
(speaking as contributor)

---

From: Eliot Lear <[email-masked]>
Date: Fri, 31 Aug 2007 13:21:32 +0200

Dan,
> change the same set of data or part of it? 

I think it should work in reverse as well, so long as the protocol 
semantics allow for it.  That is to say, you have to have an explicit 
locking capability.

Eliot

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Aug 2007 04:39:20 -0700

> convinced.  Customers, on the other hand, can codify anything they want 

Partial locking in NETCONF is a performance optimization.
I suppose if an agent implementation contained data that was
written frequently by non-NETCONF sources then it might be
worthwhile to implement partial locking for those sources.

IMO, this is an academic exercise.  I am not convinced that
SNMP has any meaningful role to play in writing NETCONF
configuration databases, let alone one that requires
performance optimizations such as partial locking.

Andy

---

From: Eliot Lear <[email-masked]>
Date: Fri, 31 Aug 2007 13:48:03 +0200

> performance optimizations such as partial locking.

This is going to be highly implementation dependent.  And it truly is 
academic as relates to SNMP if READ-WRITE is not enabled for 
configuration objects that NETCONF covers.

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 31 Aug 2007 16:39:26 +0200

Hello,
As Dan has pointed out there are a big number of protocols/interfaces that can modify device 
data. As I understand this device data will have an internal addressing/naming scheme that is 
mapped to the addressing schemes of the different protocols. It is the responsibility of the 
device to do this mapping. This mapping has to be done for partial locking as well, to map the 
area to be locked. However this internal mapping (easy or difficult) is completely device 
dependent, and I fail to see why IETF should try to standardize it.

The same mapping is needed for the base Netconf anyway.

If parts of the device data is not visible via Netconf that might be a good or bad thing, but 
does not really effect Netconf locking. Netconf can only lock what it sees.

Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Aug 2007 08:28:27 -0700

David B Harrington wrote:
>>> for that matter the CLI.  An error should be returned. 
> 
> I interpret the conceptual model to be that there is a writable
> "configuration database" that is shared by all NM interfaces.

It is a completely implementation-specific matter as to how
a NETCONF configuration database is accessed by other protocols.
One thing that is clear is that NETCONF uses XML encoded data,
and the mechanisms to manipulate NETCONF data are XML mechanisms.
So if SNMP or CLI or even COPS-PR data is encoded in XML and
part of the database, then NETCONF managers cannot tell the
difference.  There is only one agent per device and the
entire conceptual configuration database can be represented
as one XML instance document.

So as long as other protocols integrate their data into the
XML database, then an Xpath expression can be used to identify
the XML sub-trees to lock.  Adding SNMP concepts such as context ID
is not appropriate.  There are only namespaces and elements (QNames)
to lock.  (XML attributes should not be locked individually.)

Andy

> 
> When an instance of Netconf applies a global lock. it prevents the
> shared config datastore from being modified by any other NM interface
> (CLI, SNMP, COPS-PR, Web server, ...) or by any other instance/session
> of Netconf. The implementation can simply block all CLI writes, all
> SNMP write, all COPS-PR writes, and all web-server writes that wqould
> modify the shared config datastore.
> 
> Now we want to talk about partial locking, so that two parts of the
> shared config datastore can be worked on simultaneously using more
> than one interface or Netconf instance. I interpret the conceptual
> model to be a lock is taken out on **part** of the shared config
> datastore, but not the complete datastore. The partial lock prevents
> the locked part of the shared config datastore from being modified by
> any other NM interface (CLI, SNMP, COPS-PR, Web server, ...) or any
> other instance of Netconf. The implementation needs to block all
> writes to that part of the config datastore by any NM interface other
> than the one holding the lock.
> 
> My concern is how to specify the part of the shared config datastore
> that is locked, in a way that the implementation cam determine what
> configuration data is locked, in a way that it can efficiently
> determine whether a write by another NM interface would modify the
> locked part. 
> 
> My shared config datastore implementation may in fact include complex
> data structures such as hash tables or tree structures implmented in C
> that happens to be very different from the type of structure used to
> model the information in a Netconf XML-based data model, an SNMP
> SMI-based data mode, a web-page-based data model, a COPS-PR PIB data
> model, or a CLI data model. 
> 
> If the part being locked is a statically-defined
> implementation-defined subset of the shared config datastore, such as
> "the config data in physical blade7", this is probably not an issue
> for an implementation to determine what data is in the part. The
> partition is statically defined by the implementation, and probably
> means much the same set of data regardless of which NM interface is
> used to modify it, or which vendor implemented it. 
> 
> If the part being locked, however, is a dynamically-defined
> client-defined logical part of the config datastore specified in an
> NM-interface-dependent manner, such as with a Netconf XPath
> expression, that might be problematic, because the NM-interface
> modeling might not match my actual implementation. In SNMP, we have
> oft heard complaints that the two-dimensional tables do not map well
> to the actual underlying instrumentation of complex data structures.
> Operators also may want to be able to specify "that part of the config
> datastore that relates to the service I have configued for my
> customer, the ACME company."  
> 
> If there is a mismatch between the NM-interface data modeling concepts
> and actual implementations, then allowing the lock specifications to
> be dynamically defined using NM-interface-specific expressions could
> be difficult for an implementation to apply. If partial locking is
> constrained to some implementation-defined partitions, that might be
> easier for the server to apply, although this could impact
> interoperability across vendors.
> 
> I am not advocating one approach or the other. I think it is important
> that proposals for partial locking clearly identify whether the data
> partitions that can be locked are expected to be determined by the
> implementation or by the client.
> 
> David Harrington
> [email-masked]
> [email-masked]
> 
> 
>> [mailto:[email-masked]] On Behalf Of Romascanu, Dan
> (Dan)
>> Sent: Friday, August 31, 2007 6:51 AM
>> To: Eliot Lear; Andy Bierman
>> Cc: David Harrington; Netconf (E-mail)
>> Subject: RE: partial locking
>>
>> Eliot,
>>

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 31 Aug 2007 18:04:26 +0200

Hello Andy,
I have a problem with the terminology used in this mail-thread.

People are speaking about SNMP database, Netconf database etc. IMHO we should speak about the 
device's management database as the only database. Different protocols, interfaces might 
provide a complete or partial view to this database, but the data is not owned by this or that 
protocol as I understand.

Balazs

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Aug 2007 09:20:04 -0700

Balazs Lengyel wrote:
> People are speaking about SNMP database, Netconf database etc. IMHO we 
> should speak about the device's management database as the only 
> database. Different protocols, interfaces might provide a complete or 
> partial view to this database, but the data is not owned by this or that 

This may be true in the abstract sense, but for standards purposes,
I am interested in how the NETCONF protocol operations work with
the XML encoded data in the <config> element.  This data is entirely up
the agent implementor.  It may or may not represent the entire device.
What if the agent is for a virtual router, not the entire physical router?

I completely agree with you that there is one seamless XML-encoded
configuration database on the agent, which can be manipulated in
a consistent manner with standard protocol operations.  In other words,
if an implementor puts data within the <config>, then it must support the
standard operations such as <get> and <edit-config> (if writable) on that data.
This is a much different requirement than saying all configuration
data on the device must be accessible via NETCONF.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Aug 2007 09:37:36 -0700

David B Harrington wrote:
> Hi,
> 
> I agree. 
> I think there is one virtual shared configuration database that
> contains all configuration regardless of the protocol(s) used to
> access it.  
> 

yes -- this is what we have now with internal agent instrumentation
allowing various protocols to access various internal data structures.
SNMP has an tree populated with OIDs and NETCONF has a conceptual
XML instance document.  How they overlap within internal agent
instrumentation is not relevant to the NETCONF WG.

> And then there are protocol-specific representations (views) of this
> shared configuration database.

yes -- this is what we have now.  Different protocols with
different data model architectures accessing various subsets
of internal data structures on the agent.

This does not mean the NETCONF protocol needs to access the database
with special RPCs like <get-mib> and <set-mib>.
Each protocol has its own consistent view of the data, and the
agent handles any translation for multi-protocol access as
an implementation detail.

> dbh

Andy

>> From: Balazs Lengyel [mailto:[email-masked]] 
>> Sent: Friday, August 31, 2007 12:04 PM
>> To: Andy Bierman
>> Cc: David B Harrington; 'Romascanu, Dan (Dan)'; 'Eliot Lear'; 
>> 'David Harrington'; 'Netconf (E-mail)'

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Fri, 31 Aug 2007 18:57:15 +0200

That was also my point when I said that I do not understand the concept
of 'NETCONF data'. 

Dan

> Sent: Friday, August 31, 2007 7:04 PM
> To: Andy Bierman
> Cc: David B Harrington; Romascanu, Dan (Dan); 'Eliot Lear'; 
> People are speaking about SNMP database, Netconf database 
> etc. IMHO we should speak about the device's management 
> database as the only database. Different protocols, 
> interfaces might provide a complete or partial view to this 
> database, but the data is not owned by this or that protocol 
> as I understand.
> 
> Balazs
>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Aug 2007 10:22:37 -0700

David Harrington wrote:
>> From: Andy Bierman [mailto:[email-masked]] 
> [...]
>> the XML encoded data in the <config> element.  
> 
> But I'm interested in how the 'lock' works, which seems to be
> potentially larger than the view encoded in XML.
> 
>>From RFC4741:
> 
>    o  Running: The complete configuration currently active on the
>       network device.  

I do not interpret these words to mean that there a content-level
conformance requirement that all data comprising the conceptual device
configuration be supported by an agent.  I interpret it to mean
all of the configuration data managed within a single NETCONF agent,
as provided by the agent implementor.

I interpret the global lock to apply to the writable data that a
NETCONF manager can actually access within a single NETCONF agent.
It is an agent implementation detail as to how the configurations
from multiple virtual agents are combined into one instance document
within an agent for the entire physical device.

"NETCONF Data" is the conceptual data that a NETCONF manager can access
with the NETCONF protocol, as represented in various protocol
operations (e.g., <filter> or <config> elements).

Andy

> Only one configuration datastore of this type
>       exists on the device, and it is always present.  NETCONF
> protocol
>       operations refer to this datastore using the <running> element.
> 
> and
> 
>       The lock operation allows the client to lock the configuration
>       system of a device.  Such locks are intended to be short-lived
> and
>       allow a client to make a change without fear of interaction with
>       other NETCONF clients, non-NETCONF clients (e.g., SNMP and
> command
>       line interface (CLI) scripts), and human users.
> 	...
>       The lock operation takes a mandatory parameter, target.  The
>       target parameter names the configuration that will be locked.
>       When a lock is active, using the <edit-config> operation on the
>       locked configuration and using the locked configuration as a
>       target of the <copy-config> operation will be disallowed by any
>       other NETCONF session.  Additionally, the system will ensure
> that
>       these locked configuration resources will not be modified by
> other
>       non-NETCONF management operations such as SNMP and CLI. 
> 
> When you talk about a Netconf global lock, on an implementation that
> has a "running config" that is "the complete configuration currently
> active on the network device",  does the global lock apply only to the
> Netconf XML-encoded subset of the implementation's running config?
> 
> If I have multiple virtual routers within a physical router, then does
> a Netconf lock on "running" only apply to the configuration of one
> virutal router or to "the complete configuration currently active on
> the network device"?
> 
> If I parse your paragraph correctly, you are saying "the XML encoded
> data in the <config> element" + "may or may not represent the entire
> device." Doesn't that contradict the definition of <running> in
> RFC4741?

---

From: "David Harrington" <[email-masked]>
Date: Fri, 31 Aug 2007 13:01:16 -0400

[...]

   o  Running: The complete configuration currently active on the
      network device.  Only one configuration datastore of this type
      exists on the device, and it is always present.  NETCONF
protocol
      operations refer to this datastore using the <running> element.

and

dbh

---

From: "David B Harrington" <[email-masked]>
Date: Fri, 31 Aug 2007 10:15:16 -0400

Hi,

David Harrington
[email-masked]
[email-masked]

(Dan)