Subject: [bmwg] Workplan Proposal for Routing Protcool Testing

From: Nick Ambrose <[email-masked]>
Date: Mon, 18 Jun 2001 16:03:20 -0700

We are proposing that the BMWG accept the following workplan proposal.

The Routing Protcol workplan proposal intends to define terminology and test
methodology for the testing of devices that implement routing protocols.

There are a number of drafts circulating that attempt to address this issue,
but are either too narrowly focused (draft-berkowitz-bgpcon-01.txt) or only
address data 
forwarding issues and ignore control plane testing
(draft-ietf-bmwg-fib-term-01.txt). 

Our proposal attempts to address both issues as well as address testing of
multiple protocols that have a core of similar behavior. By defining metrics

common to a group of protocols, we reduce the amount of effort required to 
define test methodologies for such protocols as a whole and allow
comparisons
between certain protocols when more than one implement similar behavior
(e.g. ISIS and OSPF).

These tests will address testing both the control plane and data plane of
such devices but are designed such that a device that does not implement
data forwarding can still be tested.

The issues we intend to address are:

1.	Define terminology specific to the testing of Routing Protocols in a

    non protocol-specific manner
    
	First draft available as 
	
(http://www.ietf.org/internet-drafts/draft-ambrose-routing-protocol-term-00.
txt)

2.	Define a test methodology framework to allow testing of any protocol
    that can be used to affect the data forwarding behavior of a device
    (the device does not actually have to implement data forwarding in order
	for it to implement such protocols).
	
	These tests will address testing of both the control plane and data
plane of a
	device.

	A partial list of the proposed tests is:
    (this is subject to change as we progress through this area of work)

    - Peer Capacity
    - Routing Protocol Entry Capacity (RIB)
    - Routing Protocol Entry Capacity (FIB)
    - Peer and Routing Protocol Entry Capacity (RIB)
    - Peer and Routing Protocol Entry Capacity (FIB)
    - Performance
    - Peers accepted Rate
    - Routing Protocol Entry's accepted Rate (RIB)
    - Routing Protocol Entry's accepted Rate (FIB)
    - Forwarding Rate to stable Routes with Route flapping
    - Forwarding Rate while establishing new Peers
    - Initial Peer convergence (RIB)
    - Initial Peer convergence (FIB)
    - Convergence for new Routing Protocol Entry (RIB)
    - Convergence for new Routing Protocol Entry (FIB)
    - Convergence to a backup Routing Protocol Entry (FIB)

3.	Define subsequent RFC documents that describe how to application of
the framework
    to specific protocols

The protocols we intend this work to apply to include but are not
necessarily limited to:
- OSPF
- ISIS
- RIP
- BGP (Internal and External)

We intend to publish the first methodology document before the deadline for 
the 51st IETF meeting in London and present this, along with the existing 
terminology draft at that meeting.

We believe that this work will subsume and extend the existing drafts and
will reduce the amount of work required to define test methodology for such
protocols. 

Nick Ambrose                          
Principal Software Engineer, Ixia.
<mailto:[email-masked]>, http://www.ixiacom.com
(818) 444-2303

Ixia...    
     When the test really counts.

Nick Ambrose                          
Principal Software Engineer, Ixia.
<mailto:[email-masked]>, http://www.ixiacom.com
(818) 444-2303

Ixia...    
     When the test really counts.

_______________________________________________
bmwg mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/bmwg

---

From: "David Newman" <[email-masked]>
Date: Mon, 18 Jun 2001 16:26:24 -0700

This appears to be an effort to apply one generic set of definitions to ALL
routing protocols -- or as the authors say, to work in a "non protocol
specific manner."

As anyone who's had the pleasure of working with the various protocols can
attest, the one-size-fits-all approach is highly problematic. There are too
many protocols, with too many specific *and orthogonal* requirements, for a
generalized approach to be useful.

It's also untrue, as suggested here, that even a subset of routing protocols
share a core of similar behavior. Even among routing protocols with similar
functions (e.g., IGPs), there are significant differences in terms of table
update mechanisms,  table update contents, metric calculation, and on and
on.

May I respectfully suggest that the authors instead focus on working from
the specific toward the general? We don't today have tests for specific
protocols, and we need them.

dn

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of Nick
> Ambrose
> Sent: Monday, June 18, 2001 4:03 PM
> To: [email-masked]'
> Subject: [bmwg] Workplan Proposal for Routing Protcool Testing
>
>