Subject: [bmwg] Proposal on Protection Benchmarking

From: Al Morton <[email-masked]>
Date: Wed, 03 May 2006 15:30:29 -0400

BMWG,

The proponents of the Protection Benchmarking Work Proposal
have prepared the description of this work effort, below.

BMWG discussed this work at the Dallas IETF-65 session,
where there was strong support and involved membership
(see meeting minutes).

Please weigh-in on whether this topic should become part
of BMWG's chartered work, by

                    June 2, 2006

And, if you support the work, please say:

+  HOW you intend to support the development in BMWG,
    (by reviewing draft X by MM/DD/YY, for example),  or

+  WHY this work will be beneficial to BMWG's user community, or

+  Modifications that would make the proposal more useful
    (which we will discuss on the list), and

+  (anything else that's constructive)

And remember, we'd like to hear your opinion on the list,
even if you spoke in favor of this proposal at the meeting.

Thanks in advance for your efforts and commitment to BMWG!

Al
bmwg co-chair

=========================================================================

Proposal for Protection Benchmarking

Summary:

This is a proposal for a BMWG work item for black-box benchmarking of 
sub-IP protection mechanisms at the IP Layer. These mechanisms function at 
sub-IP layers with the purpose of protecting IP traffic, making it possible 
to develop benchmarks for these mechanisms by measuring the data plane at 
the IP layer.  This work item will develop a common terminology to be used 
for protection benchmarking at the data plane of the IP Layer and multiple 
methodology documents for each sub-IP protection mechanism.  Such 
mechanisms include but are not limited to MPLS Fast Reroute, SONET APS, and 
High Availability.  The methodologies and benchmarks can be used to compare 
performance of a protection mechanism on one device to another or to 
compare performance of two or more protection mechanisms.

Goal:

The objective is to produce a terminology document for protection 
benchmarking at the IP Layer and methodology documents for benchmarking 
protection mechanisms at the IP layer.  Path failures and rerouting are 
benchmarked through packet loss measurement on the data plane.  There will 
be a common terminology to be used for protection benchmarking at the IP 
Layer and multiple methodology documents for each sub-IP protection 
mechanism.

Scope:

The focus of this effort is to benchmark Sub-IP protection mechanisms at 
the IP Layer for a DUT or SUT.   Such mechanisms include but are not 
limited to MPLS Fast Reroute, SONET APS, and High Availability.  The 
benchmark parameters that are measured are independent of the layer 3 
routing protocol being run.  The data plane is measured to obtain the IP 
layer benchmarks. Control Plane scaling will be considered in the test 
methodologies to measure its impact to the IP Layer benchmarks. In 
addition, this effort will characterize the duplicate and reordered packets 
within the definition of recovery time. The presence of duplicate and 
errored packets may be tied to the failover event.

Motivation:

Service Providers are currently deploying sub-IP protection mechanisms to 
minimize or eliminate IP layer packet loss and control plane failure.  The 
networking device failover performance times have become critical metrics 
for deployment selection.  Faster networking device failover times have 
directly led to faster network failure recovery times.

  Network users consider packet loss a key service metric. Intervals of 
high packet loss are externally observable events having direct impact on 
user application performance. Numerous sub-IP protection mechanisms have 
been developed and are now being deployed to minimize IP packet loss upon a 
failure.  For this reason it is very important to develop uniform 
protection mechanism benchmarks for comparing networking device 
performance.  Service providers can use these benchmarks to compare 
performance of a protection mechanism on one device to another or to 
compare performance of two or more protection mechanisms.

Schedule:

"Terminology For Protection Benchmarking,"

draft-poretsky-protection-term-01.txt, Ready for WGLC 07/06 and Ready for 
IESG 10/06.

"Methodology For MPLS Protection Benchmarking,"

draft-poretsky-mpls-protection-meth-05.txt

MERGE WITH

"Methodology for benchmarking fast fail-overtime with local protection,"

draft-vapiwala-bmwg-frr-failover-meth-00.txt

TO CREATE

"Methodology for MPLS Protection Benchmarking,"

draft-ietf-bmwg-mpls-protection-meth-00.txt

FIRST SUBMITTAL 06/06 and Ready for WGLC 12/06 and Ready for IESG 04/07.

These documents (Terminology and Methodology) have already undergone 
extensive working group review.

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Al Morton <[email-masked]>
Date: Fri, 26 May 2006 16:45:34 -0400

REMINDER:

The comment period on this proposal closes next week
(and messages on this thread are few).

At 03:30 PM 5/3/2006, Al Morton wrote:
>sub-IP layers with the purpose of protecting IP traffic, making it 
>possible to develop benchmarks for these mechanisms by measuring the data 
>plane at the IP layer.  This work item will develop a common terminology 
>to be used for protection benchmarking at the data plane of the IP Layer 
>and multiple methodology documents for each sub-IP protection 
>mechanism.  Such mechanisms include but are not limited to MPLS Fast 
>Reroute, SONET APS, and High Availability.  The methodologies and 
>benchmarks can be used to compare performance of a protection mechanism on 
>one device to another or to compare performance of two or more protection 
>mechanisms.
>
>
>Goal:
>
>Layer and multiple methodology documents for each sub-IP protection mechanism.
>
>
>Scope:
>
>addition, this effort will characterize the duplicate and reordered 
>packets within the definition of recovery time. The presence of duplicate 
>and errored packets may be tied to the failover event.
>
>
>Motivation:
>
> been developed and are now being deployed to minimize IP packet loss upon 
> a failure.  For this reason it is very important to develop uniform

---

From: vgurudevappa <[email-masked]>
Date: Wed, 31 May 2006 12:43:31 -0400

Hi* *Al,
 
As I previously stated I am in favor of this work item and shall 
contribute in the form of technical reviews.  I hope the authors have 
addressed my comments in the merged draft.
 
Cheers
-Vrushi
 
-----------------------------

Al Morton wrote:

>> sub-IP protection mechanisms at the IP Layer. These mechanisms 
>> function at sub-IP layers with the purpose of protecting IP traffic, 
>> making it possible to develop benchmarks for these mechanisms by 
>> measuring the data plane at the IP layer.  This work item will 
>> develop a common terminology to be used for protection benchmarking 
>> at the data plane of the IP Layer and multiple methodology documents 
>> for each sub-IP protection mechanism.  Such mechanisms include but 
>> are not limited to MPLS Fast Reroute, SONET APS, and High 
>> Availability.  The methodologies and benchmarks can be used to 
>> compare performance of a protection mechanism on one device to 
>> another or to compare performance of two or more protection mechanisms.
>>
>>
>> Goal:
>>
>> benchmarking at the IP Layer and methodology documents for 
>> benchmarking protection mechanisms at the IP layer.  Path failures 
>> and rerouting are benchmarked through packet loss measurement on the 
>> data plane.  There will be a common terminology to be used for 
>> protection benchmarking at the IP Layer and multiple methodology 
>> documents for each sub-IP protection mechanism.
>>
>>
>> Scope:
>>
>> The focus of this effort is to benchmark Sub-IP protection mechanisms 
>> at the IP Layer for a DUT or SUT.   Such mechanisms include but are 
>> not limited to MPLS Fast Reroute, SONET APS, and High Availability.  
>> The benchmark parameters that are measured are independent of the 
>> layer 3 routing protocol being run.  The data plane is measured to 
>> obtain the IP layer benchmarks. Control Plane scaling will be 
>> considered in the test methodologies to measure its impact to the IP 
>> Layer benchmarks. In addition, this effort will characterize the 
>> duplicate and reordered packets within the definition of recovery 
>> time. The presence of duplicate and errored packets may be tied to 
>> the failover event.
>>
>>
>> Motivation:
>>
>> Service Providers are currently deploying sub-IP protection 
>> mechanisms to minimize or eliminate IP layer packet loss and control 
>> plane failure.  The networking device failover performance times have 
>> become critical metrics for deployment selection.  Faster networking 
>> device failover times have directly led to faster network failure 
>> recovery times.
>>
>>  Network users consider packet loss a key service metric. Intervals 
>> of high packet loss are externally observable events having direct 
>> impact on user application performance. Numerous sub-IP protection 
>> mechanisms have been developed and are now being deployed to minimize 
>> IP packet loss upon a failure.  For this reason it is very important 
>> to develop uniform protection mechanism benchmarks for comparing 
>> networking device performance.  Service providers can use these 
>> benchmarks to compare performance of a protection mechanism on one 
>> device to another or to compare performance of two or more protection 
>> mechanisms.
>>
>>
>> Schedule:
>>
>> draft-poretsky-protection-term-01.txt, Ready for WGLC 07/06 and Ready 
>> for IESG 10/06.
>>
>>