Subject: RE: [bmwg] MPLS forwarding Benchmarking:draft-akhter-bmwg-mpls-meth-03.txt

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Tue, 4 Dec 2007 14:53:31 -0500

Hi Silvija,

Thanks a lot for the support and careful review. 

1) Good point about multicast exclusion and IPv6. We tried to align the
MPLS benchmarking with IP (v4/v6) benchmarking documents, and none of
them included any explicit verbiage for multicast. 

Nonetheless, this is a worthy point, and we will update the scope/draft
accordingly.

2) I agree that more text can be added to clarify test setup (in section
6) wrt section 4.1.5.1. Perhaps, section 4.1.5.1 may be updated as -

"In all cases the sent traffic MUST be accounted for, whether it was
received on the wrong port, correct port or not received at all.
Specifically, Traffic loss (also referred to as frame loss) is defined
as the traffic (or frame) not received where or as expected (i.e.
received on incorrect port, or received with incorrect layer2 or above
header information etc.). In addition, the MPLS header...."

3) We will update the Section 6.5 accordingly. This was suggested by
another reviewer as well. The plan is to update the section with the
following examples -

	device/hardware reset - RouteProcessor card resetting (e.g. OIR)
	software reset - reset initiated through a CLI  

Please note that "power interruption" is to be carried out as an
additional reset per RFC2544 section 26.6. 

Thank you so much again.

Cheers,
Rajiv 

> -----Original Message-----
> From: Silvija Andrijic Dry (sdry) 
> Sent: Friday, November 30, 2007 4:14 PM
> To: Aamer Akhter (aakhter); [email-masked]
> Subject: RE: [bmwg] Updated draft on MPLS 
> forwardingBenchmarking:draft-akhter-bmwg-mpls-meth-03.txt
> 
> Hi Aamer, Rajiv,
> 
> I have read latest 03 rev as well some of your earlier ones and like
> some of changes you have made especially adding system recovery and
> reset test cases. Overall I think this is useful work and I am in
> support of it.
> 
> Here are couple of specific comments:
> - Section 2 (Scope): need to explicitly state that scope includes
> unicast only and multicast is not within the scope (or if it is
> additional considerations and test cases would definitely be needed).
> Also I noticed IPv6 was mentioned in some other places in 
> document (like
> section 4.1.2 and 5) so would be good to be clear up front in scope
> section what is covered by the draft 
> - In section 6 (test procedure) you use as criteria to move 
> to new test
> iteration "if any frame loss is detected", while in section 
> 4.1.5.1 you
> say that "all traffic MUST be acocunted for, whether it was 
> received on
> wrong port, correct port or not received at all". I would 
> make criterias
> in these two instances consistant, preferably either frame loss or
> receiption at the wrong port would require one to proceed to new test
> iteration. Perhaps you can say in section 6 something like: 
> "If traffic
> verification procedure specified in 4.1.5.1 fails proceed to new
> iteration" 
> - In section 6.5 it would be good to suggest couple of more common
> failure events besides device and software resets. For example
> switchover between any redundant components in the DUT if any 
> supported
> (e.g. main route processor, switching fabric, etc.). Look at section 7
> of MVPN Scale benchamrking draft where we make some additional common
> failure events recommendations and some might be applicable in your
> draft as well: 
> http://www.ietf.org/internet-drafts/draft-sdry-bmwg-mvpnscale-03.txt 
> - Also in section 6.5 it would be good to be more specific on what
> device and software resets mean as you can typically reset device in
> couple of different ways where not all result in same code paths and
> outage times...I think the safest method to ensure 
> consistancy would be power cycle for device reset.
> 
> Cheers,
> Silvija
> 
>  
> 
> -----Original Message-----
> From: Aamer Akhter (aakhter) 
> Sent: Monday, November 26, 2007 12:06 PM
> To: [email-masked]
> Subject: [bmwg] Updated draft on MPLS forwarding
> Benchmarking:draft-akhter-bmwg-mpls-meth-03.txt
> 
> Hello BMWG,
> 
> We would like to bring to your attention the updated draft on 
> basic MPLS
> forwarding that we will be submitting to the IETF as an individual
> submission (once submissions opens up again). There have been
> significant changes since the last version of the draft including:
> 
> * removal of many test cases thought to be out of scope
> * rewriting of many cases to be less verbose
> * addition of reload tests
> 
> http://www.employees.org/~aamer/draft-akhter-bmwg-mpls-meth-03.txt
> 
>    Over the past several years MPLS networks have gained greater 
>    popularity. However, there is no standard method to compare and 
>    contrast the varying implementations and their strong and weak 
>    points. This document proposes a methodology using common criteria 
>    for the comparison of various implementations of basic MPLS 
>    forwarding devices.
> 
> If there are any concerns or changes that need to be made 
> please do not
> hesitate to contact us.
> 
> Regards,
> 
> Aamer Akhter
> Rajiv Asati
> 
> ==================
> 
> 	Title		: MPLS Benchmarking Methodology
> 	Author(s)	: Aamer Akhter and Rajiv Asati
> 	Filename	: draft-akhter-bmwg-mpls-meth-03.txt
> 	Pages		: 21
> 	Date		: 2007-11-25
> 	
>    The purpose of this draft is to describe a methodology specific to 
>    the benchmarking of MPLS forwarding devices. The scope of this 
>    benchmarking will be limited to various types of packet-forwarding 
>    and delay measurements. It builds upon the tenets set forth in 
>    RFC2544 [RFC2544], RFC1242 [RFC1242] and other IETF Benchmarking 
>  
>    The BMWG produces two major classes of documents: Benchmarking 
>    Terminology documents and Benchmarking Methodology documents.  The 
>    Terminology documents present the benchmarks and other related 
>    terms. The Methodology documents define the procedures required to 
>    collect the benchmarks cited in the corresponding Terminology 
>    documents.
> 
> A temporary URL for this Internet-Draft is:
> http://www.employees.org/~aamer/draft-akhter-bmwg-mpls-meth-03.txt
> 
> --
> Aamer Akhter / [email-masked]
> Rajiv Asati / [email-masked]
> 
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/bmwg
> 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg