Subject: [bmwg] Feedback on draft-novak-bmwg-ipflow-meth-01.txt

From: Benoit Claise <[email-masked]>
Date: Mon, 17 Nov 2008 20:54:44 +0100

Jan,

I've been reviewing your draft. It really improved compared to the 
previous version.

Here is some feedback.

- First of all, there are some idnits to correct.
  http://tools.ietf.org/html/draft-novak-bmwg-ipflow-meth-01 -> click on 
nits

- I would use the IPFIX term in the abstract
OLD:

Abstract

   This document provides methodology and framework for quantifying
   performance implications of enabling selective monitoring of
   IP flows on a network device and export of this information to
   a collector as specified in [RFC5101 <http://tools.ietf.org/html/rfc5101>].

NEW
Abstract

   This document provides methodology and framework for quantifying
   performance implications of enabling selective monitoring of
   IP flows on a network device and export of this information to
   a collector wit the IPFIX Protocol [RFC5101 <http://tools.ietf.org/html/rfc5101>].

- 
   "The question any user
   considering its deployment asks is - And what performance
   implication Flow Monitoring will have on my network ?"
flow monitoring is not a defined term -> lower case
There are many instances of "Flow Monitoring"

- "       a. what will be CPU usage"
Isn't it: what will be the CPU impact
OR
	  what will be the flow monitoring CPU usage

- "      b. what will be the forwarding performance"
Isn't it: what will be the impact on the forwarding performance?

- Following a ".", you should have two spaces

- Formatting issue:
   The major network operator concerns always are:
       a. what will be CPU usage
       b. what will be the forwarding performance
       c. what will be bandwidth usage of the Flow Monitoring export

   when enabling Flow Monitoring on network devices. 

- You might want to clarify what you mean by application

       IETF IPFIX working group concentrates its effort on standardising
       the Flow Monitoring data export to an external collecting
       device while the actual application providing the data inside of
       a network device is left to the implementors liberty. 

- "The goal of
   this document is to address both Flow Monitoring and Flow Monitoring
   data export performance,"

I think that you should be reusing the Metering Process and Exporting Process definitions used in IPFIX.

- Why is following important?
   This document covers IP (both IPv4 and IPv6) flows or MPLS
   encapsulated IP flows, any other protocols or traffic types are out
   of the scope.

- "   Control Information       [RFC5101, section 2 <http://tools.ietf.org/html/draft-novak-bmwg-ipflow-meth-01#section-2>]"
Wrong, it's not defined in RFC5101

- "Number of Flow Records which expire"
Number of Flow Records that expire. Note that I learned that rule from the RFC-editor with the first RFC I published ;-)

    The cat, which crossed the street, was chased by the dog. - If you
    remove which part that means we are not talking about a specific
    cat. It can be any cat pussy,suzy etc

    The cat that crossed the street was chased by the dog. -- we can't
    remove logically that because we are talking about a particular cat
    who was chased by dog. So this info is necssary.. 

There are several instances where I think that "which" should be changed 
by "that"

- 2.2.4 Active Timeout

   Definition:
      The time interval from the time when first packet of a particular
      Flow was seen till the Flow will be expired while there are still
      packets arriving to the DUT which belong to the Flow.

	-> add "Flow" in the terminology section

- Flow Monitoring Processor Utilisation (FMPU)
   "Discussion:
      The CPU utilisation SHOULD BE collected from the DUT after
      sufficient time interval under the test traffic stream to obtain
      reliable average CPU usage. The recommended time before collecting
      for example the 1 minute average CPU usage is 5 to 10 minutes."
Doesn't depend on the expiration timeout?

- In figure 1 and 2, you might want to show that the device exports some flow records

- It's a hard limitations to write a statement such as:
   "DUT needs to have very basic configuration just allowing IP packet
   forwarding without any use of dynamic IP routing protocols."?
You're afraid of the CPU impact? What if the link-state protocol is stable and its CPU is taken into account in the BCPU?
You're sentence sounds like: if I run a router with OSPF, I can't use this testing methodology...

- It took me some time to understand what you meant by
      a. centralised DUT architecture - the allocation of input and output
         ports does not play any role
      b. distributed DUT architecture - the input and output ports
         allocation can be done in several different ways depending on
         the measurement objective or the expected network device usage.
         The ports can be allocated on the same modules or different
         modules to which the DUT central processor unit distributes
         forwarding control.

 Am I right to say that you speak about a distributed router architecture with line card?
 You might want to give a clear example.
 You might also explain that different LCs have different Observation Domain [RFC5101] 
 What is your advice wrt to distributed architecture: 
	- test all possibilties?
	- test a single one? Then which one?

- "Frame format to use for IPv6 traffic is specified in [RFC2464 <http://tools.ietf.org/html/rfc2464>]."
Why do you assume an ethernet interface? Is this a requirement?
I don't think we want to limit the testing only interface types.

- "Another implementation might allow the user to actually define his own
   completely arbitrary Flow Record to monitor the traffic. The
   requirement for the tests defined in this document is only the need
   for a large number of Flow Records in the Cache."

Shouldn't we note the template record when giving the test results?
Similar to the interface type, the sampling rate, the QoS features, the distributed (or not) architecture), the template Record is an important data when returning the test results.
Maybe we require a sentence such as: the following parameters (interface type, sampling rate, QoS, etc...) SHOULD be noted during the test and associated with the test results"

- "There is no commonly accepted limit to the CPU usage but typically
   usage in the range of 70 - 80 % becomes already a critical issue."
It depends which CPU:
	5 min decaying -> certainly
	instantenous (or 1 sec) -> maybe not 

- In section 4.2, last paragraph? 
On a distributed system, should we measure the RP CPU (next to the LC CPU), if the flow records export takes some CPU on the RP?

- "   The test run will produce n triplets of values as follows:
      "Packet Rate" "BPU" "FMPU"

   The CPU usage needed to update and maintain the states in the Cache is
   represented by the values of difference (FMPU - BPU) and is a function
   of the number of updates per second - in this particular set-up it is
   equal to the packet rates."

You assume a number of flow records equals to 90% of the cache size.
So the cache size SHOULD be written down. 
Or it doesn't matter? 
I mean: the cache management update sone flow every 1/10 sec, or 10 flows every 1 sec. 
Will it produce the same result? A first glance we can be sure.

-   Sender Traffic Definition:

      "Define n traffic streams while incrementing the packet rate."

That sentence can be understood in different ways.
While speaking with Jan, I now understand the meaning:
  test 1 should run with a packet rate value of X for the sum of all test flows
  test 2 should run with a packet rate value of Y for the sum of all test flows  
  test 3 should run with a packet rate value of Z for the sum of all test flows
This should be clarified.

- You excluded the bandwidth from the draft.
However, it's not difficult to include this.
You have already the flow expiration rate and flow export rate. From the Template Record definition, you know the Flow Record length.
If you write them down, you can deduce the bandwidth very easily.
At least one approximation, the only thing that you might be missing that way is:
	the Template Record export
	the Option Template Record and associated Flow Record export
However, those two items should be small compare to the Template Record Flow Records.

Regards, Benoit. 

_______________________________________________
bmwg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bmwg