Subject: [bmwg] draft-constantine-bmwg-traffic-management-00

From: Barry Constantine <[email-masked]>
Date: Wed, 6 Mar 2013 19:01:17 +0000

Hello,

Al and I discussed some of the various means that comments have been arriving; some via list, some via email, and others via Word document.

Even though the work is personal submission at this point, Al indicated that it is preferred to get comments documented on the list.

So from Shane Amante, Level 3:

****

1)  One thing that I think would be invaluable in this document is to state, in all sub-sections, to *not only* observe transmitted and dropped packets/bytes at the "Transmitting" and "Receiving" Test Host, *but also* and /most importantly/ make sure to observe that such counters exist on the DUT itself and in the appropriate MIB's on the device itself.  I would state two reasons for that:

    a) When the DUT is actually deployed to the field it is critical for operators to observe these counters to understand what the heck is happening to a customer's service; and,

    b) For SP's, like Level 3, where we do usage-based billing on individual queues ... we need to ensure the counters are accurate, otherwise we don't get paid.  :-) ... and, ultimately, we've all seen instances where some vendors do not implement any counters whatsoever (I can think of one vendor whose name starts with a letter between A & C being a prime example), which makes debugging problems in the field a royal PITA.  Anyway, putting this in the document and emphasizing it over and over will give operators yet another sledgehammer to wield against network equipment vendors to implement counters and make sure they're accurate.  :-)

2)  In Section 1.1, Traffic Mgmt Overview, you talk about Traffic Classification, Policing, Shaping, Scheduling and Congestion Mgmt.  While Section 6 does cover policing, shaping & congestion mgmt ... I didn't observe any discussion about traffic classification -or- scheduling in the doc.  If that was your intent, then it would be useful to point out in Section 1.1 that those are beyond the scope of the doc, for future study or whatever.

3)  In Section 3, I'm unclear why you mention "Firewalls".  I think it might be better to change that from "firewalls" to "middleboxes", because (unfortunately) not only firewalls can do policing, etc., but there also exist devices do "WAN compression" that also perform similar functions.  Thus, "middleboxes" captures a wider set of devices ...

4)  Throughout the document, you talk about various measurements, burst sizes, etc., but I don't recall any conversation about whether and what type(s) of "overhead" may be included (or not) in each of these calculations.  Now, I realize that every equipment vendor is different, but as you know some do include L2 framing overhead, some don't, some include L3 header overhead, etc. in these calculations.  I think at least a "warning" of some type to the reader that they need to understand this is critical to ensuring accuracy of the tests.  One place, of several, this is most apparent are, for example, Section 4.2 where you talk about "TCP Efficiency" and Transmitted & Retransmitted Bytes.  What are you including in transmitted vs. retransmitted bytes, e.g.: TCP data segments or the full IP + TCP + data PDU?

5)  In Section 6.1, Policing Tests, there is the following statement, which I believe is False.  (Policing can operate at Layer-2 -or- Layer-3):

---snip---

   Policing tests will only use stateless traffic since a policer only

   operates at Layer 2.  Stateful TCP test traffic would not yield any

   benefit to test a policer.

---snip---

6)  In Section 6, you mention values for burst sizes, etc.  As an example, in Section 6.1, there's the CBS value of 64KB.  The question I have is: where did you come up with 64KB?  In Section 6.2.2, Testing Queue with Stateful Traffic, there is a recommended (?) value of 32KB for Queue Length (QL).  Again, where did this value come from?

7)  In Section 6.2, Queue Tests, I have a few comments. I'm not sure I understand the proposed methodology for Queue Testing here. When I read this section, it sounds like you're proposing the testing of an individual queue on a physical port. However, I'm not sure that is useful, because in nearly all cases if you're only sending one class of traffic, then the /scheduling/ treatment out of the port will generally be FIFO, (the one exception is Cisco 7200 CBWFQ where they actually, by default, treat the SPQ with a policer to avoid starving inferior queues).  Either I'm gravely misunderstanding what you're proposing -- which is very likely -- or, you may need to think about rewriting this section to discuss testing of /scheduling/ out the interface, i.e.: the competition between various types of queues and the various queue scheduling disciplines: CBWFQ, DWRR and HSPQ being (I think) three of the most prevalent and widely implemented across Cisco, Juniper & ALU, respectively.  (FWIW, I think you or someone else made a similar comment to the above, but its at the bottom of Section 6.4.2).

8)  One thing that I think is missing from the document is testing of policing/shaping/scheduling functions when going between vastly different media where buffer-size (queue-length) really matters.  Specifically, think about 10 GbE uplinks on a PE feeding into a N x DS1/DS0 interface toward a customer.  Or, even 10 GbE -> 1 GbE transitions on L2 ToR switches that barely have any buffer and what little they do have has a very small fixed amount allocated to physical ports where the rest of it is "dynamically allocated".  In this case, you need to think about designing a test where you don't have a single 10 GbE sending traffic into a L2 switch and out a single 1 GbE interface as you may not be consuming all of the buffer on the switch.  Rather, you need to understand the ASIC architecture a little bit and design the test such that you're fanning out the traffic, like you would see in the real-world, to help consume more of the shared buffer and understand if they will work (or not) for whatever applications/environment you have.

Anyway, I think the above is probably enough for you to chew on, for now.

Overall, I do think this will be a valuable document to the IETF and the community at large, so I'd encourage you to keep working on it.  And, as you have future revisions I'd be happy to review them and provide comments.

***

All excellent comments and we are already discussing their incorporation.

Thank you,
Barry Constantine