Subject: [bmwg] Ignas Bagdonas' No Objection on draft-ietf-bmwg-sdn-controller-benchmark-meth-08: (with COMMENT)

From: Ignas Bagdonas <[email-masked]>
Date: Thu, 19 Apr 2018 06:50:49 -0700

Ignas Bagdonas has entered the following ballot position for
draft-ietf-bmwg-sdn-controller-benchmark-meth-08: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn-controller-benchmark-meth/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

The document seems to assume the OpenFlow dataplane abstraction model – which
is one of the possible models; the practical applicability of such model to
anything beyond experimental deployments is a completely separate question
outside of the scope of this document. The methodology tends to apply to a
broader set of central control based systems, and not only to the data plane
operations – therefore the document seems to be setting at least something
practically usable for benchmarking of such central control systems. Possibly
the document could mention such assumptions made about the overall model where
the methodology defined applies to.

A nit: s/Khasanov Boris/Boris Khasanov, unless Boris himself would insist
otherwise.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 19 Apr 2018 14:30:30 +0000

Hi Ignas and Suresh (who supported this comment),

> -----Original Message-----
> From: Ignas Bagdonas [mailto:[email-masked]]
> Sent: Thursday, April 19, 2018 9:51 AM
...
[acm] 
That's a good wording suggestion, it certainly captures the working group
consensus to prepare the methods independent from OpenFlow, for 
wider applicability.

However, we also received feedback from Stuart Bryant seeking *more* 
message specificity (appended below, see * ), which is not really possible 
for the general methods. Please try to strike a balance between 
these comments in discussion today, if possible!

Al
doc shepherd

-=-=-=-=- Stuart's comment-=-=-=-=-=-
2.3.1.3. Asynchronous Message Processing Rate

Definition:
   The number responses to asynchronous messages (such as new flow
SB> That should be the number of responses per second.

Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Device at the rate that the
   controller processes (without dropping them).

SB> So what you are testing here is the control network and the
SB> controller. This is perhaps the only practical way to run the
SB> test, but it seems a pity that you do not deconvolve these
SB> two aspects of the test.
SB>
SB> I suppose this is really network Async Msg Proc rate rather than
SB> controller Async proc rate.
SB>
*SB> We may get to this in the companion document, but doesn't there
*SB> need to be some standardization of the event message to compare
*SB> apple with apples over time?