Subject: [bmwg] Review comments of draft-rosa-bmwg-vnfbench

From: "Luis M. Contreras" <[email-masked]>
Date: Wed, 31 Jul 2019 17:33:01 +0200

Hi all,

as committed last week during BMWG session, I have performed a review
of draft-rosa-bmwg-vnfbench.

These are the comments coming from my review.

/General comments/

.- it would be good to have some text in the draft indicating how this
benchmarking methodology relates with the activities in ETSI NFV TST
working group

.- an special case for benchmarking could be the redundancy. There are
different schemas of redundancy (e.g., VNFs with active / standby VNFCs,
M:1 redundant VNFCs, non-redundant VNFCs, etc). Would the redundancy be
part of the scope of the draft/methodology that you describe? And if so,
how this could be included as part of the descriptors / setup that you
describe?

.- Can we in general terms assume that Agents represent active probing
while Monitors represent passive probing? If so, it would be probably good
to make it explicit

/Specific comments/

.- Section 5, bullet on VNF (after Fig. 1): with relation to the VNFCs, can
be those components tested individually or should them be always tested as
part of the comprehensive VNF?

.- Figure 1: the Monitor box, as depicted, it is not too much clear. The
boundaries of the box overlap with the boundaries of VNF component and
Execution environment. This could be done on purpose, but the figure
becomes a bit confusing, at least to me. Additionally, I can see arrows
to/from the agents, but no arrows to/from the monitor. From/to where the
information is send to Monitor box?

.- Section 6.1: should it be included information about the kind of VIM,
MANO, etc to use for on-boarding, managing and running the VNF? Should the
NMS of the VNF part of the tests (maybe assisting on the
configuration/collection of information)? Should it be also declared the
usage of VM, containers, etc in the setup? Where?

.- Section 6.3.1: should it be included there the duration of the tests?

.- Section 6.2.2: the VNF processing / Active metrics, are those equivalent
to the kind of metrics can be obtained from a PNF? Any difference? If not,
it could be maybe convenient to reflect that fact since same metric
description could be reused by the operators for performing the tests (and
for comparison, as well)

.- Section 6.3.2: if the Manager collects all measurements, then it has to
support some kind of interface for information retrieval, hasn’t it? If so,
it could be maybe convenient to reflect it in figure 1 and in the text.

.- Section 6.4.3: failure handling can be considered as active or passive
testing?

.- Section 8: during the VNF benchmarking, it could be considered the
running of security tests? For instance DDoS, etc. If so, it could be
mentioned.

/Editorial comments/

.- The Agent/Prober and Monitor/Listener bullets should be better aligned,
maybe using different levels of bullets and skipping existing space lines

.- section 6.3.2, bullet 1: s/ … compose the all the permutations … / …
compose all the permutations …

.- section 6.4.1, 1st paragraph:  s/ … to mitigateside effects … / … to
mitigate side effects …

I would like to tahnks the authors for the very good document produced so
far.

Best regards

Luis

-- 
___________________________________________
Luis M. Contreras
[email-masked]
[email-masked]
Global CTIO unit / Telefonica