Subject: Re: [bmwg] New Version for draft Methodology for VNF Benchmarking Automation

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 2 Nov 2020 03:41:05 +0100

Hi Raphael,

On 20/10/2020 21.16, Raphael Vicente Rosa wrote:
> ...
> 7. The reference implementation realizes all the methodology (getting 
> some results to present).

I am in the process of reviewing the draft. Can you post a link to some 
results (XML instance data) of a benchmark report generated according to 
the YANG data model?

That said here are some notes I made so far:

* I am a bit uncertain about what "VNF-BD YANG path of the variable" is? 
Have you considered using "type instance-identifier;"?

        leaf path {
          type string;
          description "The VNF-BD YANG path of the
          variable.";
        }

* IMO The short description in "5.3.  VNF Benchmarking Report (VNF-BR)" 
can be improved with some examples. Appendix with complete example 
report data instance file.

* IMO rfc8345 should be used for the topology definitions that are 
significant part of the /vnf-bd:scenario - nodes, connection-points, 
links etc. [email-masked] can be augmented with 
additional context specific data.

* ETS14a,b,c,e reference links have typos and do not work. ETS14d, f  work.

* use "ietf-yang-types:date-and-time" instead of "string" for timestamps.

* use "ietf-yang-types:uuid" types instead of "string"

* Add YANG tree diagrams:

    $ pyang -f tree  --path 
/usr/share/yuma/modules/ietf:/usr/share/yuma/modules/ietf-drafts:. 
vnf-bd.yang

    module: vnf-bd
      +--rw experiments
      |  +--rw trials?   uint32
      |  +--rw tests?    uint32
     +--rw scenario
      |  +--rw nodes* [id]

...

* Drafts with YANG modules should use <CODE BEGINS> file ... <CODE ENDS> 
so that the modules are detected and validated by the IETF tools.

/Vladimir

---

From: Raphael Vicente Rosa <[email-masked]>
Date: Tue, 17 Nov 2020 11:24:47 -0300

Hi Vladimir,

Thanks for the comments, I'm reading your draft too.

I really appreciate the comments about the YANG modules, indeed I'm not
experienced with them. With your comments I'm going to extensively review
them and update the draft.

Regarding the examples of parsed YANG modules, you can check the link
https://github.com/raphaelvrosa/gym/tree/master/examples/results
The experiments realized by gym with vnf-br[1,2,3] contain demonstrations
of simple prototypes using containers only. That is sufficient to elaborate
the draft content and showcase the realization of the proposed methodology.
Since gym is almost stable, I'm going to implement cases using more robust
traffic generators (moongen, t-rex, etc) and different scenarios, besides
containers.
You can also try the experiment by yourself,
https://github.com/raphaelvrosa/gym.

When I come up with the draft updates I'll properly answer your email
suggestions.

Thanks,
Raphael