Subject: [Gen-art] Gen-ART LC review of draft-ietf-bmwg-igp-dataplane-conv-meth-23

From: "Roni Even" <[email-masked]>
Date: Tue, 16 Aug 2011 10:36:12 +0300

I am the assigned Gen-ART reviewer for this draft. For background on
Gen-ART, please see the FAQ at
<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Please resolve these comments along with any other Last Call comments you
may receive.

Document: draft-ietf-bmwg-igp-dataplane-conv-meth-23

Reviewer: Roni Even

Review Date: 2011-8-16

IETF LC End Date: 2011-8-19

IESG Telechat date: 

Summary: This draft is ready for publication as an informational RFC.  

Major issues:

Minor issues:

I find that the document provides in depth testing procedures and reporting.
What I think may be helpful is a recommendation about how many times to
repeat each of the tests to get more accurate results. Is there any
experience.

Maybe it will be good to have some text on experience from actual testing
that were done based on the document.

Nits/editorial comments:  

I noticed that the issue of normative references was discussed and I had
similar observation since the references in the terminology section are
normative and Po11t is in draft forms. Did you consider why they are
normative and not informative.

---

From: Al Morton <[email-masked]>
Date: Thu, 18 Aug 2011 20:31:59 -0400

>Summary: This draft is ready for publication as an informational RFC.

Thanks for your review, Roni.

After 8 years, four of them following the first
trip to IESG, all DISCUSSes are *finally cleared*.
We'll be going ahead with the current text + an RFC Editor's note.

The reference issue is well-known in a WG that makes normative
references to its earlier work (obviously) and
has traditionally produced all its RFCs in the
Informational category - it's the relatively new
classification of normative/informative that causes
this issue, because the BMWG pre-dates the requirement to
separate references in these categories.

regards,
Al
bmwg chair

---

From: "Roni Even" <[email-masked]>
Date: Fri, 19 Aug 2011 13:12:31 +0300

Hi Al,
I trust that you considered the references issue and have no issue with
them.
I had another question whether there is some experience from actually
conducting the tests based on the long time this draft was under
development.
Regards
Roni

> -----Original Message-----
> From: Al Morton [mailto:[email-masked]]
> Sent: Friday, August 19, 2011 3:32 AM
> To: Roni Even; draft-ietf-bmwg-igp-dataplane-conv-
> [email-masked]
> Cc: [email-masked]
> Subject: Re: Gen-ART LC review of draft-ietf-bmwg-igp-dataplane-conv-
> meth-23
>

---

From: Al Morton <[email-masked]>
Date: Fri, 19 Aug 2011 15:04:06 -0400

>conducting the tests based on the long time this draft was under

Roni,

There were several labs conducting these tests before
the drafts were prepared, in fact, the state of the art
for convergence testing advanced while these drafts were in
development - which is one reason we had difficulty in the
first IESG review. So there is plenty of test experience.

You actually asked if we could provide guidance on how many
times to repeat tests, but the WG has declined to do this.
I'm sure you understand that with a wide range of equipment
and testing circumstances only general guidance can be given,
such as:

section 5.6
... It is RECOMMENDED to repeat a test multiple times with
    different random ranges of the header fields such that convergence
    time benchmarks are measured for different distributions of traffic
    over the available paths.

and
5.8. Measurement Statistics

    The benchmark measurements may vary for each trial, due to the
    statistical nature of timer expirations, cpu scheduling, etc.
    Evaluation of the test data must be done with an understanding of
    generally accepted testing practices regarding repeatability,
    variance and statistical significance of a small number of trials.

hope this helps,
Al

---

From: "Roni Even" <[email-masked]>
Date: Sat, 20 Aug 2011 19:10:43 +0300

Hi Al,
Thanks for the explanation, I am OK if this is the group consensus.
Roni

> Sent: Friday, August 19, 2011 10:04 PM
> Subject: RE: Gen-ART LC review of draft-ietf-bmwg-igp-dataplane-conv-
> meth-23
>