Subject: [Last-Call] Tsvart last call review of draft-ietf-bmwg-evpntest-07

From: Jana Iyengar via Datatracker <[email-masked]>
Date: Mon, 24 May 2021 10:38:25 -0700

Reviewer: Jana Iyengar
Review result: Ready with Issues

This document has been reviewed as part of the transport area review team's
ongoing effort to review key IETF documents. These comments were written
primarily for the transport area directors, but are copied to the document's
authors and WG to allow them to address any issues raised and also to the IETF
discussion list for information.

When done at the time of IETF Last Call, the authors should consider this
review as part of the last-call comments they receive. Please always CC
[email-masked] if you reply to or forward this review.

The document seems fine overall. There are some minor grammar and consistency
things, but I expect that the RFC-editors will handle those.

The one thing that stuck out to me is the following: It helps in documents such
as this to be more precise about exactly what a measurement tool or tester
should consider success or failure. One piece of text where this precision
should be improved is in the Soak Test (both 3.12 and 4.11):
  "The CPU spike is determined as the CPU usage which shoots at 40 to 50
  percent of the average usage.
    The average value vary from device to device.  Memory leak is determined
   by increase usage of the memory for EVPN process.  The expectation is
   under steady state the memory usage for EVPN process should not
   increase."
Perhaps something like the following for defining CPU spikes might be helpful:
"A CPU spike is defined as a sudden increase and subsequent decrease in usage
from average usage to about 150% of average usage." Similarly, memory leak is
very weakly defined. Do you mean _any increase_ in memory usage, or is there a
threshold that you want to propose? Do you mean consistent increase over time?
Can you define a leak more precisely in the context of your test?

---

From: Sarah Banks <[email-masked]>
Date: Tue, 25 May 2021 09:24:39 -0700

Hi Jana,
   Thank you for your review. The authors are reviewing the feedback, and plan to update the document shortly. Your catch is a good one - I hadn't noticed this the first time through, and I agree with your feedback. It might be also worthwhile simply documenting the CPU usage; conversely, establishing a recorded baseline before the test starts, and then noting the CPU usage might help the tester draw conclusions around "spikes", but with either approach, the data is captured and documented. Sudhin, this is something for you to consider as you work through the document feedback.

Thank you,
Sarah

> On May 24, 2021, at 10:38 AM, Jana Iyengar via Datatracker <[email-masked]> wrote:
> 
> Reviewer: Jana Iyengar
> -- 
> last-call mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/last-call