Subject: Re: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

From: "MORTON JR., AL" <[email-masked]>
Date: Sat, 16 Oct 2021 22:12:27 +0000

Hi authors and BMWG,

Thank you for all efforts to complete the review, comment resolution, and document revisions!

I have completed the "first-pass" document shepherd's review of this draft.

I attached and uploaded the current version of the shepherd's review form, which contains Q&A with some action items for the authors.

I want to be sure that the authors considered the "early" Security review from Kathleen Moriarty:
https://datatracker.ietf.org/doc/review-ietf-bmwg-ngfw-performance-00-secdir-early-moriarty-2019-07-08/

Also, I have a few small suggestions below.

Thanks again,
Al
bmwg co-chair

We haven't tied the "in-line" terminology to the Figures; I think it would be good to do that. Many non-benchmarking experts will read this doc in the coming months (and this is an easy future comment to avoid).
OLD
3.  Scope

   This document provides testing terminology and testing methodology
   for modern and next-generation network security devices that are
   configured in Active ("Inline") mode.
NEW

3.  Scope

   This document provides testing terminology and testing methodology

   for modern and next-generation network security devices that are

   configured in Active ("Inline", see Figures 1 and 2) mode.

-=-=-=-=-=-

In section 4.3.3, the word "balanced\" appears, and the trailing slash needs to be deleted.

-=-=-=-=-=-=-

In section 7.1.1, the sentence beginning

OLD

   Based on customer use case, users can choose...

tripped me up, and a few more words will help, I think:

NEW

   Based on the test customer's specific use case, testers can choose...

(note that there is only one instance of "customer" in the doc, so we should make the customer's role clear here)

-=-=-=-=-=-=-

From: bmwg <[email-masked]> On Behalf Of MORTON JR., AL
Sent: Sunday, September 26, 2021 12:54 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

***Security Advisory: This Message Originated Outside of AT&T ***
Reference http://cso.att.com/EmailSecurity/IDSP.html for more information.
BMWG,

Those who have submitted comments on recent versions (8,9) should check the diffs now.
Please confirm that your comments have been addressed in version 10, by e-mail, ASAP.

thanks,
Al
bmwg co-chair

From: bmwg <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Sunday, September 26, 2021 10:31 AM
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

Folks,

An update to draft-ietf-bmwg-ngfw-performance has been posted. I believe the next step is WG Chair review.

Version 10 has gone through multiple reviews . As a result, we have corrected a number of typos and grammatical errors. Additionally, we clarified wording in a few spots. The following has also been changed as a result of comments/discussions from/with Sarah Banks. (Sarah, thank you.)

  *   Removed NGIDS from the draft
  *   Added the following text in section 3 for "Inline" mode and "Fail-open" clarification: " This document provides testing terminology and testing methodology for modern and next-generation network security devices that are configured in Active ("Inline") mode."
  *   Also, we added the following text in section 4.2:  "DUT/SUT MUST be configured in "Inline" mode so that the traffic is actively inspected by the DUT/SUT.  Also "Fail-Open" behavior MUST be disabled on the DUT/SUT."
  *   Added more clarification for the parameters and values defined in section "4.3.1.  Client Configuration": This section specifies which parameters SHOULD be considered while configuring clients using test equipment.  Also, this section specifies the RECOMMENDED values for certain parameters.  The values are the defaults used in most of the client operating systems currently.
  *   Rephrased section 5 " Testbed Consideration". Added recommended steps for reference test.
  *   Explained the usage of the parameter "Initial throughput": "Initial throughput is not a KPI to report.  This value is configured on the traffic generator and used to perform Step 1: "Test Initialization and Qualification" described under the Section 7.1.4."
Brian

---------
Brian Monkman
Executive Director, NetSecOPEN
Office: +1-717-610-0808
Fax: +1-717-506-0460
Mobile: +1-717-462-5422

https://www.netsecopen.org<https://urldefense.com/v3/__https:/www.netsecopen.org__;!!BhdT!wpaM8GOInxvNJdztL0uqXkYCL1UWmjmmAASp8hiMJEKN0JgjzEyGqz2m7207$>

---

From: <[email-masked]>
Date: Sat, 16 Oct 2021 20:25:18 -0400

Thanks Al. These are reasonable suggestions.

Is there any additional input we should wait for before posting a new
version?

Brian

From: MORTON JR., AL <[email-masked]> 
Sent: Saturday, October 16, 2021 6:12 PM
To: MORTON JR., AL <[email-masked]>; [email-masked];
[email-masked]
Cc: [email-masked]; 'Carsten Rossenhoevel' <[email-masked]>
Subject: RE: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

Hi authors and BMWG,

Thank you for all efforts to complete the review, comment resolution, and
document revisions!

I attached and uploaded the current version of the shepherd's review form,
which contains Q&A with some action items for the authors.

I want to be sure that the authors considered the "early" Security review
from Kathleen Moriarty:

https://datatracker.ietf.org/doc/review-ietf-bmwg-ngfw-performance-00-secdir
-early-moriarty-2019-07-08/

Thanks again,

Al

bmwg co-chair

We haven't tied the "in-line" terminology to the Figures; I think it would
be good to do that. Many non-benchmarking experts will read this doc in the
coming months (and this is an easy future comment to avoid).

OLD 

3.  Scope

NEW 

3.  Scope

-=-=-=-=-=-
 
In section 4.3.3, the word "balanced\" appears, and the trailing slash needs
to be deleted.
 
-=-=-=-=-=-=-

 
(note that there is only one instance of "customer" in the doc, so we should
make the customer's role clear here)
-=-=-=-=-=-=-

 

From: bmwg <[email-masked] <mailto:[email-masked]> > On Behalf
Of MORTON JR., AL
Sent: Sunday, September 26, 2021 12:54 PM
To: [email-masked] <mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> 
Cc: [email-masked] <mailto:[email-masked]> 
Subject: Re: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

BMWG,

Those who have submitted comments on recent versions (8,9) should check the
diffs now.

Please confirm that your comments have been addressed in version 10, by
e-mail, ASAP.

thanks,

Al

bmwg co-chair

From: bmwg <[email-masked] <mailto:[email-masked]> > On Behalf
Of [email-masked] <mailto:[email-masked]> 
Sent: Sunday, September 26, 2021 10:31 AM
To: [email-masked] <mailto:[email-masked]> 
Cc: [email-masked] <mailto:[email-masked]> 
Subject: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

Folks,

An update to draft-ietf-bmwg-ngfw-performance has been posted. I believe the
next step is WG Chair review.

Version 10 has gone through multiple reviews . As a result, we have
corrected a number of typos and grammatical errors. Additionally, we
clarified wording in a few spots. The following has also been changed as a
result of comments/discussions from/with Sarah Banks. (Sarah, thank you.)

*	Removed NGIDS from the draft
*	Added the following text in section 3 for "Inline" mode and
"Fail-open" clarification: " This document provides testing terminology and
testing methodology for modern and next-generation network security devices
that are configured in Active ("Inline") mode."
*	Also, we added the following text in section 4.2:  "DUT/SUT MUST be
configured in "Inline" mode so that the traffic is actively inspected by the
DUT/SUT.  Also "Fail-Open" behavior MUST be disabled on the DUT/SUT."
*	Added more clarification for the parameters and values defined in
section "4.3.1.  Client Configuration": This section specifies which
parameters SHOULD be considered while configuring clients using test
equipment.  Also, this section specifies the RECOMMENDED values for certain
parameters.  The values are the defaults used in most of the client
operating systems currently.
*	Rephrased section 5 " Testbed Consideration". Added recommended
steps for reference test.
*	Explained the usage of the parameter "Initial throughput": "Initial
throughput is not a KPI to report.  This value is configured on the traffic
generator and used to perform Step 1: "Test Initialization and
Qualification" described under the Section 7.1.4."

Brian

---------

Brian Monkman

Office: +1-717-610-0808 

Fax: +1-717-506-0460

Mobile: +1-717-462-5422

https://www.netsecopen.org
<https://urldefense.com/v3/__https:/www.netsecopen.org__;!!BhdT!wpaM8GOInxvN
JdztL0uqXkYCL1UWmjmmAASp8hiMJEKN0JgjzEyGqz2m7207$>

---

From: "MORTON JR., AL" <[email-masked]>
Date: Mon, 18 Oct 2021 10:44:08 +0000

From: [email-masked] <[email-masked]>
Sent: Saturday, October 16, 2021 8:25 PM
To: MORTON JR., AL <[email-masked]>; [email-masked]
Cc: [email-masked]; 'Carsten Rossenhoevel' <[email-masked]>
Subject: RE: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

Is there any additional input we should wait for before posting a new version?
[acm]
Hi Brian,

We have allowed 3 weeks (since Sept 26) for folks who commented on recent versions to check that their comments are addressed (see my message in the thread below).

I think it's time to begin work on the next revision, and prepare for next steps (AD-review).

Al
bmwg co-chair

Brian

From: MORTON JR., AL <[email-masked]<mailto:[email-masked]>>
Sent: Saturday, October 16, 2021 6:12 PM
To: MORTON JR., AL <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; 'Carsten Rossenhoevel' <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

Hi authors and BMWG,

I want to be sure that the authors considered the "early" Security review from Kathleen Moriarty:
https://datatracker.ietf.org/doc/review-ietf-bmwg-ngfw-performance-00-secdir-early-moriarty-2019-07-08/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/review-ietf-bmwg-ngfw-performance-00-secdir-early-moriarty-2019-07-08/__;!!BhdT!woM2hDX-E_U_pSFgGJ_QYw1td2qBKmSPty3zxubJhBT3EDxLTlRXnbuiX5Mu$>

Thanks again,
Al
bmwg co-chair

3.  Scope

-=-=-=-=-=-

-=-=-=-=-=-=-

OLD

NEW

-=-=-=-=-=-=-

From: bmwg <[email-masked]<mailto:[email-masked]>> On Behalf Of MORTON JR., AL
Sent: Sunday, September 26, 2021 12:54 PM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [bmwg] Version 10, draft-ietf-bmwg-ngfw-performance

thanks,
Al
bmwg co-chair

Folks,

---------
Brian Monkman