Subject: [bmwg] AD Review of draft-ietf-bmwg-ngfw-performance

From: Warren Kumari <[email-masked]>
Date: Wed, 15 Dec 2021 10:37:28 -0500

Hi there all,

I'd like to apologize to the authors and WG for how long it has taken me to
post this review; I read the document a while back, but forgot to post it
:-(

I only had a few editorial / readability suggestions -- they are
sufficiently minor (and I'm embarrassed by how long my review took!) that
I'll kick off LC without asking for a new version -- authors, please
consider them if you post a new version addressing LC comments, or to
address IESG Eval comments.

Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up
Underway" - I'm assuming that this is from:
https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU -
can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.

W

1.  Introduction

   18 years have passed since IETF recommended test methodology and
   terminology for firewalls initially ([RFC3511]).  The requirements
   for network security element performance and effectiveness have
   increased tremendously since then.  Security function implementations

[O] 18 years have passed since IETF recommended test methodology and

   terminology for firewalls initially ([RFC3511]).  The requirements
   for network security element performance and effectiveness have
   increased tremendously since then.
[P] In the eighteen years since [RFC3511] was published, recommending
test methodology and terminology for firewalls, requirements and
expectations for network security elements has increased tremendously.

   have evolved to more advanced areas and have diversified into
   intrusion detection and prevention, threat management, analysis of
   encrypted traffic, etc.  In an industry of growing importance, well-
   defined, and reproducible key performance indicators (KPIs) are
   increasingly needed as they enable fair and reasonable comparison of

[O] as they enable
[P] to enable

   network security functions.

-- 
Perhaps they really do strive for incomprehensibility in their specs.
After all, when the liturgy was in Latin, the laity knew their place.
-- Michael Padlipsky

---

From: "MORTON JR., AL" <[email-masked]>
Date: Wed, 15 Dec 2021 16:26:31 +0000

Warren,
you wrote:
... Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up Underway" - I'm assuming that this is from: https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU__;!!BhdT!wptYbr7_rABXnlwTDr-uojLzp5chs82rns2i3R4L0xCYckS-_5CYI7dLYf3U$> - can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.

Done,
Al

From: bmwg <[email-masked]> On Behalf Of Warren Kumari
Sent: Wednesday, December 15, 2021 10:37 AM
To: [email-masked]; [email-masked]
Subject: [bmwg] AD Review of draft-ietf-bmwg-ngfw-performance

Hi there all,

I'd like to apologize to the authors and WG for how long it has taken me to post this review; I read the document a while back, but forgot to post it :-(

I only had a few editorial / readability suggestions -- they are sufficiently minor (and I'm embarrassed by how long my review took!) that I'll kick off LC without asking for a new version -- authors, please consider them if you post a new version addressing LC comments, or to address IESG Eval comments.

Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up Underway" - I'm assuming that this is from: https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU__;!!BhdT!wptYbr7_rABXnlwTDr-uojLzp5chs82rns2i3R4L0xCYckS-_5CYI7dLYf3U$> - can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.

W

1.  Introduction

[P] In the eighteen years since [RFC3511] was published, recommending test methodology and terminology for firewalls, requirements and expectations for network security elements has increased tremendously.

[O] as they enable

[P] to enable

---

From: Warren Kumari <[email-masked]>
Date: Wed, 15 Dec 2021 11:48:27 -0500

On Wed, Dec 15, 2021 at 11:26 AM MORTON JR., AL <[email-masked]> wrote:

> Warren,
>
> you wrote:
>
> ... Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up
> https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU
> <https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU__;!!BhdT!wptYbr7_rABXnlwTDr-uojLzp5chs82rns2i3R4L0xCYckS-_5CYI7dLYf3U$>
> - can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.
>

Thank you, IETF LC requested / started.

W

>
>
> Done,
>
> Al
>
>
>
> *From:* bmwg <[email-masked]> *On Behalf Of *Warren Kumari
> *Sent:* Wednesday, December 15, 2021 10:37 AM
> *To:* [email-masked]; [email-masked]
> *Subject:* [bmwg] AD Review of draft-ietf-bmwg-ngfw-performance
>
>
>
> Hi there all,
>
>
>
> I'd like to apologize to the authors and WG for how long it has taken me
> to post this review; I read the document a while back, but forgot to post
> it :-(
>
>
>
> https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU
> <https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU__;!!BhdT!wptYbr7_rABXnlwTDr-uojLzp5chs82rns2i3R4L0xCYckS-_5CYI7dLYf3U$>
> - can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.
>
>
>
>
>
> W
>
>
>
>
>
> 1.  Introduction
>
>
>

-- 
The computing scientist’s main challenge is not to get confused by the
complexities of his own making.
  -- E. W. Dijkstra

---

From: tim copley <[email-masked]>
Date: Thu, 16 Dec 2021 13:59:30 -0700

I also, wanted to chime in regarding this and again, I also apologize for
not reviewing earlier.

I've not really been too involved in security devices until the last couple
years, but boy has it changed
drastically from when I was.  Also, I noted that this document is basically
following prior standards that
are also a bit out dated.

Some of the evolution, I've noticed which I think should at least be
addressed in.  I think you tried
to exclude these in the Scope.?  However I'm not really sure you are
covering but maybe 1/2 of the devices
that are being deployed at this time without addressing:

Virtualization. /  Remote FWs in the next gen.

*) Large FWs are compartmentalizing Customers and serving multiple
customers with the same firewall.  So while
you have said what would happen to a customer when The Firewall is UT, you
don't really address what's going
on with all the other customers on this DUT.  If there are 10 Customers
going through 1 large FW, what are the
other 9 experiencing when this is under Test.  Would be interesting to have
multiple parallel tests underway
and verify that behaviour is consistent across every "VDOM"  within the
system?

*) Time to Change.  If things are changed during your Load profile.perhaps
even on a different virtual Firewall
within the system,

*) Sandbox dips.  Several of the firewall vendors send stuff through labs /
temp work spaces either online or
offline that allow them to test against Zero Day scenarios.  What happens
to traffic flows during anomalies
in the traffic flows.  Does it change the throughput of the DUT?  I know
this is kind of an open ended question.
but not sure you have good benchmark stats for a FW without it...

*) Cloud based scenarios.  If the firewall is removed, and I'm not really
sure how you would test this with
consistent results, but if the FW is offsite and the traffic is being
routed through a Cloud based FW  What does
that do to your traffic results?

TimC
Fruth Group
[email-masked]

On Wed, Dec 15, 2021 at 8:38 AM Warren Kumari <[email-masked]> wrote:

> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bmwg
>

-- 
Timothy Copley
602.350.0633

---

From: <[email-masked]>
Date: Fri, 17 Dec 2021 11:18:45 -0500

Warren,

Thank you. And, of course, you suggested changes are fine.

Brian

From: Warren Kumari <[email-masked]> 
Sent: Wednesday, December 15, 2021 10:37 AM
To: [email-masked]; [email-masked]
Subject: AD Review of draft-ietf-bmwg-ngfw-performance

Hi there all,

Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up Underway" - I'm assuming that this is from: https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU - can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.

W

1.  Introduction

 

--