Subject: [bmwg] Lars Eggert's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)

From: Lars Eggert via Datatracker <[email-masked]>
Date: Thu, 03 Feb 2022 05:27:15 -0800

Lars Eggert has entered the following ballot position for
draft-ietf-bmwg-ngfw-performance-13: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/blog/handling-iesg-ballot-positions/
for more information about how to handle DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ngfw-performance/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

This document needs TSV and ART people to help with straightening out a lot of
issues related to TCP, TLS, and H1/2/3. Large parts of the document don't
correctly reflect the complex realities of what "HTTP" is these days (i.e.,
that we have H1 and H2 over either TCP or TLS, and H3 over only QUIC.) The
document is also giving unnecessarily detailed behavioral descriptions of TCP
and its parameters, while at the same time not being detailed enough about TLS,
H2 and esp. QUIC/H3. It feels like this stared out as an H1/TCP document that
was then incompletely extended to H2/H3.

Section 4.3.1.1. , paragraph 2, discuss:
>    The TCP stack SHOULD use a congestion control algorithm at client and
>    server endpoints.  The IPv4 and IPv6 Maximum Segment Size (MSS)
>    SHOULD be set to 1460 bytes and 1440 bytes respectively and a TX and
>    RX initial receive windows of 64 KByte.  Client initial congestion
>    window SHOULD NOT exceed 10 times the MSS.  Delayed ACKs are
>    permitted and the maximum client delayed ACK SHOULD NOT exceed 10
>    times the MSS before a forced ACK.  Up to three retries SHOULD be
>    allowed before a timeout event is declared.  All traffic MUST set the
>    TCP PSH flag to high.  The source port range SHOULD be in the range
>    of 1024 - 65535.  Internal timeout SHOULD be dynamically scalable per
>    RFC 793.  The client SHOULD initiate and close TCP connections.  The
>    TCP connection MUST be initiated via a TCP three-way handshake (SYN,
>    SYN/ACK, ACK), and it MUST be closed via either a TCP three-way close
>    (FIN, FIN/ACK, ACK), or a TCP four-way close (FIN, ACK, FIN, ACK).

There are a lot of requirements in here that are either no-ops ("SHOULD use a
congestion control algorithm"), nonsensical ("maximum client delayed ACK SHOULD
NOT exceed 10 times the MSS") or under the sole control of the stack. This
needs to be reviewed and corrected by someone who understands TCP.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Section 1. , paragraph 2, comment:
>    18 years have passed since IETF recommended test methodology and
>    terminology for firewalls initially ([RFC3511]).  The requirements
>    for network security element performance and effectiveness have
>    increased tremendously since then.  In the eighteen years since

These sentences don't age well - rephrase without talking about particular
years?

Section 4.3.2.3. , paragraph 2, comment:
>    The server pool for HTTP SHOULD listen on TCP port 80 and emulate the
>    same HTTP version (HTTP 1.1 or HTTP/2 or HTTP/3) and settings chosen
>    by the client (emulated web browser).  The Server MUST advertise

An H3 server will not listen on TCP port 80. In general, the document needs to
be checked for the implicit assumption that HTTP sues TCP; there is text
throughout that is nonsensical for H3 (like this example).    The Server MUST
advertise

Section 6.3. , paragraph 6, comment:
>       The average number of successfully established TCP connections per
>       second between hosts across the DUT/SUT, or between hosts and the
>       DUT/SUT.  The TCP connection MUST be initiated via a TCP three-way
>       handshake (SYN, SYN/ACK, ACK).  Then the TCP session data is sent.
>       The TCP session MUST be closed via either a TCP three-way close
>       (FIN, FIN/ACK, ACK), or a TCP four-way close (FIN, ACK, FIN, ACK),
>       and MUST NOT by RST.

This prohibits TCP fast open, why? Also, wouldn't it be enough to say that the
connection needs to not abnormally reset, rather than describing the TCP packet
sequences that are acceptable? Given that those are not the only possible
sequences, c.f., loss and reordering.

Section 6.3. , paragraph 6, comment:
>       The average number of successfully completed transactions per
>       second.  For a particular transaction to be considered successful,
>       all data MUST have been transferred in its entirety.  In case of
>       HTTP(S) transactions, it MUST have a valid status code (200 OK),
>       and the appropriate FIN, FIN/ACK sequence MUST have been
>       completed.

H3 doesn't do FIN/ACK, etc. See above.

Section 7.1.3.4. , paragraph 4, comment:
>    a.  Number of failed application transactions (receiving any HTTP
>        response code other than 200 OK) MUST be less than 0.001% (1 out
>        of 100,000 transactions) of total attempted transactions.
>
>    b.  Number of Terminated TCP connections due to unexpected TCP RST
>        sent by DUT/SUT MUST be less than 0.001% (1 out of 100,000
>        connections) of total initiated TCP connections.

Why is a 0.001% failure rate deemed acceptable? (Also elsewhere.)

Section 7.2.1. , paragraph 2, comment:
>    Using HTTP traffic, determine the sustainable TCP connection
>    establishment rate supported by the DUT/SUT under different
>    throughput load conditions.

H3 doesn't do TCP.

Section 7.2.3.2. , paragraph 9, comment:
>    The client SHOULD negotiate HTTP and close the connection with FIN
>    immediately after completion of one transaction.  In each test
>    iteration, client MUST send GET request requesting a fixed HTTP
>    response object size.

H3 doesn't do TCP FIN.

Section 7.2.3.3. , paragraph 6, comment:
>    c.  During the sustain phase, traffic SHOULD be forwarded at a
>        constant rate (considered as a constant rate if any deviation of
>        traffic forwarding rate is less than 5%).

What does this mean? How would traffic NOT be forwarded at a constant rate?

Section 7.2.3.3. , paragraph 5, comment:
>    d.  Concurrent TCP connections MUST be constant during steady state
>        and any deviation of concurrent TCP connections SHOULD be less
>        than 10%. This confirms the DUT opens and closes TCP connections
>        at approximately the same rate.

What does it mean for a TCP connection to be constant?

Section 7.4.1. , paragraph 4, comment:
>    Scenario 1: The client MUST negotiate HTTP and close the connection
>    with FIN immediately after completion of a single transaction (GET
>    and RESPONSE).

H3 sessions don't send TCP FINs. (Also elsewhere.)

Section 7.7. , paragraph 1, comment:
> 7.7.  HTTPS Throughput

Is this HTTPS as in H1, H2 or H3? All of the above?

Found terminology that should be reviewed for inclusivity; see
https://www.rfc-editor.org/part2/#inclusive_language for background and more
guidance:

 * Term "dummy"; alternatives might be "placeholder", "sample", "stand-in",
   "substitute".

Thanks to Matt Joras for their General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/NUycZt5uKAZejOvCr6tdi_7SvPA).

-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

Section 4.1. , paragraph 8, nit:
>  actively inspected by the DUT/SUT. Also "Fail-Open" behavior MUST be disable
>                                     ^^^^
A comma may be missing after the conjunctive/linking adverb "Also".

Section 4.2. , paragraph 9, nit:
> security vendors implement ACL decision making.) The configured ACL MUST NOT
>                                ^^^^^^^^^^^^^^^
The noun "decision-making" (= the process of deciding something) is spelled
with a hyphen.

Section 4.2.1. , paragraph 1, nit:
>  the MSS. Delayed ACKs are permitted and the maximum client delayed ACK SHOUL
>                                     ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

Section 4.3.1.3. , paragraph 3, nit:
>  the MSS. Delayed ACKs are permitted and the maximum server delayed ACK MUST
>                                     ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

Section 4.3.1.3. , paragraph 4, nit:
> IPv6 with a ratio identical to the clients distribution ratio. Note: The IAN
>                                    ^^^^^^^
An apostrophe may be missing.

Section 4.3.3.1. , paragraph 2, nit:
> S throughput performance test with smallest object size. 3. Ensure that any
>                                    ^^^^^^^^
A determiner may be missing.

Section 6.1. , paragraph 19, nit:
> sion with a more specific Kbit/s in parenthesis. * Time to First Byte (TTFB)
>                                  ^^^^^^^^^^^^^^
Did you mean "in parentheses"? "parenthesis" is the singular.

Section 7.5.3. , paragraph 2, nit:
> s and key strengths as well as forward looking stronger keys. Specific test
>                                ^^^^^^^^^^^^^^^
This word is normally spelled with a hyphen.

Section 7.5.4.2. , paragraph 3, nit:
> SHOULD NOT be reported, if the above mentioned KPI (especially inspected thro
>                                ^^^^^^^^^^^^^^^
The adjective "above-mentioned" is spelled with a hyphen.

Section 7.6.1. , paragraph 4, nit:
> s and key strengths as well as forward looking stronger keys. Specific test
>                                ^^^^^^^^^^^^^^^
This word is normally spelled with a hyphen.

Section 7.9.3.4. , paragraph 1, nit:
> * Accuracy of DUT/SUT statistics in term of vulnerabilities reporting A.2. T
>                                  ^^^^^^^^^^
Did you mean the commonly used phrase "in terms of"?

Section 7.9.4. , paragraph 2, nit:
> tected attack traffic MUST be dropped and the session SHOULD be reset A.3.2.
>                                      ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

---

From: Carsten Rossenhoevel <[email-masked]>
Date: Thu, 3 Feb 2022 14:41:48 +0100

Lars,

I think it is a moot point to respond to your comments.

As author who has been available for discussions and reviews of this 
draft across more than three years, not speaking about my professional 
background as CTO of an independent test lab having dealt with telecom 
network testing for more than 25 years, I personally feel that your 
comments are highly disrespectful.

This kind of review is not appropriate.  In case you'd like to say we 
need to take our work to another SDO, just plainly say that you don't 
want to deal with it.

Best regards, Carsten

-- 
Carsten Rossenhövel
Managing Director, EANTC AG (European Advanced Networking Test Center)
Salzufer 14, 10587 Berlin, Germany
office +49.30.3180595-21, fax +49.30.3180595-10, mobile +49.177.2505721
[email-masked], https://www.eantc.de

Place of Business/Sitz der Gesellschaft: Berlin, Germany
Chairman/Vorsitzender des Aufsichtsrats: Herbert Almus
Managing Directors/Vorstand: Carsten Rossenhövel, Gabriele Schrenk
Registered: HRB 73694, Amtsgericht Charlottenburg, Berlin, Germany
EU VAT No: DE812824025

---

From: Lars Eggert <[email-masked]>
Date: Thu, 3 Feb 2022 15:47:07 +0200

Hi Carsten,

On 2022-2-3, at 15:41, Carsten Rossenhoevel <[email-masked]> wrote:
> As author who has been available for discussions and reviews of this draft across more than three years, not speaking about my professional background as CTO of an independent test lab having dealt with telecom network testing for more than 25 years, I personally feel that your comments are highly disrespectful.
> 
> This kind of review is not appropriate.  In case you'd like to say we need to take our work to another SDO, just plainly say that you don't want to deal with it.

I'm sorry if you have understood my review comments as some sort of personal attack. I'd appreciate it if you would let me know which parts you found disrespectful?

Thanks,
Lars

---

From: Carsten Rossenhoevel <[email-masked]>
Date: Thu, 3 Feb 2022 15:09:36 +0100

- "There are a lot of requirements in here that are either no-ops [..], 
non-sensical [..]"

- "This document needs TSV and ART people to help with straightening out 
a lot of issues"

- "The document is also giving unnecessarily detailed behavioral 
descriptions [..]"

These do not feel like comments from a professional to a peer group of 
similar professionals.

/C

---

From: Lars Eggert <[email-masked]>
Date: Thu, 3 Feb 2022 16:27:55 +0200

Hi Carsten,

thanks for clarifying!

First, let me point you at https://www.ietf.org/about/groups/iesg/statements/handling-ballot-positions/, which hopefully clarifies the intent behind these IESG reviews and how they are best handled.

Second, let me try and provide a bit more context on the issues you flagged:

On 2022-2-3, at 16:09, Carsten Rossenhoevel <[email-masked]> wrote:
> - "There are a lot of requirements in here that are either no-ops [..], non-sensical [..]"

That was in reference to the TCP paragraph. Specifically:

* "SHOULD use a congestion control algorithm" is superfluous (= a no-op), because the TCP standard already requires that (at MUST level)

* the delay for delayed ACKs is typically specified as a time interval; the document suggests to compute it as a byte size ("10 times the MSS"), which does not immediately make sense

* "Internal timeout SHOULD be dynamically scalable per RFC 793." It is unclear what is meant by the "internal timeout" or how it would dynamically scale. The reference to RFC 793 does not provide clarity here.

> - "This document needs TSV and ART people to help with straightening out a lot of issues"

I was writing this is a hint to the WG on where to go for help with addressing these issues.

> - "The document is also giving unnecessarily detailed behavioral descriptions [..]"

One reason I flagged this is because these details rule out existing (e.g., fast open) or future TCP extensions. Another reason is that they are inaccurate, because although they are detailed they still do not capture all possible permitted behaviors (e.g., connection-close packet exchanges.)

Thanks,
Lars