Subject: [bmwg] How to apply RFC 4814 pseudorandom source and destination port numbers, when benchmarking tunneling technologies?

From: Lencse Gábor <[email-masked]>
Date: Sat, 19 Sep 2020 22:08:05 +0200

Dear Authors of RFC 4814 and also BMWG Members,

RFC 4814 Section 4.5 recommends to use pseudorandom source and 
destination port numbers. My experience show that their usage is an 
essential precondition to receive realistic results, when multi-core 
devices are benchmarked. (I have recently added this feature to 
siitperf, my DPDK-based RFC 8219 compliant SIIT tester: 
https://github.com/lencsegabor/siitperf/tree/varport Please see my 
report on the BMWG mailing list for the details.)

However, I am not sure, how random ports should be used, when tunneling 
technologies are benchmarked. My MSc student is working on the 
implementation of a DS-Lite tester. He is going to build two separate 
Testers for the B4 and for the AFTR parts. For simplicity, let us 
consider B4 first (as the stateful nature of the AFTR further 
complicates the problem). The Single DUT test and traffic setup of RFC 
8219 would look like this:

                            +--------------------+
                            |                    |
               +------------|IPv4   Tester   IPv6|<-------------+
               |            |              tunnel|              |
               |            +--------------------+              |
               |                                                |
               |            +--------------------+              |
               |            |                    |              |
               +----------->|IPv4     B4     IPv6|--------------+
                            |              tunnel|
                            +--------------------+

Although the arrows are unidirectional, bidirectional traffic is 
required. On the left side, the Tester may send IPv4 traffic with 
pseudorandom source and destination port numbers. The DUT encapsulates 
the IPv4 packets into IPv6 packets and forwards them to the IPv6 port of 
the DUT. However, on the right side, the DUT has to use fixed UDP port 
numbers, when sending IPv6 traffic, as there is a tunnel there. Of 
course, the encapsulated IPv4 packets may have pseudorandom UDP port 
numbers, but they are not taken into consideration by the NIC of the B4 
device. Thus the interrupts caused by the arrival of the IPv6 packets 
will not be hashed to all the cores of the B4 devices.

IMHO, we may say that it is NOT a problem regarding the benchmarking 
method, because of the following consideration:

We use pseudorandom port numbers for benchmarking in order to 
approximate the nature of the real life traffic, which usually has 
different IP addresses and port numbers. However, in this case, the real 
life traffic does have fixed port numbers due to the tunnel, thus having 
fixed UDP port numbers of over IPv6 complies with the real life traffic. 
(In addition to that, B4 is usually a small CE device, which perhaps 
does not have too many cores.)

Am I right, or do I miss something?

The situation of the AFTR is very different.

                            +--------------------+
                            |                    |
               +------------|IPv6   Tester   IPv4|<-------------+
               |            |tunnel              |              |
               |            +--------------------+              |
               |                                                |
               |            +--------------------+              |
               |            |                    |              |
               +----------->|IPv6    AFTR    IPv4|--------------+
                            |tunnel              |
                            +--------------------+

On the left side, the Tester needs to send IPv6 packets that encapsulate 
IPv4 packets. In real life, a high number of B4 devices are connected to 
an AFTR. Thus although the port numbers are fixed due to the tunnels, 
the IPv6 source addresses are different, and thus the interrupts caused 
by the packets are hashed more or less equally to the several CPU cores 
of the AFTR. (Of course, the encapsulated IPv4 packets have pseudorandom 
port numbers, but the NIC does not "see" them.)

Am I right, that here the real spirit of RFC 4814 is that the Tester 
should use different source IPv6 addresses?

Of course, their number (e.g. 256 or 65536) is a further question.

As for the IPv4 traffic in the other direction, varying port number may 
be (and should be) used. However, they may not be arbitrarily due to the 
stateful NAPT44 in the AFTR. To that end, I have an idea: the tester 
should send preliminary traffic in the left to right direction, where 
the encapsulated IPv4 packets have varying (not random but rather 
one-by-one increasing) port numbers, and it should store the observed 
port numbers on the right side. Then it may randomly select from among 
the observed port numbers, when generating traffic in the right to left 
direction. But it is beyond the scope of my current question.

I really appreciate any suggestions, comments, ideas!

Best regards,

Gábor Lencse

-----------------------------------------------------
Dr. Gabor Lencse
Professor
Department of Telecommunications
Szechenyi Istvan University
Egyetem ter 1.  Gyor, H-9026, HUNGARY
FAX: +36.96.613.646
E-Mail: [email-masked]
URL: http://www.hit.bme.hu/people/lencse/index_en.htm

---

From: David Newman <[email-masked]>
Date: Wed, 23 Sep 2020 15:02:01 -0700

We did not consider middleboxes when writing 4814, which was intended to
address deficiencies in benchmarking of one or a small set of switches
and routers.

Per standard bmwg practice at the time, we treated the DUT/SUT as a
single black box, where ingress and egress ports didn't know or care
what happened inside the box.

However, a possible problem with the pseudorandom-everywhere approach
can arise when ingress/egress ports and one or more intermediate
tunneling mechanisms require different and possibly conflicting patterns
to obtain optimal traffic distributions at tunneling vs. egress ports.

Obviously, if you're testing some sort of encapsulation scheme, then the
specifics of that protocol are fixed, but you, as the tester, need to
understand the traffic you're generating and whether it represents a
fair test for arbitrary boxes or has been specifically tailored to your
device or system.

The spirit of 4814 is to use your knowledge of the test environment and
protocols to provide a way to  scale test traffic that is free from bias
when used to compare arbitrary boxes that perform the function that is
tested.

The original intent was to scale endpoints, e.g. MAC addresses, IP
addresses, etc. If you're testing a protocol that is between two fixed
endpoints, then scaling addresses might not be a reasonable approach for
traffic scaling, even if it provides better performance because hashing
is easier.

You really have to use your best judgement, as a knowledgeable tester,
to be fair and not cheat.

Also, a wording nit: We have a strong allergy to the term "real-life" in
the context of bmwg benchmarking. As former bmwg chair Scott Bradner
once asked, "This begs the question: Real for whom?"

As with approaches to encapsulation, there is no valid one-size-fits-all
answer to that question.

dn and tcp