Subject: Re: [bmwg] I-D IPv6 transition technologies benchmarking

From: "Bhuvaneswaran Vengainathan" <[email-masked]>
Date: Mon, 8 Dec 2014 10:52:58 +0530

Hi Marius,

I gave a quick look at the draft. The draft is interesting and seems to be
in the right direction.

Here are some if my comments on this draft.

1.       Section 1.1 - May need to consider tunneling mechanism (e.g., GRE)
as one of the transport technology for sending IPvX/Y frames over
non-supported core networks.

2.       Section 3.2 Encapsulation/Translation based transition technologies
- I think the MTU settings can be recommended for the interfaces between CE
and PE DUTs.

3.       Section 4 Test Traffic - This section limits the test traffic to
unicast. How about handling the Multicast traffic?

4.       Section 6.0 - Benchmarking Tests. The routing mode (static or
dynamic) between edges may influence the benchmarking test results. Should
it be explicitly mentioned?

5.       Section 6.0 - How about including a test for Jitter?

6.       Section 7.2- Reporting Format - Should the reporting format also
include a column for Traffic Type (e.g., TCP or UDP)? Also it is recommended
to capture Traffic Type as part of Section 4.

7.       Appendix A.1 - The Frame size and fps calculation explained here is
for UDP frames? For TCP, the frame size/fps depends on TCP window sizes’
too.

Best Regards,

Bhuvan | Cloud and SDN Solutions

Phone: +91 44 4567 2222 x242

Mobile: +91 988 428 6693

Description: Description: Description: Description: Description:
Description: Description: Description: Description: Veryx-Logo for Webinar

www.veryxtech.com <http://www.veryxtech.com/>

From: bmwg [mailto:[email-masked]] On Behalf Of Marius Georgescu
Sent: Wednesday, September 24, 2014 9:38 AM
To: [email-masked]
Subject: [bmwg] I-D IPv6 transition technologies benchmarking

Dear BMWG Members,

My name is Marius Georgescu and I am a graduate student at Nara Institute of
Science and Technology (NAIST) in Japan. My main research focus is IPv6
transition technologies benchmarking.  Recently I have been working on an
Internet draft which can be found here:

http://www.ietf.org/id/draft-georgescu-ipv6-transition-tech-benchmarking-00.
txt

I hope to present the draft in the next BMWG meeting, in IETF91. In the
meantime, here is the summary of the draft.

The draft is complementary to the recommendations of RFC2544 and RFC5180,
focusing on IPv6 transition technologies benchmarking. It includes a
tentative classification of transition technologies and proposes associated
test setups. It also proposes a method to acknowledge the frame size
overhead and how it should be considered when calculating the maximum
theoretical frame rates, in order to avoid exceeding the bandwidth of the
employed media.

The  draft also includes a tentative benchmark for scalability in the
context of network devices.

Scalability is seen as the ability of network devices to accommodate network
growth. As network growth can produce performance degradation, quantifying
this degradation can offer insights on the scalability of a certain network
devices.

The network growth  can be generated by the tester by creating multiple
network flows. After measuring the performance in the context of multiple
flows, the performance degradation can be expressed as relative change
between the multiple flow results  and the single flow results.

As the draft is very close to the scope of the BMWG, I was hoping it can
become a BMWG draft at some point. Any comments and suggestion are welcome.
Thank you very much.

Best regards,

Marius Georgescu 　(マリウス ジョルジェスク)

Internet Engineering Laboratory

Nara Institute of Science and Technology

mailto: [email-masked]

IPv6NET Project: http://www.ipv6net.ro/

---

From: "Marius Georgescu" <[email-masked]>
Date: Mon, 8 Dec 2014 17:16:19 +0900

Hello Bhuvan,

Thank you very much for taking the time to review the draft. All the
comments are pertinent and I will try my best to address them in the next
version of the draft. To make sure I understood your comments correctly,
here are some inline short answers. I started my answers with MG: …

-          MG: GRE is only one of the encapsulation technologies. There are
many other suitable examples: 6in4, 4in6, LISP… My idea is to classify them
generically using the 4 mentioned categories. In this case a network device
using GRE could fit in the Encapsulation-based technologies and use the
associated test setup.

-          MG: I agree that a MTU recommendation can be very useful for the
reader, especially in the case of encapsulation. I will include it in the
next version

-          MG: The main scope of the methodology is unicast traffic. I think
treating multicast traffic can   overcomplicate the test setups.  Maybe I
should make the scope clearer.

-          MG: I agree the routing can influence the results. I will make
clear recommendations in the  next version.

-          MG: I was thinking about including jitter as well J I will
definitely include it in the next version. Aside from jitter, I was thinking
about some operational benchmarks as well.

-          MG: One of the things  I understood from the IETF91 meeting: TCP
benchmarking  is a “don’t go there” zone because of the unpredictability
of TCP flows. As Scott Bradner and Al Morton very well pointed out, the
methodology should result in a reasonable basis for comparison.
Consequently, I plan to note clearly that the test traffic SHOULD be UDP.

-          MG: It’s clear I did not explain very well the formula. I added
RFC5180 as a reference for the formula, but I will  add some more
explanatory text. The 20 bytes in the formula is the sum of the Ethernet
preamble (8 bytes) and the inter-frame gap (12 bytes); it only covers Layer
2 recommendations. As I explained for the previous comment, UDP will be the
recommended transport layer protocol.

Best regards,

Marius

From: Bhuvaneswaran Vengainathan
[mailto:[email-masked]]
Sent: Monday, December 08, 2014 2:23 PM
To: 'Marius Georgescu'; [email-masked]
Subject: RE: [bmwg] I-D IPv6 transition technologies benchmarking

Hi Marius,

Best Regards,

Mobile: +91 988 428 6693

Dear BMWG Members,

Best regards,

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 8 Dec 2014 09:59:42 -0500

Hi Marius and Bhuvan,

Every few years we have a discussion of "Jitter" on BMWG list.
There are many ways to measure this metric, and how you measure
should be determined by the way you will use the results.
IOW, there doesn't seem to be a generic answer - some
specific need determines your next step.  Some user applications
are less sensitive to "jitter" than others, so they do not
influence the choice of metric - others do.

To that end, I usually cite this RFC:
http://tools.ietf.org/html/rfc5481
and let folks think about it.

regards,
Al

From: bmwg [mailto:[email-masked]] On Behalf Of Marius Georgescu
Sent: Monday, December 08, 2014 3:16 AM
To: 'Bhuvaneswaran Vengainathan'; [email-masked]
Subject: Re: [bmwg] I-D IPv6 transition technologies benchmarking

Hello Bhuvan,

Thank you very much for taking the time to review the draft. All the comments are pertinent and I will try my best to address them in the next version of the draft. To make sure I understood your comments correctly, here are some inline short answers. I started my answers with MG: …

1.       Section 1.1 - May need to consider tunneling mechanism (e.g., GRE) as one of the transport technology for sending IPvX/Y frames over non-supported core networks.

-          MG: GRE is only one of the encapsulation technologies. There are many other suitable examples: 6in4, 4in6, LISP… My idea is to classify them generically using the 4 mentioned categories. In this case a network device using GRE could fit in the Encapsulation-based technologies and use the associated test setup.

2.       Section 3.2 Encapsulation/Translation based transition technologies - I think the MTU settings can be recommended for the interfaces between CE and PE DUTs.

-          MG: I agree that a MTU recommendation can be very useful for the reader, especially in the case of encapsulation. I will include it in the next version

3.       Section 4 Test Traffic - This section limits the test traffic to unicast. How about handling the Multicast traffic?

-          MG: The main scope of the methodology is unicast traffic. I think treating multicast traffic can   overcomplicate the test setups.  Maybe I should make the scope clearer.

4.       Section 6.0 - Benchmarking Tests. The routing mode (static or dynamic) between edges may influence the benchmarking test results. Should it be explicitly mentioned?

-          MG: I agree the routing can influence the results. I will make clear recommendations in the  next version.

-          MG: I was thinking about including jitter as well :) I will definitely include it in the next version. Aside from jitter, I was thinking about some operational benchmarks as well.

6.       Section 7.2- Reporting Format - Should the reporting format also include a column for Traffic Type (e.g., TCP or UDP)? Also it is recommended to capture Traffic Type as part of Section 4.

-          MG: One of the things  I understood from the IETF91 meeting: TCP benchmarking  is a “don’t go there” zone because of the unpredictability of TCP flows. As Scott Bradner and Al Morton very well pointed out, the methodology should result in a reasonable basis for comparison. Consequently, I plan to note clearly that the test traffic SHOULD be UDP.

7.       Appendix A.1 - The Frame size and fps calculation explained here is for UDP frames? For TCP, the frame size/fps depends on TCP window sizes’ too.

-          MG: It’s clear I did not explain very well the formula. I added RFC5180 as a reference for the formula, but I will  add some more explanatory text. The 20 bytes in the formula is the sum of the Ethernet preamble (8 bytes) and the inter-frame gap (12 bytes); it only covers Layer 2 recommendations. As I explained for the previous comment, UDP will be the recommended transport layer protocol.

Best regards,
Marius

From: Bhuvaneswaran Vengainathan [mailto:[email-masked]]
Sent: Monday, December 08, 2014 2:23 PM
To: 'Marius Georgescu'; [email-masked]<mailto:[email-masked]>
Subject: RE: [bmwg] I-D IPv6 transition technologies benchmarking

Hi Marius,

I gave a quick look at the draft. The draft is interesting and seems to be in the right direction.
Here are some if my comments on this draft.

1.       Section 1.1 - May need to consider tunneling mechanism (e.g., GRE) as one of the transport technology for sending IPvX/Y frames over non-supported core networks.

2.       Section 3.2 Encapsulation/Translation based transition technologies - I think the MTU settings can be recommended for the interfaces between CE and PE DUTs.

3.       Section 4 Test Traffic - This section limits the test traffic to unicast. How about handling the Multicast traffic?

4.       Section 6.0 - Benchmarking Tests. The routing mode (static or dynamic) between edges may influence the benchmarking test results. Should it be explicitly mentioned?

6.       Section 7.2- Reporting Format - Should the reporting format also include a column for Traffic Type (e.g., TCP or UDP)? Also it is recommended to capture Traffic Type as part of Section 4.

7.       Appendix A.1 - The Frame size and fps calculation explained here is for UDP frames? For TCP, the frame size/fps depends on TCP window sizes’ too.

Best Regards,

Bhuvan | Cloud and SDN Solutions
Phone: +91 44 4567 2222 x242
Mobile: +91 988 428 6693
www.veryxtech.com<http://www.veryxtech.com/>

From: bmwg [mailto:[email-masked]] On Behalf Of Marius Georgescu
Sent: Wednesday, September 24, 2014 9:38 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [bmwg] I-D IPv6 transition technologies benchmarking

Dear BMWG Members,

My name is Marius Georgescu and I am a graduate student at Nara Institute of Science and Technology (NAIST) in Japan. My main research focus is IPv6 transition technologies benchmarking.  Recently I have been working on an Internet draft which can be found here:
http://www.ietf.org/id/draft-georgescu-ipv6-transition-tech-benchmarking-00.txt

I hope to present the draft in the next BMWG meeting, in IETF91. In the meantime, here is the summary of the draft.

The draft is complementary to the recommendations of RFC2544 and RFC5180, focusing on IPv6 transition technologies benchmarking. It includes a tentative classification of transition technologies and proposes associated test setups. It also proposes a method to acknowledge the frame size overhead and how it should be considered when calculating the maximum theoretical frame rates, in order to avoid exceeding the bandwidth of the employed media.

The  draft also includes a tentative benchmark for scalability in the context of network devices.
Scalability is seen as the ability of network devices to accommodate network growth. As network growth can produce performance degradation, quantifying this degradation can offer insights on the scalability of a certain network devices.
The network growth  can be generated by the tester by creating multiple network flows. After measuring the performance in the context of multiple flows, the performance degradation can be expressed as relative change between the multiple flow results  and the single flow results.

As the draft is very close to the scope of the BMWG, I was hoping it can become a BMWG draft at some point. Any comments and suggestion are welcome. Thank you very much.

Best regards,

mailto: [email-masked]<mailto:[email-masked]>