Subject: [nvo3] AD review of draft-ietf-nvo3-vmm-16

From: Martin Vigoureux <[email-masked]>
Date: Mon, 16 Nov 2020 14:21:32 +0100

Hi,

I have several concerns with this document.

There hasn't been a single response to the WG LC, which by the way 
happened more than two years ago and the draft has undergone 13 
revisions since then.

This document has an evident formatting issue (text width, section 
titles, Status of This Memo, references, ...), as well as a reasonable 
number of typos / unclear sentences making it quite hard to understand.

It apparently aims at addressing Virtual Machine Mobility, but in fact 
seems to only cover IP address preservation during such type of event. 
This must be clarified.

It is not clear to me what is specific to NVO3 networks in this document.

This document is said to describe solutions *commonly* used in data 
centers. Also, it primarily only describes what needs to be done but not 
how.
This makes me wonder what benefit does it bring to the community and to 
operators of data centers.

This document refers to other specifications which themselves do not 
provide the missing pieces e.g., RFC7666 does not describe how to 
transfer VM states; RFC8014 is not a specification for an NVE-to-NVA 
protocol.

How section 7 relates to the rest of the document is unclear. It seems 
to restate some elements described in 4.2 and 4.3 but not all.

Section 8 and 9 seem to be out of the scope of this document.

In its current state the document is not ready for review by the IESG 
and I'm returning it to the WG. I encourage the WG to evaluate the 
benefit of publishing this as an RFC, to discuss whether this should be 
the product of NVO3 and would an alternate publication stream be more 
appropriate. In any case this document needs more work.

-m

---

From: "Bocci, Matthew (Nokia - GB)" <[email-masked]>
Date: Tue, 17 Nov 2020 11:00:14 +0000

Authors

Please can you address Martin’s comments below.

We need to see these resolved in the WG first, and then further review from the WG before deciding whether and how to proceed with the draft in NVO3.

Thanks

Matthew

From: Martin Vigoureux <[email-masked]>
Date: Monday, 16 November 2020 at 13:21
To: [email-masked] <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>
Subject: AD review of draft-ietf-nvo3-vmm-16
Hi,

-m

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 17 Nov 2020 19:38:09 +0000

Martin,

Thank you very much for finally squeezing time to review the draft, after 2 years of WGLC and long journey of revisions to address Areas Directorates comments.  Being an AD is not easy, so many drafts to review..

Replies to your comments are inserted below:

-----Original Message-----
From: Martin Vigoureux <[email-masked]>
Sent: Monday, November 16, 2020 7:22 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: AD review of draft-ietf-nvo3-vmm-16

Hi,

There hasn't been a single response to the WG LC, which by the way happened more than two years ago and the draft has undergone 13 revisions since then.
[Linda] There have been a lot of comments during the Area Directorate reviews, and private comments from NVO3 Chairs.

This document has an evident formatting issue (text width, section titles, Status of This Memo, references, ...), as well as a reasonable number of typos / unclear sentences making it quite hard to understand.
[Linda] Can you elaborate more on those issues? With English as a second language, I am hoping to learn from the mistakes. NVO3 Chair Matthews had a private session with us pointing English language issues, we had revised draft to correct those issues.   We would like to learn more.

It apparently aims at addressing Virtual Machine Mobility, but in fact seems to only cover IP address preservation during such type of event.
This must be clarified.
[Linda] one of the big issues of VM Mobility in Layer 3 is when the VMs need to maintain their original IP addresses after their move. If VMs addresses can be re-assigned after their move, network becomes very simple but applications running on the VMs  can’t be moved freely.
Layer 2 addresses don’t have hierarchical structure as Layer 3 IP addresses. All switches learn the host Layer 2 addresses on the fly and time out the learned addresses in the Forwarding table when there is no packets has the addresses in the SA/DST fields for certain period of time.
That is why the draft is focusing on practices and actions when VMs need to maintain their addresses after the move.

It is not clear to me what is specific to NVO3 networks in this document.
[Linda] The draft is to address the NVO3 Charter:
      The NVO3 WG will develop solutions for network virtualization based on
      the following architectural tenets:
      - Support for an IP-based underlay data plane

This document is said to describe solutions *commonly* used in data centers. Also, it primarily only describes what needs to be done but not how.
This makes me wonder what benefit does it bring to the community and to operators of data centers.
[Linda] This document provides guidelines to DC operators if they want to maintain VMs IP addresses during VM migration. Being able to move VMs dynamically, from one server to another, makes it possible for dynamic load balancing or work distribution. Therefore, dynamic VM Mobility is highly desirable for large scale multi-tenant DCs

This document refers to other specifications which themselves do not provide the missing pieces e.g., RFC7666 does not describe how to transfer VM states;
[Linda] That is why we wrote this draft to indicate that VMs states transferred are needed. But during the numerous reviews, we had been advised to remote the actual protocols of moving states and change the document to be “Informational” (from Standard Track).

RFC8014 is not a specification for an NVE-to-NVA protocol.
[Linda] RFC8014 describes the NVE-to-NVA functions.

How section 7 relates to the rest of the document is unclear. It seems to restate some elements described in 4.2 and 4.3 but not all.
[Linda] Section 4 is the Overview.  Section 7 describes the details of the Overview of Section 4.

Section 8 and 9 seem to be out of the scope of this document.
[Linda] Section 8 and 9 describes what triggers the VMs move. It was suggested from Aera Directorates review comments.

In its current state the document is not ready for review by the IESG and I'm returning it to the WG. I encourage the WG to evaluate the benefit of publishing this as an RFC, to discuss whether this should be the product of NVO3 and would an alternate publication stream be more appropriate. In any case this document needs more work.
[Linda]   We are going around in circles.

-m

---

From: Martin Vigoureux <[email-masked]>
Date: Thu, 19 Nov 2020 16:41:52 +0100

Linda,

please see in-line.

Le 2020-11-17 à 20:38, Linda Dunbar a écrit :
> Martin,
> Thank you very much for finally squeezing time to review the draft, 
> after 2 years of WGLC and long journey of revisions to address Areas 
> Directorates comments. Being an AD is not easy, so many drafts to review..
> [Linda] There have been a lot of comments during the Area Directorate 
> reviews, and private comments from NVO3 Chairs.
Indeed, but that doesn't explain the silence at WG LC stage, and is in 
fact a strong incentive for running a new one, and in fact have a 
broader discussion about this draft.

> [Linda] Can you elaborate more on those issues? With English as a second 
> language, I am hoping to learn from the mistakes. NVO3 Chair Matthews 
> had a private session with us pointing English language issues, we had 
> revised draft to correct those issues. We would like to learn moreI'm not a native english speaker/writer, but it is at least easy to 
notice the formatting issue your document has. I can go through the 
document an pick the nits/typos and hard-to-parse sentences but that is 
not the core of the issue.

> [Linda] one of the big issues of VM Mobility in Layer 3 is when the VMs 
> need to maintain their original IP addresses after their move. If VMs 
> addresses can be re-assigned after their move, network becomes very 
> simple but applications running on the VMs can’t be moved freely.
> Layer 2 addresses don’t have hierarchical structure as Layer 3 IP 
> addresses. All switches learn the host Layer 2 addresses on the fly and 
> time out the learned addresses in the Forwarding table when there is no 
> packets has the addresses in the SA/DST fields for certain period of time.
> That is why the draft is focusing on practices and actions when VMs need 
> to maintain their addresses after the move.I understood that. My point is that this document should therefore not 
be called virtual machine mobility.

> /The NVO3 WG will develop solutions for network virtualization based on/
> /the following architectural tenets:/
> /- Support for an IP-based underlay data plane/
That sort of confirms this is not NVO3 specific.

> [Linda] This document provides guidelines to DC operators if they want 
> to maintain VMs IP addresses during VM migration. Being able to move VMs 
> dynamically, from one server to another, makes it possible for dynamic 
> load balancing or work distribution. Therefore, dynamic VM Mobility is 
> highly desirable for large scale multi-tenant DCs
possibly, but if those techniques are already in use, what is the 
benefit of having an RFC describing them? virtual machine mobility seems 
to be fairly well documented already and much more broadly than just 
about IP addressing preservation. It is also very challenging for the 
reader to determine whether all aspects of IP address preservation have 
been covered.

> transfer VM states;
> [Linda] That is why we wrote this draft to indicate that VMs states 
> transferred are needed. But during the numerous reviews, we had been 
> advised to remote the actual protocols of moving states and change the 
> document to be “Informational” (from Standard Track).
I'm ok with that recommendation, because moving states has little to do 
with IP address preservation. The issue I point to is that you refer to 
RFC7666 for more information on moving states, but nothing in RFC7666 
talks about that.

> [Linda] RFC8014 describes the NVE-to-NVA functions.
indeed but your document doesn't say clearly which functions need to be 
activated and how is effectively used the protocol to achieve the 
desired objective.

> the Overview of Section 4.
hmmm, then we can add this to the things that need to be clarified.
Section 7 mas much less information than 4.2/4.3. In fact it is 42 lines 
long while 4.2+4.3 are 142 (and section 4, as a whole, 233). How can the 
overview be longer than the details?

> [Linda] Section 8 and 9 describes what triggers the VMs move. It was 
> suggested from Aera Directorates review comments.
In the intro you say that this document covers planned VM moves.
Section 8 talks about other reasons for a move (failover, load, ...). 
This is confusing and in fact doesn't add anything to IP address 
preservations. Section 9 itself says that it is outside the scope of the 
document.

> [Linda] We are going around in circles.
Not all drafts become RFCs and not all paths to RFC are straight.
I'm not taking a decision here, I'm asking the WG to have a discussion.

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 19 Nov 2020 17:39:08 +0000

Martin, 

Does IETF still have RFC editors who can help correcting English errors? I remember there one many years ago. 

Other comments are inserted below:

-----Original Message-----
From: Martin Vigoureux <[email-masked]> 
Sent: Thursday, November 19, 2020 9:42 AM
To: Linda Dunbar <[email-masked]>; [email-masked]; [email-masked]; [email-masked]
Subject: Re: AD review of draft-ietf-nvo3-vmm-16

Linda,

please see in-line.

Le 2020-11-17 à 20:38, Linda Dunbar a écrit :
> Subject: AD review of draft-ietf-nvo3-vmm-16 Hi, I have several 
> concerns with this document.
Indeed, but that doesn't explain the silence at WG LC stage, and is in fact a strong incentive for running a new one, and in fact have a broader discussion about this draft.
[Linda] Okay. 

> [Linda] Can you elaborate more on those issues? With English as a 
> second language, I am hoping to learn from the mistakes. NVO3 Chair 
> Matthews had a private session with us pointing English language 
> issues, we had revised draft to correct those issues. We would like to 
> learn moreI'm not a native english speaker/writer, but it is at least 
> easy to
notice the formatting issue your document has. I can go through the document an pick the nits/typos and hard-to-parse sentences but that is not the core of the issue.
[Linda] Since your time is much more valuable, is it easier for an RFC editor to help with the English? 

> [Linda] one of the big issues of VM Mobility in Layer 3 is when the 
> VMs need to maintain their original IP addresses after their move. If 
> VMs addresses can be re-assigned after their move, network becomes 
> very simple but applications running on the VMs can’t be moved freely.
> addresses. All switches learn the host Layer 2 addresses on the fly 
> and time out the learned addresses in the Forwarding table when there 
> is no packets has the addresses in the SA/DST fields for certain period of time.
> That is why the draft is focusing on practices and actions when VMs 
> need to maintain their addresses after the move.I understood that. My 
> point is that this document should therefore not
be called virtual machine mobility.
[Linda] Is it better to call the draft "Solutions to preserve IP Addresses during VM mobility in Layer 2 and Layer 3 Networks"?

> /The NVO3 WG will develop solutions for network virtualization based 
> on/ /the following architectural tenets:/
That sort of confirms this is not NVO3 specific.
[Linda] Do you suggest to go through independent RFC stream? 

> centers. Also, it primarily only describes what needs to be done but 
> not how.
> to maintain VMs IP addresses during VM migration. Being able to move 
> VMs dynamically, from one server to another, makes it possible for 
> dynamic load balancing or work distribution. Therefore, dynamic VM 
> Mobility is highly desirable for large scale multi-tenant DCs
possibly, but if those techniques are already in use, what is the benefit of having an RFC describing them? virtual machine mobility seems to be fairly well documented already and much more broadly than just about IP addressing preservation. It is also very challenging for the reader to determine whether all aspects of IP address preservation have been covered.
[Linda] We have aimed to cover all aspects of IP address preservation. If we miss some aspects, can you elaborate? 

> transfer VM states; [Linda] That is why we wrote this draft to 
> indicate that VMs states transferred are needed. But during the 
> numerous reviews, we had been advised to remote the actual protocols 
> of moving states and change the document to be “Informational” (from 
> Standard Track).
I'm ok with that recommendation, because moving states has little to do with IP address preservation. The issue I point to is that you refer to
RFC7666 for more information on moving states, but nothing in RFC7666 talks about that.

> [Linda] RFC8014 describes the NVE-to-NVA functions.
indeed but your document doesn't say clearly which functions need to be activated and how is effectively used the protocol to achieve the desired objective.
[Linda] Thank you for the specific action. we can add. 

hmmm, then we can add this to the things that need to be clarified.
Section 7 mas much less information than 4.2/4.3. In fact it is 42 lines long while 4.2+4.3 are 142 (and section 4, as a whole, 233). How can the overview be longer than the details?
[Linda]Interesting point. Will ask the WG if more needs to be added or removed. 

In the intro you say that this document covers planned VM moves.
Section 8 talks about other reasons for a move (failover, load, ...). 
This is confusing and in fact doesn't add anything to IP address preservations. Section 9 itself says that it is outside the scope of the document.

[Linda] "failover, planned VM moves" are some of the reasons triggering the "VM Moving". 

> benefit of publishing this as an RFC, to discuss whether this should 
> be the product of NVO3 and would an alternate publication stream be 
> more appropriate. In any case this document needs more work.

[Linda] sure.

---

From: Martin Vigoureux <[email-masked]>
Date: Tue, 24 Nov 2020 14:59:27 +0100

Linda,

> [Linda] Is it better to call the draft "Solutions to preserve IP Addresses during VM mobility in Layer 2 and Layer 3 Networks"?
that would work

> [Linda] Do you suggest to go through independent RFC stream?
IMO, if the WG thinks that the content is worth publishing as an RFC and 
considers that this is not specific to NVO3 then the WG should consider 
that option.

> [Linda] We have aimed to cover all aspects of IP address preservation. If we miss some aspects, can you elaborate?
I didn't say you were missing some.
In any case consistently with my previous point, I think the WG needs to 
discuss the added value this brings. It would be great to have feedback 
from DC operators as they are the targetted community of this document.

> [Linda] "failover, planned VM moves" are some of the reasons triggering the "VM Moving".
indeed but again, Section 8 is not clear. It is called "Other options". 
But it is not clear other options to what. Section 8 starts by talking 
about "unexpected events" so that leads me to thinking this is not in 
scope of the document because of the intro which says "planed moves".
But then Section 8 goes on talking about Hot Failover. Do you mean to 
cover in section 8 the "Hot" mobility and in Section 7 the Warm and Cold 
mobility? But Section 7 does have some text on "Hot".
All in all, at the very least, some clarifications are needed.