Subject: [nvo3] Section 4.4 Multi-Homing of NVEs/draft-ietf-nvo3-arch

From: Mingui Zhang <[email-masked]>
Date: Sun, 15 Feb 2015 09:42:45 +0000

Hi authors,

Multi-homing of NVEs means one Tenant System (TS) is attached to multiple NVEs or a single NVE which has more than one underlay IP addresses. When this TS communicates with a TS attached to a remote NVE, this remote NVE will see one TS address being mapped to multiple NVE IP addresses.

When I read the text of Section 4.4, I find there are several places where the description might not be accurate. 

1. "That is, an NVE may have more than one IP address associated with it on the underlay network."

>From your second multi-homing scenario we know that even one NVE has only one IP address, it is also possible for it to offer multi-homing when this NVE is one of the multiple NVEs to which a TS is connected.

2. "In both cases, the NVE address mapping tables need to support one-to-many mappings..."

This is inaccurate because there are ways to avoid installing multiple (TS-Addr->NVE-Addr) mapping per TS-Addr into mapping tables. For example, the NVE can keep the one-to-many mapping at the control plane or management plane while installs an one-to-one mapping into the mapping table. In other words, the NVE need to support one-to-many mapping but it does not have to support it using its _mapping tables_.

3. "...or indeed, having all traffic flow through a single NVE would be considered unacceptable..."

The bare metal server may have multiple uplink connections to this NVE and these connections are implemented using LAG. Hence, we know that even it *is* all traffic flow through a single NVE, it can still be acceptable (desirable).

I suggest we reword the text to fix above issues.

Thanks,
Mingui

---

From: "Black, David" <[email-masked]>
Date: Mon, 16 Feb 2015 23:14:04 +0000

Hi Mingui,

Thanks for the careful read.  Comments inline.

--David

> -----Original Message-----
> From: nvo3 [mailto:[email-masked]] On Behalf Of Mingui Zhang
> Sent: Sunday, February 15, 2015 4:43 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [nvo3] Section 4.4 Multi-Homing of NVEs/draft-ietf-nvo3-arch
> 
> Hi authors,
> 
> Multi-homing of NVEs means one Tenant System (TS) is attached to multiple NVEs
> or a single NVE which has more than one underlay IP addresses. When this TS
> communicates with a TS attached to a remote NVE, this remote NVE will see one
> TS address being mapped to multiple NVE IP addresses.
> 
> >From your second multi-homing scenario we know that even one NVE has only one
> IP address, it is also possible for it to offer multi-homing when this NVE is
> one of the multiple NVEs to which a TS is connected.

Yes, although whether to allow this is something that the WG should discuss
further; if this is allowed, prevention of hybrid loop formation (loop via VN
on nvo3 side of NVEs and non-nvo3 interconnect of NVEs on other side) will need
to be considered.

> This is inaccurate because there are ways to avoid installing multiple (TS-
> Addr->NVE-Addr) mapping per TS-Addr into mapping tables. For example, the NVE
> can keep the one-to-many mapping at the control plane or management plane
> while installs an one-to-one mapping into the mapping table. In other words,
> the NVE need to support one-to-many mapping but it does not have to support it
> using its _mapping tables_.

Ok, the intent was "to support one-to-many mapping" and I understand the
clarification that the table may not always need to do that.  It looks
like "mapping tables" -> "mapping functionality" would address this
concern - is that sufficient?  

> The bare metal server may have multiple uplink connections to this NVE and
> these connections are implemented using LAG. Hence, we know that even it *is*
> all traffic flow through a single NVE, it can still be acceptable (desirable).
> 

Please suggest a specific rewording for this concern.  The complete
sentence is:

   Having only a single physical path to an upstream
   NVE, or indeed, having all traffic flow through a single NVE would be
   considered unacceptable in highly-resilient deployment scenarios that
   seek to avoid single points of failure.

In particular, the expected response to LAG link failure would cause the
traffic involved to switch NVEs; prohibition of that response would result
in a single point of failure.

> Thanks,
> Mingui
> 
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 17 Feb 2015 02:00:07 +0000

Hi David,

>on nvo3 side of NVEs and non-nvo3 interconnect of NVEs on other side) will need

According to my envision, the first (single NVE multiple IPs) and second scenarios (multiple NVEs and each with its IP) are just the same to the hybrid loop formation you mentioned. That is, the "non-nvo3 interconnect of NVEs on the other side" will see the one-to-many mapping of TS-Addr->NVE-IP-Addr and it is required to handle this kind of mapping correctly. 

Actually, the first issue comes into my mind is the MAC flip-flop issue though there may be other kinds of issues, such as duplication, loop, black-hole, etc. But I believe we can design solutions to solve all these issues, as long as the remote NVE can sense all NVEs in a multi-homing group.

>concern - is that sufficient?

Yes, I think that is sufficient. Thanks.

>Please suggest a specific rewording for this concern.  The complete
<snip>

How about we just replace the NVE with VAP in the sentence? That is,

OLD
NEW
   VAP, or indeed, having all traffic flow through a single VAP would be

Of course, we may need update some other sentences if we take this.

>in a single point of failure.

The VAPs connected to one end of the LAG may reside in one NVE or multiple NVEs. In either way, the single failure of a VAP will not fail the whole LAG.

Thanks,
Mingui

>From: Black, David [mailto:[email-masked]]
>Sent: Tuesday, February 17, 2015 7:14 AM
>To: Mingui Zhang; [email-masked]
>Cc: [email-masked]; Black, David
>Subject: RE: Section 4.4 Multi-Homing of NVEs/draft-ietf-nvo3-arch
>
>Hi Mingui,
>
>> 2. "In both cases, the NVE address mapping tables need to support
>one-to-many
>> mappings..."
>>

---

From: "Black, David" <[email-masked]>
Date: Tue, 17 Feb 2015 02:12:45 +0000

Inline ... --David

> Sent: Monday, February 16, 2015 9:00 PM
> To: Black, David; [email-masked]
> Cc: [email-masked]
> Subject: Re: [nvo3] Section 4.4 Multi-Homing of NVEs/draft-ietf-nvo3-arch
> 
> Hi David,
> 
> >on nvo3 side of NVEs and non-nvo3 interconnect of NVEs on other side) will
> >need to be considered.
> 
> According to my envision, the first (single NVE multiple IPs) and second
> scenarios (multiple NVEs and each with its IP) are just the same to the hybrid
> loop formation you mentioned. That is, the "non-nvo3 interconnect of NVEs on
> the other side" will see the one-to-many mapping of TS-Addr->NVE-IP-Addr and
> it is required to handle this kind of mapping correctly.

I don't agree - if any TS is only connected to one NVE, then this sort of loop
prevention can be accomplished by ensuring that traffic received at an NVE
from another NVE is always forwarded to the non-nvo3 side of the NVE and never
onward to another NVE.  The number of IP addresses is irrelevant.

> Actually, the first issue comes into my mind is the MAC flip-flop issue though
> there may be other kinds of issues, such as duplication, loop, black-hole,
> etc. But I believe we can design solutions to solve all these issues, as long
> as the remote NVE can sense all NVEs in a multi-homing group.

And I think the WG gets to decide whether to take on this set of issues.

> Yes, I think that is sufficient. Thanks.

Ok, one down ...

> How about we just replace the NVE with VAP in the sentence? That is,

I don't understand how this would address the concern, because each VAP is
bound to a single NVE - see Section 4.5.  One could have multiple VAPs on
the same NVE, but the sentence is concerned with NVE failure, not just VAP
failure.

> The VAPs connected to one end of the LAG may reside in one NVE or multiple
> NVEs. In either way, the single failure of a VAP will not fail the whole LAG.
> 
> Thanks,
> Mingui
> 
> 
> 
> >> When I read the text of Section 4.4, I find there are several places where
> the
> >> description might not be accurate.
> >>

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 17 Feb 2015 02:46:39 +0000

Hi David,

Inline below with [Mingui].

>Sent: Tuesday, February 17, 2015 10:13 AM

[Mingui] OK. They are the same to part of the issues (e.g., the MAC flip-flop issue). For the loop issue they do have some difference in detail.

[Mingui] For the scenario that one TS is connected to multiple NVEs, the counter measure is quite similar as you described above. We also ensure that traffic received at an NVE from another NVE (in a multi-homing group) always forwarding to the non-nvo3 side of the NVE and never onward to another NVE. Moreover, traffic from received at an NVE1 from another NVE2 (these two NVEs are in the same group) MUST NOT be forwarded to the non-nvo3 side of the NVE1 since the traffic has already been forwarded by NVE2. 

>And I think the WG gets to decide whether to take on this set of issues.

[Mingui] If multi-homing is what we need, then I suggest we take on these issues:).

>the same NVE, but the sentence is concerned with NVE failure, not just VAP

[Mingui] Well, the TS may only concern with the VAP. It is the service point the TS is attached to. We should concern the issue from the angle of TS or from the angle of the network?

[Mingui] NVE failure will definitely lead to VAP failure (s). That means the cases of VAP failures can cover the cases of NVE failures from the angle of TS.

>> >> Multi-homing of NVEs means one Tenant System (TS) is attached to
>multiple
>> >NVEs
>> >> communicates with a TS attached to a remote NVE, this remote NVE will
>see
>> >one
>> >> 1. "That is, an NVE may have more than one IP address associated with it
>on
>> >> the underlay network."
>> >>
>> need
>> >> Addr->NVE-Addr) mapping per TS-Addr into mapping tables. For example,
>the
>> >NVE
>> >> the NVE need to support one-to-many mapping but it does not have to
>support
>> >it
>> >> these connections are implemented using LAG. Hence, we know that even
>it
>> >*is*

---

From: "Black, David" <[email-masked]>
Date: Tue, 17 Feb 2015 03:14:59 +0000

> [Mingui] Well, the TS may only concern with the VAP. It is the service point
> the TS is attached to. We should concern the issue from the angle of TS or
> from the angle of the network?

This is an architecture draft, so (IMHO) the answer is "yes, both" and the network
perspective is definitely important.

Thanks,
--David

> From: Mingui Zhang [mailto:[email-masked]]
> Sent: Monday, February 16, 2015 9:47 PM
> [Mingui] OK. They are the same to part of the issues (e.g., the MAC flip-flop
> issue). For the loop issue they do have some difference in detail.
> 
> [Mingui] For the scenario that one TS is connected to multiple NVEs, the
> counter measure is quite similar as you described above. We also ensure that
> traffic received at an NVE from another NVE (in a multi-homing group) always
> forwarding to the non-nvo3 side of the NVE and never onward to another NVE.
> Moreover, traffic from received at an NVE1 from another NVE2 (these two NVEs
> are in the same group) MUST NOT be forwarded to the non-nvo3 side of the NVE1
> since the traffic has already been forwarded by NVE2.
> 
> 
> >
> [Mingui] Well, the TS may only concern with the VAP. It is the service point
> the TS is attached to. We should concern the issue from the angle of TS or
> from the angle of the network?
> 
> [Mingui] NVE failure will definitely lead to VAP failure (s). That means the
> cases of VAP failures can cover the cases of NVE failures from the angle of
> TS.
> 
> 
> 
> 
> 
> >
> >>
> >> OLD
> >> >> IP address, it is also possible for it to offer multi-homing when this
> NVE
> >> is