Subject: [nvo3] NVO3 Terminology changes

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 01:12:20 +0000

At the NVO3 WG meeting in Orlando, I brought up some suggestions for terminology changes/additions and had an AI to bring this to the list…so here goes.  I would prefer to see constructive suggestions in responses.  In other words, please suggest something better if you object to these.

1) Oracle -> Information Mapping Authority

Stewart mentioned that he has copyright concerns with using the term "oracle", and others have expressed distaste as well.  In draft-kreeger-nvo3-hypervisor-nve-cp-01 we replaced the term with "Information Mapping Authority" (IMA).  We would like to get consensus on using this new term in all WG documents going forward.  In the meeting Linda  expressed a concern that IMA might get confused with IMA being confused with the acronym for Inverse Multiplexing for ATM, and suggested something like "Directory Service" to which David Black replied that she might have trouble convincing people that BGP can be categorized as a "Directory Service".

2) VNIC -> Tenant System Interface

The term VNIC is actually used in the framework document, but never defined.  In kreeger-nvo3-hypervisor-nve-cp-01 we defined a VNIC as "A Virtual NIC that connects a Tenant System to a Virtual Network Instance (VNI)."  In NVO3 (myself included) we often use VM when we are talking about "Tenant Systems" and talk about VMs connecting to a VNI; However, a VM can actually connect to multiple VNIs through multiple VNICs…but VNICs are very specific to Virtual Machines.  If we are to use the more correct "Tenant System" instead of VM, we should use a more generic term for the interface on the tenant system itself than VNIC.  We have suggested using "Tenant System Interface" (TSI) for this, which we would like to see formally defined in the Framework document and shown to correspond with VAPs within the NVE.

Looking forward to your feedback, Larry

---

From: Qin Wu <[email-masked]>
Date: Tue, 9 Apr 2013 03:20:55 +0000

[Qin]: Looks good to me, also you may change Oracle into Oracle backend System.

The term VNIC is actually used in the framework document, but never defined.  In kreeger-nvo3-hypervisor-nve-cp-01 we defined a VNIC as "A Virtual NIC that connects a Tenant System to a Virtual Network Instance (VNI)."  In NVO3 (myself included) we often use VM when we are talking about "Tenant Systems" and talk about VMs connecting to a VNI; However, a VM can actually connect to multiple VNIs through multiple VNICs...but VNICs are very specific to Virtual Machines.  If we are to use the more correct "Tenant System" instead of VM, we should use a more generic term for the interface on the tenant system itself than VNIC.  We have suggested using "Tenant System Interface" (TSI) for this, which we would like to see formally defined in the Framework document and shown to correspond with VAPs within the NVE.

[Qin]: Can Tenant System interface be a physical interface? If not, I suggest to change Tenant System virtual interface.
I believe one tenant system can host multiple VMs, each VM may have multiple vNIC adapters that it uses to communicate with both the virtual and physical networks.
So VM can use multiple vNIC to connect to multiple VN. When one vNIC are assigned with multiple IP addresses and a single MAC addess, each vNIC can use multiple IP address to connect to multiple VN.

Regarding vNIC -> Tenant System interface, I am not sure we have to replace vNIC with Tenant System interface since vNIC and Tenant System interface seems two different things and can be mapped in
1 to 1 relation. So I think both term can be used and how they are related to each other can be clarified when needed.

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 03:35:50 +0000

Hi Qin,

My responses are below marked with LK>.  - Larry

From: Qin Wu <[email-masked]<mailto:[email-masked]>>
Date: Monday, April 8, 2013 8:20 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: NVO3 Terminology changes

LK> I see no reason why a Tenant System Interface must be virtual (although it is quite likely) - the definition in the framework for a tenant system says "A physical or  virtual system…" .  I don't see that adding the word "virtual" helps.

LK> A VM is one example of a tenant system…so it would not host VMs.  You may be thinking of "End Device".

LK> I agree that there is no need to globally replace VNIC with TSI, just as there is no need to globally replace VM with TS.  A VM (with a VNIC) is just one common example of a TS (with a TSI).  However, when being general the TS/TSI terminology is clearer in that it covers all possible cases, not just the common VM/VNIC case.

---

From: Qin Wu <[email-masked]>
Date: Tue, 9 Apr 2013 03:55:36 +0000

Hi, Larry:
Please see my followup comments below.

Regards!
-Qin
发件人: Larry Kreeger (kreeger) [mailto:[email-masked]]
发送时间: 2013年4月9日 11:36
收件人: Qin Wu; [email-masked]
主题: Re: NVO3 Terminology changes

Hi Qin,

>Stewart mentioned that he has copyright concerns with using the term "oracle", and others have expressed distaste as well.  In draft-kreeger-nvo3-hypervisor-nve-cp-01 we replaced the term with "Information >Mapping Authority" (IMA).  We would like to get consensus on using this new term in all WG documents going forward.  In the meeting Linda  expressed a concern that IMA might get confused with IMA >being confused with the acronym for Inverse Multiplexing for ATM, and suggested something like "Directory Service" to which David Black replied that she might have trouble convincing people that BGP can be >categorized as a "Directory Service".

>The term VNIC is actually used in the framework document, but never defined.  In kreeger-nvo3-hypervisor-nve-cp-01 we defined a VNIC as "A Virtual NIC that connects a Tenant System to a Virtual Network Instance
> (VNI)."  In NVO3 (myself included) we often use VM when we are talking about "Tenant Systems" and talk about VMs connecting to a VNI; However, a VM can actually connect to multiple VNIs through multiple
>VNICs…but VNICs are very specific to Virtual Machines.  If we are to use the more correct "Tenant System" instead of VM, we should use a more generic term for the interface on the tenant system itself than
>VNIC.  We have suggested using "Tenant System Interface" (TSI) for this, which we would like to see formally defined in the Framework document and shown to correspond with VAPs within the NVE.

[Qin]: In that case, vNIC is not equivalent to Tenant System interface since tenant system interface can be either physical interface or virtual interface.
The reason I propose such change is vNIC is virtual NIC not physical NIC, therefore if you replace vNIC with Tenant system interface, that means Tenant system interface only corresponds to virtual interfacel.

[Qin]: Not sure about that, the definition of “Tenant system” in Framework said:
“
       Tenant System: A physical or virtual system that can play the role
       of a host, or a forwarding element such as a router, switch,
       firewall, etc. It belongs to a single tenant and connects to one or
       more VNs of that tenant.
”
So tenant system can be a host and host one or multiple VMs on it. What am I missing?

[Qin]: Agree, my interpretation of common case you mentioned is just a simple case.  For the case where VM  has multiple vNIC, you still can tread each vNIC as a VM that only has one  vNIC and so such case is still a simple case.  The only complicated case is one vNIC connect to Multiple VN using multiple IP addresses.

---

From: "Pat Thaler" <[email-masked]>
Date: Tue, 9 Apr 2013 17:38:46 +0000

My responses marked with my initials. [pat] Pat

From: [email-masked] [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Monday, April 08, 2013 8:56 PM
To: Larry Kreeger (kreeger); [email-masked]
Subject: [nvo3] 答复: NVO3 Terminology changes

Hi, Larry:

Regards!
-Qin
发件人: Larry Kreeger (kreeger) [mailto:[email-masked]]
发送时间: 2013年4月9日 11:36
收件人: Qin Wu; [email-masked]<mailto:[email-masked]>
主题: Re: NVO3 Terminology changes

Hi Qin,

> [Qin]: Looks good to me, also you may change Oracle into Oracle backend System.
[pat] Can it be Mapping Authority? What does Information add? I’m not concerned about the acronym overlapping with an ATM acronym – but I prefer a shorter name.

[pat] Even if it was always going to be virtual, Tenant System Interface is a clear and distinct name – it’s a name, not a full description. But vNIC was probably inconsistent as we came to an understanding that the interface could be virtual or physical. There is no reason to restrict the tenant interfaces to being virtual.

[pat] A tenant system doesn’t have to have anything virtual about it.  A tenant system can be a physical system with no VMs – just a plain old system connected to the network through an NVE.

>[Qin]So VM can use multiple vNIC to connect to multiple VN. When one vNIC are assigned with multiple IP addresses and a single MAC addess, each vNIC can use multiple IP address to connect to multiple VN.

[pat] I think that we could choose to define a TSI as having a list of associated IP addresses or as always having a single IP address. In the latter case, a vNIC with multiple IP addresses would be modeled as multiple TSIs.

---

From: Sharon <[email-masked]>
Date: Tue, 9 Apr 2013 11:58:24 -0700

Just a light content comment (summary does reflect the meeting content in my view)

Seems perfectly logical that an Overlay will need a method of sharing global information per entities "beyond" the Underlay, and that it's "Oracle-Like" mechanism e.g you ask it questions and get answers .. even in Delphi the Oracle didn't pre guess questions ... 

.. Also mapping /IMA seems a perfectly reasonable reduction of what this service offers, and distributed directory captures how it offers it - after all if there is an Overlay there is also an Underlay - a perfectly functioning network to anchor such s mechanism ..

What's not clear is why insisting on BGP if there is a clear claim BGP is not a directory oriented protocol ? 

Sent from my iPhone 650 492 0794

On Apr 8, 2013, at 6:12 PM, "Larry Kreeger (kreeger)" <[email-masked]> wrote:

> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: "Black, David" <[email-masked]>
Date: Tue, 9 Apr 2013 17:37:04 -0400

I believe that BGP is among the possible ways to implement this service.
I think that “directory” is not a good characterization of BGP.
Hence “directory” does not appear to be a good word to use as part of the replacement of the “Oracle” term.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Sharon
Sent: Tuesday, April 09, 2013 2:58 PM
To: Larry Kreeger (kreeger)
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

On Apr 8, 2013, at 6:12 PM, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>> wrote:
At the NVO3 WG meeting in Orlando, I brought up some suggestions for terminology changes/additions and had an AI to bring this to the list…so here goes.  I would prefer to see constructive suggestions in responses.  In other words, please suggest something better if you object to these.

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: "Luyuan Fang (lufang)" <[email-masked]>
Date: Tue, 9 Apr 2013 21:45:11 +0000

Agree.
The term should be general enough to cover all cases.
Luyuan

From: <Black>, David <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 9, 2013 5:37 PM
To: Sharon <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] NVO3 Terminology changes

Thanks,
--David

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Sharon
Sent: Tuesday, April 09, 2013 2:58 PM
To: Larry Kreeger (kreeger)
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] NVO3 Terminology changes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 21:45:35 +0000

Followups with LK2> - Larry

From: Qin Wu <[email-masked]<mailto:[email-masked]>>
Date: Monday, April 8, 2013 8:55 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: 答复: NVO3 Terminology changes

Hi, Larry:

Regards!
-Qin

Hi Qin,

LK2> Again, I don't want to replace all instances of VNIC with TSI, but use the more general term TSI when it is not a specific example of a TS that is a Virtual Machine.  So, a VNIC is one type of TSI, but all TSIs are not VNICs.

LK2> I think that you are assuming that "host" is synonymous with "Hypervisor".  In the definition above, I believe the term host relates to the more traditional definition of an internet host such as in RFC 1122.

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 21:55:21 +0000

Hi Pat,

My responses are inline, marked as LK2>. - Larry

From: Pat Thaler <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 9, 2013 10:38 AM
To: Qin Wu <[email-masked]<mailto:[email-masked]>>, Larry Kreeger <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: NVO3 Terminology changes

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Monday, April 08, 2013 8:56 PM
To: Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Subject: [nvo3] 答复: NVO3 Terminology changes

Hi, Larry:

Regards!
-Qin

Hi Qin,

LK2> I would not object to that.  We had used the word "Information" to replace the word "Address" from a different suggestion for the term "Address Mapping Authority" since we felt the oracle would hold more than just address mappings.

LK2> I see advantages to packaging all the MAC and IP addresses for a TSI together when they share a common root, such as the example of a VNIC that has one MAC but may have multiple IPs on it.  It saves on coming up with a unique TSI ID and repeating the MAC for each new IP address.  It also allows disconnect/migration messages to contain only the one TSI ID regardless of how many IP addresses there are.

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 18:18:45 -0400

On 4/8/2013 9:12 PM, Larry Kreeger (kreeger) wrote:
> At the NVO3 WG meeting in Orlando, I brought up some suggestions for
> terminology changes/additions and had an AI to bring this to the list…so
> here goes.  I would prefer to see constructive suggestions in responses.
>  In other words, please suggest something better if you object to these.
> 
> Stewart mentioned that he has copyright concerns with using the term
> "oracle", and others have expressed distaste as well.
>  In draft-kreeger-nvo3-hypervisor-nve-cp-01 we replaced the term with
> "Information Mapping Authority" (IMA).  We would like to get consensus
> on using this new term in all WG documents going forward.  In the
> meeting Linda  expressed a concern that IMA might get confused with IMA
> being confused with the acronym for Inverse Multiplexing for ATM, and
> suggested something like "Directory Service" to which David Black
> replied that she might have trouble convincing people that BGP can be
> categorized as a "Directory Service".
> 

Perhaps we're over thinking this one.  We have NVE, so why not
NV Controller (NVC) or NV Orchestrator (NVO)?

Lou

>...

---

From: Lucy yong <[email-masked]>
Date: Tue, 9 Apr 2013 22:24:15 +0000

LK2> I would not object to that.  We had used the word "Information" to replace the word "Address" from a different suggestion for the term "Address Mapping Authority" since we felt the oracle would hold more than just address mappings.
[Lucy] For NVO3, this is for address mapping authority. Although the authority process may require holding other information, the result is the address mapping. IMO: the term of Address Mapping Authority (AMA) is fine although the sound likes IMA.

Lucy

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 23:25:03 +0000

Lou,

Do you see the terms "Controller" or "Orchestrator" standing up to David
Black's "BGP" litmus test?

 - Larry

On 4/9/13 3:18 PM, "Lou Berger" <[email-masked]> wrote:

>> terminology changes/additions and had an AI to bring this to the listŠso

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 9 Apr 2013 23:28:01 +0000

Hi Lucy,

So, would you consider the VN-Context to be an address?  Others have also suggested that the oracle may contain policy information as well, which I would not consider an address mapping.

 - Larry

From: Lucy yong <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 9, 2013 3:24 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, Pat Thaler <[email-masked]<mailto:[email-masked]>>, Qin Wu <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] NVO3 Terminology changes

Lucy

---

From: Sharon <[email-masked]>
Date: Tue, 9 Apr 2013 16:45:43 -0700

Policies are typically mapped to addressable identities so a bit off but not totally .. this does raise that litmus question again .. take a policy say ACL mapped to addressable identity .. this needs a lookup service over a very large flat space the kind you typically use directories and databases for .. so why isn't this metric /  benchmark the litmus test rather then the protocol?

On Apr 9, 2013, at 4:28 PM, "Larry Kreeger (kreeger)" <[email-masked]> wrote:

> From: Lucy yong <[email-masked]>
> To: Larry Kreeger <[email-masked]>, Pat Thaler <[email-masked]>, Qin Wu <[email-masked]>, "[email-masked]" <[email-masked]>

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 19:51:58 -0400

Larry,
	I think the NVC and NVO terms are fairly solution neutral, i.e., I
don't see why not.

I guess your question does lead to yet another option
NVS: Network Virtualization Service

I personally like NVC, but given that we have NVEs, I'd be happy with
anything of the form "Network Virtualization <foo>" as long as <foo>
doesn't start with an "E".

Lou

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 00:10:36 +0000

Hi Sharon,

The goal is to have a name for a "logically centralized" entity (oracle) that can carry mapping information for the NVEs, such as VN-Name to VN-Context, Overlay (inner) Address to Underlay (outer) Address.  People start getting uncomfortable when they see a name that implies a particular interaction model between the NVE and the oracle.  I think it all comes down to push vs pull and the scope of how much information is pushed/pulled to the NVE and when.

To me a Directory Service implies a directory lookup (pull), where a Controller/Orchestrator implies an intelligent push to only the NVEs that need it.  An "authority" does not imply either model to me (similar to oracle).  So, for me the litmus test is whether the name implies push or pull.  I think David brought up BGP as an example of a protocol that is more of a push and therefore doesn't fit into the pull model implied by a directory, nor in my opinion the added intelligence of a controller/orchestrator.

 - Larry

From: Sharon <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 9, 2013 4:45 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>
Cc: Pat Thaler <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Qin Wu <[email-masked]<mailto:[email-masked]>>, Lucy yong <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] NVO3 Terminology changes

On Apr 9, 2013, at 4:28 PM, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>> wrote:

Hi Lucy,

 - Larry

Lucy

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 00:27:35 +0000

Lou,

Given your criteria for starting with NV, we could potentially have one of
the following:

NVS - Network Virtualization Service (service seems vague and perhaps an
overloaded term)
NVA - Network Virtualization Authority (may be vague)
NVMA - Network Virtualization Mapping Authority (less vague)
NVO - Network Virtualization Oracle (assuming oracle is OK when it has the
NV prefix)

 - Larry

---

From: Qin Wu <[email-masked]>
Date: Wed, 10 Apr 2013 00:40:10 +0000

That is the option I have thought about. I used this term (i.e.,NVE controller) in my presentation of Orlando meeting.

Regards!
-Qin
-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表 Lou Berger
发送时间: 2013年4月10日 6:19
收件人: Larry Kreeger (kreeger)
抄送: [email-masked]
主题: Re: [nvo3] NVO3 Terminology changes

Lou

---

From: Pankaj Garg <[email-masked]>
Date: Wed, 10 Apr 2013 00:39:27 +0000

The suggestion by Lou about prefixing the term with NV sounds like a good idea. I agree that a directory service implies pull whereas controller implies push model. We should try and select a term that is independent of the underlying implementation.

How about "Network Virtualization Policy Authority"?

The reason I suggest the word policy is because the "oracle" may store information beyond address mapping such as routing, and potentially even filtering information such as ACL.

Thanks,
Pankaj

From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Tuesday, April 09, 2013 5:11 PM
To: Sharon
Cc: Pat Thaler; [email-masked]; Qin Wu; Lucy yong
Subject: Re: [nvo3] NVO3 Terminology changes

Hi Sharon,

 - Larry

 - Larry

Lucy

>VNICs...but VNICs are very specific to Virtual Machines.  If we are to use the more correct "Tenant System" instead of VM, we should use a more generic term for the interface on the tenant system itself than

>[Qin]: Can Tenant System interface be a physical interface? If not, I suggest to change Tenant System virtual interface.

LK> I see no reason why a Tenant System Interface must be virtual (although it is quite likely) - the definition in the framework for a tenant system says "A physical or  virtual system..." .  I don't see that adding the word "virtual" helps.

[pat] Even if it was always going to be virtual, Tenant System Interface is a clear and distinct name - it's a name, not a full description. But vNIC was probably inconsistent as we came to an understanding that the interface could be virtual or physical. There is no reason to restrict the tenant interfaces to being virtual.

>I believe one tenant system can host multiple VMs, each VM may have multiple vNIC adapters that it uses to communicate with both the virtual and physical networks.

LK> A VM is one example of a tenant system...so it would not host VMs.  You may be thinking of "End Device".
_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Sharon <[email-masked]>
Date: Tue, 9 Apr 2013 17:51:05 -0700

Hey Larry,ok, got it. These paragraphs make perfect sense .. to me at least :)

A logically centralized (physically distributed ?) entity that carries any mapping .. using any resolution model push, pull, and push-pull (pubsub) depending on the mapping space, identity space, change rate, coherency, latency, acid levels .. 
(macs, acls, vips, nats, ucasts, mcasts healths ..)
sounds good .. and hope this discussion will yield the best name for this Map job, it's a key piece in many people's view.

On Apr 9, 2013, at 5:10 PM, "Larry Kreeger (kreeger)" <[email-masked]> wrote:

> From: Sharon <[email-masked]>
> To: Larry Kreeger <[email-masked]>
> Cc: Pat Thaler <[email-masked]>, "[email-masked]" <[email-masked]>, Qin Wu <[email-masked]>, Lucy yong <[email-masked]>

---

From: Qin Wu <[email-masked]>
Date: Wed, 10 Apr 2013 00:51:22 +0000

[Qin Wu]: Agree. That is what I am concerning about.

[Qin]: Agree, the definition of tenant system has already mentioned that a tenant system could play the role of forwarding element besides play the role of host, that means a tenant system could play both virtual system role and physical system role.

[Qin]: Exactly, that is also my understanding.

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 20:57:28 -0400

> where a
> Controller/Orchestrator implies an intelligent push to only the NVEs
> that need it.

Humm, where does this implication come from?  I don't see that connotation.

Do you think Server implies one model over the other?

Lou

On 4/9/2013 8:10 PM, Larry Kreeger (kreeger) wrote:
> Hi Sharon,
> 
> The goal is to have a name for a "logically centralized" entity (oracle)
> that can carry mapping information for the NVEs, such as VN-Name to
> VN-Context, Overlay (inner) Address to Underlay (outer) Address.  People
> start getting uncomfortable when they see a name that implies a
> particular interaction model between the NVE and the oracle.  I think it
> all comes down to push vs pull and the scope of how much information is
> pushed/pulled to the NVE and when.
> 
> To me a Directory Service implies a directory lookup (pull), where a
> Controller/Orchestrator implies an intelligent push to only the NVEs
> that need it.  An "authority" does not imply either model to me (similar
> to oracle).  So, for me the litmus test is whether the name implies push
> or pull.  I think David brought up BGP as an example of a protocol that
> is more of a push and therefore doesn't fit into the pull model implied
> by a directory, nor in my opinion the added intelligence of a
> controller/orchestrator.
> 
>  - Larry
> 
> From: Sharon <[email-masked] <mailto:[email-masked]>>
> To: Larry Kreeger <[email-masked] <mailto:[email-masked]>>
> Cc: Pat Thaler <[email-masked] <mailto:[email-masked]>>,
> "[email-masked] <mailto:[email-masked]>" <[email-masked]
> <mailto:[email-masked]>>, Qin Wu <[email-masked]
> <mailto:[email-masked]>>, Lucy yong <[email-masked]
> <mailto:[email-masked]>>
> Policies are typically mapped to addressable identities so a bit off but
> not totally .. this does raise that litmus question again .. take a
> policy say ACL mapped to addressable identity .. this needs a lookup
> service over a very large flat space the kind you typically use
> directories and databases for .. so why isn't this metric /  benchmark
> the litmus test rather then the protocol?
> 
>> So, would you consider the VN-Context to be an address?  Others have
>> also suggested that the oracle may contain policy information as well,
>> which I would not consider an address mapping.
>>
>>  - Larry
>>
>> From: Lucy yong <[email-masked] <mailto:[email-masked]>>
>> To: Larry Kreeger <[email-masked] <mailto:[email-masked]>>, Pat
>> Thaler <[email-masked] <mailto:[email-masked]>>, Qin Wu
>> <[email-masked] <mailto:[email-masked]>>, "[email-masked]
>> <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>> LK2> I would not object to that.  We had used the word "Information"
>> to replace the word "Address" from a different suggestion for the term
>> "Address Mapping Authority" since we felt the oracle would hold more
>> than just address mappings.
>>
>> */[Lucy] For NVO3, this is for address mapping authority. Although the
>> authority process may require holding other information, the result is
>> the address mapping. IMO: the term of Address Mapping Authority (AMA)
>> is fine although the sound likes IMA./*
>>
>> */ /*
>>
>> */Lucy/*
>>
>>  
>>
>> >The term VNIC is actually used in the framework document, but never
>> defined.  In kreeger-nvo3-hypervisor-nve-cp-01 we defined a VNIC as "A
>> Virtual NIC that connects a Tenant System to a Virtual Network Instance
>>
>> >(VNI)."  In NVO3 (myself included) we often use VM when we are talking
>> about "Tenant Systems" and talk about VMs connecting to a VNI;
>> However, a VM can actually connect to multiple VNIs through multiple
>>
>> >VNICs…but VNICs are very specific to Virtual Machines.  If we are to
>> use the more correct "Tenant System" instead of VM, we should use a
>> more generic term for the interface on the tenant system itself than
>>
>> >VNIC.  We have suggested using "Tenant System Interface" (TSI) for
>> this, which we would like to see formally defined in the Framework
>> document and shown to correspond with VAPs within the NVE.
>>
>>  
>>
>> LK> I see no reason why a Tenant System Interface must be virtual
>> (although it is quite likely) - the definition in the framework for a
>> tenant system says "A physical or  virtual system…" .  I don't see
>> that adding the word "virtual" helps.
>>
>>  
>>
>> [Qin]: In that case, vNIC is not equivalent to Tenant System interface
>> since tenant system interface can be either physical interface or
>> virtual interface.
>>
>> The reason I propose such change is vNIC is virtual NIC not physical
>> NIC, therefore if you replace vNIC with Tenant system interface, that
>> means Tenant system interface only corresponds to virtual interfacel.
>>
>>  
>>
>> [pat] Even if it was always going to be virtual, Tenant System
>> Interface is a clear and distinct name – it’s a name, not a full
>> description. But vNIC was probably inconsistent as we came to an
>> understanding that the interface could be virtual or physical. There
>> is no reason to restrict the tenant interfaces to being virtual.
>>
>>  
>>
>> [email-masked] <mailto:[email-masked]>

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 01:03:26 +0000

>Humm, where does this implication come from?  I don't see that

LK> The immediate example that comes to mind when I hear "network
controller" is OpenFlow

>Do you think Server implies one model over the other?

LK> No

>>> >VNICsŠbut VNICs are very specific to Virtual Machines.  If we are to
>>> >[Qin]: Can Tenant System interface be a physical interface? If not, I
>>>suggest to change Tenant System virtual interface.
>>>
>>>  
>>>
>>> tenant system says "A physical or  virtual systemŠ" .  I don't see
>>> Interface is a clear and distinct name  it¹s a name, not a full
>>> >I believe one tenant system can host multiple VMs, each VM may have
>>>multiple vNIC adapters that it uses to communicate with both the
>>>virtual and physical networks.
>>>
>>>  
>>>
>>> LK> A VM is one example of a tenant systemŠso it would not host VMs.
>>>  You may be thinking of "End Device".
>>>

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 21:04:33 -0400

Also...

NVS - Network Virtualization Server
NVC - Network Virtualization Controller
NVO - Network Virtualization Orchestrator

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 21:10:41 -0400

> LK> The immediate example that comes to mind when I hear "network

We've had controllers for a long time.  I suspect that if we worry about
every possible connotation (or similar trade name), we'll never agree on
anything.

BTW not that it really matters but "mapping" makes me think of DNS and
"oracle" = the obvious ;-)

Lou

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 01:35:04 +0000

>every possible connotation (or similar trade name), we'll never agree on

Hey, if others agree with NV Controller, I'd love to move past this! -
Larry

>>>> On Apr 9, 2013, at 4:28 PM, "Larry Kreeger (kreeger)"
>>>><[email-masked]
>>>> <mailto:[email-masked]>> wrote:
>>>>
>>>>> Hi Lucy,
>>>>>

---

From: Sharon <[email-masked]>
Date: Tue, 9 Apr 2013 18:37:04 -0700

Have to say that on this aspect I get Larry's and David's point. 

A server or controller turfs  the problem elsewhere .. a magical controller that knows everything there is to know and consequently everything there is to do everywhere in the network.. 

A very non IP, almost theological view of network design.

.. where as mapping owns up to the problem. Push/Pull or PubSub ALL you get at the NVO edges is just MMap. Now work with that.

---

From: Lou Berger <[email-masked]>
Date: Tue, 09 Apr 2013 22:15:46 -0400

> A very non IP, almost theological view of network design.

guess you never came across a route server...

> .. where as mapping owns up to the problem. Push/Pull or PubSub ALL you get at the NVO edges is just MMap. Now work with that.  

The composition of this WG is certainly interesting.  Clearly lots of
different perspectives.  For example, you say mmap and I hear memory
mapped I/O!

Lou

---

From: Melinda Shore <[email-masked]>
Date: Tue, 09 Apr 2013 18:19:18 -0800

> Hey, if others agree with NV Controller, I'd love to move past this! -

Yes, please (oh please oh please).

I'd like to avoid collision with product namespace.  Other than
that I'd be fine with anything up to and including calling it
a "pony."  "NV controller" is fine.

Melinda

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 10 Apr 2013 02:52:51 +0000

> -----邮件原件-----
> 发件人: [email-masked] [mailto:[email-masked]] 代表 Larry
> Kreeger (kreeger)
> 发送时间: 2013年4月10日 9:35
> 收件人: Lou Berger
> 抄送: [email-masked]
> Hey, if others agree with NV Controller, I'd love to move past this! 

Why use the "controller" term? Because it's a fashionable word? If so, why not "SDN controller":)  

Personally I think the term "Address Mapping Service" or "Address Mapping Authority" is fine to represent the mapping service of Overlay (inner) Address to Underlay (outer) Address. As for other mapping services like VN-Name to VN-Context, VN-Context to ACL etc, it'd better to use separate names for each of them, especially when considering the fact whether each of them is a must for NVO3 is not sure.

Best regards,
Xiaohu

---

From: "Luyuan Fang (lufang)" <[email-masked]>
Date: Wed, 10 Apr 2013 02:56:34 +0000

NV Controller sounds fine to me.
Luyuan

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 03:16:36 +0000

On 4/9/13 7:52 PM, "Xuxiaohu" <[email-masked]> wrote:

>Personally I think the term "Address Mapping Service" or "Address Mapping
>Authority" is fine to represent the mapping service of Overlay (inner)
>Address to Underlay (outer) Address. As for other mapping services like
>VN-Name to VN-Context, VN-Context to ACL etc, it'd better to use separate
>names for each of them, especially when considering the fact whether each
>of them is a must for NVO3 is not sure.

Having separate entities for every type of information mapping seems like
overkill to me.  I think we should keep it simple and have only one for
now.  If a particular mapping is not required, it doesn't have to be
implemented, but I don't see why we need a different name for each mapping
type.

>> >>>> particular interaction model between the NVE and the oracle.  I
>>think
>> >>>>it
>> >>>> all comes down to push vs pull and the scope of how much
>>information
>> >>>>is
>> >>>> Policies are typically mapped to addressable identities so a bit
>>off
>> >>>>but
>> >>>>> authority process may require holding other information, the
>>result
>> >>>>>is
>> >>>>> defined.  In kreeger-nvo3-hypervisor-nve-cp-01 we defined a VNIC
>>as
>> >>>>>"A
>> >>>>>> [Qin]: Can Tenant System interface be a physical interface? If
>>not,
>> >>>>>>I

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 10 Apr 2013 03:36:31 +0000

> 发送时间: 2013年4月10日 11:17
> 收件人: Xuxiaohu; Lou Berger
> 抄送: [email-masked]

How about using "Tenant/VN Profile Authority" or "Tenant/VN Policy Authority" to present all other information service (e.g., ACL, policy, NV-Name to NV-Context mapping...) than the addressing mapping service? Since address mapping service is a key service, it deserves a separate and clear term, rather than a too generic term which mix all different things together, IMHO. In the latter case, you would have to clarify what's the difference between this NV controller and other controllers (e.g., OF controller and SDN controller).

Best regards,
Xiaohu

> >> >>>>> To: Larry Kreeger <[email-masked]
> <mailto:[email-masked]>>,
> >> Pat

---

From: Truman Boyes <[email-masked]>
Date: Wed, 10 Apr 2013 00:47:54 -0400

Hi,

I think the below two suggestions provide the clearest terminology and description for the function:

BGP might not be a mapping service; but with a stretch of the imagination the mapping of NLRI to prefix/communities/etc could fulfil the litmus test. 

The below words of 'server', 'controller', and 'orchestrator' carry a lot of baggage/meaning at the moment and might convey other meanings. I like the NVMA as a functional element.

NVMA +1

Truman

---

From: Sharon <[email-masked]>
Date: Tue, 9 Apr 2013 21:59:36 -0700

NVMA +1

---

From: Pankaj Garg <[email-masked]>
Date: Wed, 10 Apr 2013 06:33:54 +0000

I re-suggest :) NVPA - Network Virtualization Policy Authority

Mapping feels like overloading a specific term for a generic use case.

Pankaj
________________________________
From: Sharon<mailto:[email-masked]>
Sent: ‎4/‎9/‎2013 10:00 PM
To: Truman Boyes<mailto:[email-masked]>
Cc: Lou Berger<mailto:[email-masked]>; Larry Kreeger (kreeger)<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] NVO3 Terminology changes

NVMA +1

On Apr 9, 2013, at 9:47 PM, Truman Boyes <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

NVMA +1

Truman

On 9 Apr, 2013, at 9:04 PM, Lou Berger <[email-masked]<mailto:[email-masked]>> wrote:

Also...

- Larry

On 4/9/13 4:51 PM, "Lou Berger" <[email-masked]<mailto:[email-masked]>> wrote:

Larry,

Lou

- Larry

On 4/9/13 3:18 PM, "Lou Berger" <[email-masked]<mailto:[email-masked]>> wrote:

On 4/8/2013 9:12 PM, Larry Kreeger (kreeger) wrote:
At the NVO3 WG meeting in Orlando, I brought up some suggestions for
terminology changes/additions and had an AI to bring this to the
list?so
here goes.  I would prefer to see constructive suggestions in
responses.
In other words, please suggest something better if you object to
these.

meeting Linda  expressed a concern that IMA might get confused with
IMA

Lou

...

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Wed, 10 Apr 2013 07:37:33 +0000

I'd suggest to keep it simple, as Lou suggested.

Hence NVC is my favorite. Obviously, whichever acronym gets picked shall carry in its definition the mapping function.

Marc

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Sharon
Sent: Wednesday, April 10, 2013 7:00 AM
To: Truman Boyes
Cc: Lou Berger; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

NVMA +1

Hi,

NVMA +1

Truman

Also...

- Larry

Larry,

Lou

- Larry

listŠso

Lou

...

---

From: Jon Hudson <[email-masked]>
Date: Wed, 10 Apr 2013 01:09:41 -0700

My opinion: Either the words the acronym is representing need to be descriptive & clear. Which I think this one does well.

NVMA - Network Virtualization Mapping Authority 

Or it needs to be a nice acronym that rolls easily of the tongue and so will be freely accepted into techlish. I think this one could fill that nicely.

NVO - Network Virtualization Orchestrator/Oracle 

For a combination of both goals maybe something like this

OMA - Overlay Mapping Authority 

Or keep it simple with a beloved tech culture reference ;-)

MCP - Mapping Control Program

End of line.

---

From: "Reith, Lothar" <[email-masked]>
Date: Wed, 10 Apr 2013 10:49:54 +0000

What about the following naming proposal:

-          Virtual Network Endpoint Reachability Information Distribution Authority (VNERIDA)

It is a functional description  - as least AFAIK.
It leaves it open if the information distribution model is push or pull or mixed push&pull, uses BGP or DNS or LISP or you name it.
The pronounciation of the acronym is internationally fairly unambiguous, and it can be fairly easily pronounced.

It does not use the term “network virtualization mapping” which is IMHO too vague, as neither networks nor virtualizations get mapped, rather the overlay address of a network endpoint in a virtual network gets mapped to a network endpoint identity belonging to a network endpoint in an underlay network.  Please note that there may be a requirement for a second mapping if said underlay network endpoint identity is not a locator type routable address.  In this case, this unterlay network endpoint identity must be mapped to a routable underlay network endpoint identity – presumably an IP address.

It emphasizes that this is all about network endpoints which are vertices in a network according to mathematical graph theory. Most other names proposed so far have been given the – mathematically speaking – wrong  impression, that this “so far nameless concept” may have anything to do with the arcs in a graph theory network.

But maybe I am wrong. May I ask that those please speak up who believe that  there is indeed a requirement that this  “so far nameless concept” should  indeed also be responsible for the distribution of  information on arcs, i.e. on underlay network topology changes which do not affect the location of an overlay network endpoint.

And while I am at it please note my opinion on the proposal to rename the term “VNIC”

This is a No-Go. I would strongly recommend to leave the term VNIC as is. The compute centric colleagues from “the other side” of virtualization have defined it to be able to abstract it as the virtual equivalent of a physical NIC in a virtual mashine, allowing all things possible with physical NICs in a server (including NIC-Bonding = LAG) to be done with virtual NICs of a VM.  We simply do not have the authority to change the meaning of that, because it may break virtualization concepts in the datacenter.

But we should define the tem VNIC from the network centric side.

My proposal:
A VNIC is a virtual network endpoint. One physical NIC may have multiple VNICs associated to it, each with its own MAC-Layer.

This definition also closes the loop to the network endpoint centric terminology proposal I made in the first part of this mail.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Jon Hudson
Gesendet: Mittwoch, 10. April 2013 10:10
An: Sharon; [email-masked]
Cc: Truman Boyes; Lou Berger; Larry Kreeger (kreeger)
Betreff: Re: [nvo3] NVO3 Terminology changes

End of line.

On Apr 9, 2013, at 9:59 PM, Sharon <[email-masked]<mailto:[email-masked]>> wrote:
NVMA +1

NVMA +1

Truman

Also...

Lou,

- Larry

Larry,

Lou

Lou,

- Larry

Lou

...

---

From: <[email-masked]>
Date: Wed, 10 Apr 2013 15:24:10 +0200

+1

Thanks
Manuel

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of LASSERRE, MARC (MARC)
Sent: Wednesday, April 10, 2013 9:38 AM
To: Sharon; Truman Boyes
Cc: Lou Berger; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

Marc

NVMA +1

Hi,

NVMA +1

Truman

Also...

- Larry

Larry,

Lou

- Larry

Lou

...

---

From: "Black, David" <[email-masked]>
Date: Wed, 10 Apr 2013 09:57:49 -0400

NVMA +1, and I think Mapping is ok wrt BGP, as nvo3’s use of BGP would be to distribute (address) mapping information.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Sharon
Sent: Wednesday, April 10, 2013 1:00 AM
To: Truman Boyes
Cc: Lou Berger; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

NVMA +1

NVMA +1

Truman

Also...

Lou,

- Larry

Larry,

Lou

Lou,

- Larry

Lou

...

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 10 Apr 2013 15:28:00 +0000

We are talking about an entity that has information on inner-outer address mapping, and potentially other host (VMs or Servers) location information. This entity is orthogonal to "virtualization".

Therefore, I think "Network Mapping Authority" or "Host Location Information Service" is more accurate.

Linda

From: [email-masked] [mailto:[email-masked]] On Behalf Of Truman Boyes
Sent: Tuesday, April 09, 2013 11:48 PM
To: Lou Berger
Cc: [email-masked]; Larry Kreeger (kreeger)
Subject: Re: [nvo3] NVO3 Terminology changes

Hi,

NVMA +1

Truman

Also...

Lou,

- Larry

Larry,

Lou

Lou,

- Larry

Lou

...

---

From: Lou Berger <[email-masked]>
Date: Wed, 10 Apr 2013 12:13:01 -0400

On 4/10/2013 11:28 AM, Linda Dunbar wrote:
> We are talking about an entity that has information on inner-outer
> address mapping, and potentially other host (VMs or Servers) location
> information. This entity is orthogonal to “virtualization”.

Do I understand your point correctly, that you're suggesting that this
discussion is orthogonal to the "V" in technologies such as L3VPN,
L2VPN/EVPN, NVGRE, VXLAN, ...?

Lou

> 
>  
> 
> Therefore, I think “Network Mapping Authority” or “Host Location
> Information Service” is more accurate.
> 
>  
> 
> Linda
> 
>  
> 
> *From:[email-masked] [mailto:[email-masked]] *On Behalf
> Of *Truman Boyes
> *Sent:* Tuesday, April 09, 2013 11:48 PM
> *To:* Lou Berger
> *Cc:* [email-masked]; Larry Kreeger (kreeger)
> *Subject:* Re: [nvo3] NVO3 Terminology changes
> 
>  
> 
> Hi,
> 
>  
> 
> BGP might not be a mapping service; but with a stretch of the
> imagination the mapping of NLRI to prefix/communities/etc could fulfil
> the litmus test. 
> 
>  
> 
> The below words of 'server', 'controller', and 'orchestrator' carry a
> lot of baggage/meaning at the moment and might convey other meanings. I
> like the NVMA as a functional element.
> 
>  
> 
> NVMA +1
> 
>  
> 
> Truman
> 
>  
>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 10 Apr 2013 16:22:14 +0000

> L2VPN/EVPN, NVGRE, VXLAN, ...?

[Linda] and TRILL, and PBB ...

---

From: Sharon <[email-masked]>
Date: Wed, 10 Apr 2013 10:00:27 -0700

That's a great and non trivial point.
In fact it's not simple at all to explain the subtle differences between layers like IP & Ethernet, virtual layers like vlans & vpns, and overlays like lisp & vxlan ..

.. to someone that didn't setup routing interfaces-peering on modular line-cards, reset 5000 bridges cause maintenance  moved an rj45 port, or rearranged subnet stochastics cause some fixed IP servers had to be moved from one building to another..

Hopefully NVO3 rfc will capture these differences and the flexibility-mobility of overlaying any IP network on any other IP network, if you can map between the two, and map the attributes making use of such "ball-bearing" indirection.

On Apr 10, 2013, at 8:28 AM, Linda Dunbar <[email-masked]> wrote:

> We are talking about an entity that has information on inner-outer address mapping, and potentially other host (VMs or Servers) location information. This entity  is orthogonal to “virtualization”.
>  
> Therefore, I think “Network Mapping Authority” or “Host Location Information Service” is more accurate.
>  
> Linda
>

---

From: Lou Berger <[email-masked]>
Date: Wed, 10 Apr 2013 13:04:14 -0400

> [Linda] and TRILL, and PBB ...

So how do you feel about the name of the WG?
(Particularly, the "V" and "O3" parts...)

Lou

---

From: Truman Boyes <[email-masked]>
Date: Wed, 10 Apr 2013 13:54:27 -0400

Let's not re-use NVO for Orchestrator. NVO is already for *Overlay*. Next
...

Everything in this WG is about network virtualization, and while there may
be underlay and overlay networking concepts being discussed, I think
underlay networking parameters  that may loosely relate to an overlay
virtual network construct can still be described by a NVMA (network
virtualized mapping authority) aka Policy server / directory, etc...

We are talking about a place to retrieve authoritative information on a
network construct.

I do not understand the TRILL and PBB statement highlighted above. Nor do I
see how "V" is orthogonal to anything we are discussing.

Truman

---

From: "Fedyk, Donald (Don)" <[email-masked]>
Date: Wed, 10 Apr 2013 19:10:31 +0000

I'd vote for NVC

PS. Most Layer 2 Ethernet technologies are full of V's (VLANs, VLAN Ids). 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: Wednesday, April 10, 2013 1:04 PM
To: Linda Dunbar
Cc: Truman Boyes; [email-masked]; Larry Kreeger (kreeger)
Subject: Re: [nvo3] NVO3 Terminology changes

On 4/10/2013 12:22 PM, Linda Dunbar wrote:
>>> address mapping, and potentially other host (VMs or Servers) 
>>> location information. This entity is orthogonal to “virtualization”.
>>
>> Do I understand your point correctly, that you're suggesting that 
>> this discussion is orthogonal to the "V" in technologies such as 
>> L3VPN, L2VPN/EVPN, NVGRE, VXLAN, ...?
> 

Lou

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 10 Apr 2013 19:19:46 +0000

I do not understand the TRILL and PBB statement highlighted above.

[Linda] They are all about ingress boundary node encapsulating another header (IP header, MPLS header, TRILL header, or MAC header) that consists of DA being the egress boundary node, SA being the ingress boundary node, and an additional "Instance ID" to further differentiate instances in the core (or underlay).
The data frames to Ingress boundary node are Ethernet frames, and the decapsulated frames by egress nodes to attached hosts are Ethernet frames.

They can all be benefited by this "inner - outer mapping service" in the similar way.

Nor do I see how "V" is orthogonal to anything we are discussing.

[Linda] everything has a "V": VLAN, VPN, .... The mapping service can help them all.

---

From: Lucy yong <[email-masked]>
Date: Wed, 10 Apr 2013 19:23:14 +0000

Hi Larry,

I am fine with NVC name. 

One clarification question: does this mean that we will work out both NV controller based NVo3 solution and Control Plane based NVo3 solution? The latter is in "distributed mode".  

Cheers,
Lucy

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Fedyk, Donald (Don)
> Sent: Wednesday, April 10, 2013 2:11 PM
> To: Lou Berger; Linda Dunbar

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 10 Apr 2013 19:37:24 +0000

Lucy,

We are discussing a single name that covers both a centralized and a
distributed model for the oracle.  The control plane between the NVE and
the oracle would not be different if the oracle is centralized vs
distributed.  That is why we have been saying that the oracle is
"logically centralized".  This is where an Architecture document would be
useful to clarify these things.

 - Larry

On 4/10/13 12:23 PM, "Lucy yong" <[email-masked]> wrote:

>Hi Larry,
>
>I am fine with NVC name.
>
>One clarification question: does this mean that we will work out both NV
>controller based NVo3 solution and Control Plane based NVo3 solution? The
>latter is in "distributed mode".
>
>Cheers,
>Lucy
>
>> Lou Berger
>> >>> location information. This entity is orthogonal to ³virtualization².
>> >>
>> >>> Therefore, I think ³Network Mapping Authority² or ³Host Location
>> >>> Information Service² is more accurate.
>> >>>

---

From: Lucy yong <[email-masked]>
Date: Wed, 10 Apr 2013 19:56:42 +0000

I see. Two cases: 1) if the oracle is on NVE, the control plane between the NVE and the oracle is just internal API. You will need the control plane between the oracles. 2) if the oracle is not on NVE, the control plane between the NVE and the oracle need some protocols. You may still need the control plane between the oracles if there are multiple. 

Logical centralized means no one-on-one mapping between NVE and oracle, right?

Lucy

> Larry Kreeger (kreeger)
> Sent: Wednesday, April 10, 2013 2:37 PM
> To: Lucy yong; Fedyk, Donald (Don); Lou Berger; Linda Dunbar
> Cc: Truman Boyes; [email-masked]

---

From: Melinda Shore <[email-masked]>
Date: Wed, 10 Apr 2013 12:35:22 -0800

On 4/10/2013 11:56 AM, Lucy yong wrote:
> I see. Two cases: 1) if the oracle is on NVE, the control plane
> between the NVE and the oracle is just internal API. You will need
> the control plane between the oracles. 2) if the oracle is not on
> NVE, the control plane between the NVE and the oracle need some
> protocols. You may still need the control plane between the oracles
> if there are multiple.

I think you can pretty clearly keep a functional separation.
The term that was current when VoIP was fresh and young was
"decomposed," and in the interest of modularity it's not a
bad idea to keep logically distinct components separate, even
if they're in the same box.  If nothing else, you don't
want to force an implementation model on something that hasn't
even been designed yet, really.

Melinda

---

From: "Pat Thaler" <[email-masked]>
Date: Wed, 10 Apr 2013 22:43:22 +0000

I agree that the V isn't orthogonal to what we are discussing.

There are (at least) two separate kinds of virtualization -

-          end point virtualization (e.g. VMs) which is related to but orthogonal to what we are talking about, and

-          network virtualization

-
NVO is working on layer 3 network virtualization
PBB and other IEEE 802.1Q technologies including VLANs and Provider Bridging are about layer 2 network virtualization. I don't think that it is relevant to the NVO work. It's already covered in IEEE 802.1Q

Pat

From: [email-masked] [mailto:[email-masked]] On Behalf Of Truman Boyes
Sent: Wednesday, April 10, 2013 10:54 AM
To: Lou Berger
Cc: [email-masked]; Larry Kreeger (kreeger); Linda Dunbar
Subject: Re: [nvo3] NVO3 Terminology changes

Let's not re-use NVO for Orchestrator. NVO is already for *Overlay*. Next ...

Everything in this WG is about network virtualization, and while there may be underlay and overlay networking concepts being discussed, I think underlay networking parameters  that may loosely relate to an overlay virtual network construct can still be described by a NVMA (network virtualized mapping authority) aka Policy server / directory, etc...

We are talking about a place to retrieve authoritative information on a network construct.

I do not understand the TRILL and PBB statement highlighted above. Nor do I see how "V" is orthogonal to anything we are discussing.

Truman

On Wed, Apr 10, 2013 at 1:04 PM, Lou Berger <[email-masked]<mailto:[email-masked]>> wrote:

On 4/10/2013 12:22 PM, Linda Dunbar wrote:
>>> information. This entity is orthogonal to "virtualization".
>>

Lou

---

From: "Pat Thaler" <[email-masked]>
Date: Wed, 10 Apr 2013 22:43:21 +0000

I'd prefer 
NVA - Network Virtualization Authority
- for an authoritative repository for mapping and possibly other information on Network Virtualization

NVMA - Network Virtualization Mapping Authority 
- my second choice but I don't see an need for the additional word even if we end up specifying it only doing mapping

"Controller" to me implies active control - so leans toward push rather than pull. Also, controller is used in the industry for the device on the NIC controlling access to the network. 

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: Tuesday, April 09, 2013 6:05 PM
To: Larry Kreeger (kreeger)
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

Also...

---

From: "Pat Thaler" <[email-masked]>
Date: Wed, 10 Apr 2013 22:43:23 +0000

The inner to outer address mapping supports creation of Tenant Virtual Networks over a physical network Therefore the mapping is about network virtualization (not about other types of virtualization such as virtual machines).

Pat

From: [email-masked] [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Wednesday, April 10, 2013 8:28 AM
To: Truman Boyes; Lou Berger
Cc: [email-masked]; Larry Kreeger (kreeger)
Subject: Re: [nvo3] NVO3 Terminology changes

Linda

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Truman Boyes
Sent: Tuesday, April 09, 2013 11:48 PM
To: Lou Berger
Cc: [email-masked]<mailto:[email-masked]>; Larry Kreeger (kreeger)
Subject: Re: [nvo3] NVO3 Terminology changes

Hi,

NVMA +1

Truman

Also...

- Larry

Larry,

Lou

- Larry

Lou

...

---

From: Lou Berger <[email-masked]>
Date: Wed, 10 Apr 2013 18:54:17 -0400

Pat,
	I can see that some might have an issue with the NIC association.
While I still prefer NVC, I think NVA is a reasonable alternative.

I also think map/mapping has a bunch of context-specific connotations
beyond what we're considering, so either of the above TLAs would
definitely be better than NVMA...

Lou

---

From: Truman Boyes <[email-masked]>
Date: Wed, 10 Apr 2013 18:57:07 -0400

This is a sound argument. Agreed with comments about mapping. NVA makes sense. 

Kind regards,
Truman Boyes

---

From: Yizhou Li <[email-masked]>
Date: Thu, 11 Apr 2013 15:10:32 +0800

I tends to agree with Pat.

NV controller would be expected to be simplified as 'controller' in
conversation while controller is a vague term. Authority is a more distinct
term used by NVO3 and the basic implication of it is/should be mapping
information holder.

Yizhou
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Pat
Thaler
Sent: Thursday, April 11, 2013 6:43 AM
To: Lou Berger; Larry Kreeger (kreeger)
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

I'd prefer 
NVA - Network Virtualization Authority
- for an authoritative repository for mapping and possibly other information
on Network Virtualization

NVMA - Network Virtualization Mapping Authority 
- my second choice but I don't see an need for the additional word even if
we end up specifying it only doing mapping

"Controller" to me implies active control - so leans toward push rather than
pull. Also, controller is used in the industry for the device on the NIC
controlling access to the network. 

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Lou
Berger
Sent: Tuesday, April 09, 2013 6:05 PM
To: Larry Kreeger (kreeger)
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

Also...

---

From: Mach Chen <[email-masked]>
Date: Thu, 11 Apr 2013 08:25:54 +0000

+1 to NVA, it's simple and clear!

Best regards,
Mach

> Sent: Thursday, April 11, 2013 6:57 AM
> To: Lou Berger
> Cc: Pat Thaler; [email-masked]; Larry Kreeger (kreeger)
> pull. Also, controller is used in the industry for the device on the NIC controlling
> access to the network.
> >>
> >> Pat
> >>

---

From: Qin Wu <[email-masked]>
Date: Thu, 11 Apr 2013 09:59:46 +0000

>LK2> Again, I don't want to replace all instances of VNIC with TSI, but use the more general term TSI when it is not a specific example of a TS that is a Virtual Machine.  So, a VNIC is one type of TSI, but all TSIs are not >VNICs.

[Qin]: Great, we are on the same page.

“
”

>LK2> I think that you are assuming that "host" is synonymous with "Hypervisor".  In the definition above, I believe the term host relates to the more traditional definition of an internet host such…

[Qin]: So “Host” in the definition of tenant system seems misleading since we two have different interpretation to it. I agree Hypersor or Server or Server blade can host multiple VMs, however in the framework document, it also said, a host can be server or server blade in the definition of End device.

Suppose one tenant A have 2 VMs resided in the server1 or hypervisor1. Tenant B have 3 VMs resided in the server 1.

Can we say each VM belonging to the same tenant is a tenant system or multiple VMs of each tenant sharing the same
Server belong to the same single tenant system?  i.e., Tenant System A corresponding to Tenant A contains 2 VMs. Tenant System B corresponding to Tenant B contains 3 VMs.

---

From: Pankaj Garg <[email-masked]>
Date: Wed, 10 Apr 2013 00:30:02 +0000

Thanks,
Pankaj

Hi Sharon,

 - Larry

 - Larry

Lucy

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Fri, 12 Apr 2013 00:29:33 +0000

From: Qin Wu <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 11, 2013 2:59 AM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [nvo3] 答复: 答复: NVO3 Terminology changes

LK3> Great!

“
”

LK3> Only servers or server blades that are running hypervisors can host multiple VMs.  When we mean hypervisor we must say hypervisor (what form factor of server it is running on is irrelevant).  All hypervisors run on servers, but all servers are not running hypervisor software.

Can we say each VM belonging to the same tenant is a tenant system

LK3> Yes

or multiple VMs of each tenant sharing the same Server belong to the same single tenant system?  i.e., Tenant System A corresponding to Tenant A contains 2 VMs. Tenant System B corresponding to Tenant B contains 3 VMs.

LK3> No, that would be a really bad idea IMO.  The fact that two VMs happen to be running on the same server hardware is an ephemeral thing.  From one moment to the next a VM may migrate to a different server where it becomes separated from the other VMs.  Furthermore, I see no advantage in defining a Tenant System to be a grouping of VMs from a networking perspective since the thing that connects multiple VMs together is the network.

---

From: Sharon <[email-masked]>
Date: Fri, 12 Apr 2013 21:44:45 -0700

NVA
NVMA
NVMAP

all good. No?

---

From: Qin Wu <[email-masked]>
Date: Mon, 15 Apr 2013 07:16:52 +0000

“
”

LK3> Only servers or server blades that are running hypervisors can host multiple VMs.  When we mean hypervisor we must say hypervisor (what form factor of server it is running on is irrelevant).  All hypervisors run on servers,

[Qin]: Yes.

>but all servers are not running hypervisor software.

[Qin]: Are you saying that not all server are running hypervisor software? If yes, I agree. If not, would you like to clarify a little bit?

LK3> Yes

[Qin]: Good point, I fully agree with you. Here is the update to http://www.ietf.org/internet-drafts/draft-wu-nvo3-nve2nve-03.txt, (NVO3 control plane requirement mostly for NVE to NV Authority/Controller interface) which clears terminology confusing based on our discussion.

---

From: "Reith, Lothar" <[email-masked]>
Date: Mon, 15 Apr 2013 11:17:44 +0000

I believe that it is not possible to replace the term vNIC with “TSI”, because multiple vNICs may form a single interface.

 AFAIK it is common practice to use vNIC bonding (LAG) for high availability, where two vNICs of one VM are associated with different physical NICs of the virtualization host running the VMs. Creating a LAG of these 2 vNICs ( NIC-Bonding) creates a single Interface of the VM which survives the outage of one of the both physical NICs.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Qin Wu
Gesendet: Montag, 15. April 2013 09:17
An: Larry Kreeger (kreeger); [email-masked]
Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

“
”

[Qin]: Yes.

LK3> Yes

---

From: Qin Wu <[email-masked]>
Date: Tue, 16 Apr 2013 01:14:09 +0000

I am not sure how vNIC bonding works? Is it common?
However I think we can simply replace vNIC with Tenant System,
when vNIC have multiple IP address, such vNIC with multiple IP address could be treated as multiple TSIs, i.e., one tenant system with multiple TS interfaces.
This also gets in line with what Pat comment on this list.

Regards!
-Qin
发件人: Reith, Lothar [mailto:[email-masked]]
发送时间: 2013年4月15日 19:18
收件人: Qin Wu; Larry Kreeger (kreeger); [email-masked]
主题: AW: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Anoop Ghanwani <[email-masked]>
Date: Mon, 15 Apr 2013 22:19:47 -0700

On Mon, Apr 15, 2013 at 4:17 AM, Reith, Lothar <[email-masked]>wrote:

>  I believe that it is not possible to replace the term vNIC with “TSI”,
> because multiple vNICs may form a single interface.
>

I think it makes sense to use the term TSI.  While it may not offer 1:1
equivalence
to vNICs when it comes to implementation, it is probably OK for the way
that the
draft is written to replace vNIC with TSI.  I think vNIC was the wrong term
to use
to begin with because it mixes architecture with implementation,

Anoop

---

From: "Black, David" <[email-masked]>
Date: Tue, 16 Apr 2013 18:32:06 -0400

I think this may reflect some confusion around possible meanings of “interface”.  I believe that the notion of “interface” that is desired here corresponds to a vNIC, not a bonded set of vNICs (and likewise to a NIC, not a bonded set of NICs).  If that clarification is clear, I think TSI would be a reasonable term.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Reith, Lothar
Sent: Monday, April 15, 2013 7:18 AM
To: Qin Wu; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 16 Apr 2013 16:47:09 -0700

When I think of TSI, I think of something different than a vNIC (or, or in
the case of non-virtualized systems, a NIC).  Of course, it's possible to
create an implementation where a TSI is always a vNIC (or NIC), but I don't
think it is always limited to that.

It might help to clarify the following properties of a TSI:
- Can a TSI be associated with multiple NVEs?
- Does a TSI carry traffic from only one tenant or multiple tenants?

Anoop

On Tue, Apr 16, 2013 at 3:32 PM, Black, David <[email-masked]> wrote:

> I think this may reflect some confusion around possible meanings of
> “interface”.  I believe that the notion of “interface” that is desired here
> corresponds to a vNIC, not a bonded set of vNICs (and likewise to a NIC,
> not a bonded set of NICs).  If that clarification is clear, I think TSI
> would be a reasonable term.****
>
> ** **
>
> Thanks,
> --David****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Reith, Lothar
> *Sent:* Monday, April 15, 2013 7:18 AM
> *To:* Qin Wu; Larry Kreeger (kreeger); [email-masked]
> *Subject:* Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes****
>
> ** **
>
> because multiple vNICs may form a single interface.****
>
> ** **
>
>  AFAIK it is common practice to use vNIC bonding (LAG) for high
> availability, where two vNICs of one VM are associated with different
> physical NICs of the virtualization host running the VMs. Creating a LAG of
> these 2 vNICs ( NIC-Bonding) creates a single Interface of the VM which
> survives the outage of one of the both physical NICs.****
>
> ** **
>
> ** **
>
> Lothar****
>
> ** **
>
> *Von:* [email-masked] [mailto:[email-masked]] *Im Auftrag
> von *Qin Wu
> *Gesendet:* Montag, 15. April 2013 09:17
> *An:* Larry Kreeger (kreeger); [email-masked]
> *Betreff:* [nvo3] 答复: 答复: 答复: NVO3 Terminology changes****
>
> ** **
>
> multiple vNIC adapters that it uses to communicate with both the virtual
> and physical networks.****
>
>  ****
>
> >LK> A VM is one example of a tenant system…so it would not host VMs.
>  You may be thinking of "End Device".****
>
>  ****
>
> >[Qin]: Not sure about that, the definition of “Tenant system” in
> Framework said:****
>
> “****
>
> ****
> >       of a host, or a forwarding element such as a router, switch,****
>
> >       firewall, etc. It belongs to a single tenant and connects to one
> or****
>
> >       more VNs of that tenant.****
>
> ”****
>
> >So tenant system can be a host and host one or multiple VMs on it. What
> am I missing?****
>
>  ****
>
> >LK2> I think that you are assuming that "host" is synonymous with
> "Hypervisor".  In the definition above, I believe the term host relates to
> the more traditional definition of an internet host such as in RFC 1122.**
> **
>
>  ****
>
>
> [Qin]: So “Host” in the definition of tenant system seems misleading since
> we two have different interpretation to it. I agree Hypersor or Server or
> Server blade can host multiple VMs, however in the framework document, it
> also said, a host can be server or server blade in the definition of End
> device.****
>
> ** **
>
> LK3> Only servers or server blades that are running hypervisors can host
> multiple VMs.  When we mean hypervisor we must say hypervisor (what form
> factor of server it is running on is irrelevant).  All hypervisors run on
> servers, ****
>
> ** **
>
> [Qin]: Yes.****
>
> ** **
>
> >but all servers are not running hypervisor software.****
>
> ** **
>
> [Qin]: Are you saying that not all server are running hypervisor software?
> If yes, I agree. If not, would you like to clarify a little bit?****
>
>  ****
>
> Suppose one tenant A have 2 VMs resided in the server1 or hypervisor1.
> Tenant B have 3 VMs resided in the server 1. ****
>
>  ****
>
> Can we say each VM belonging to the same tenant is a tenant system ****
>
> ** **
>
> LK3> Yes****
>
> ** **
>
> or multiple VMs of each tenant sharing the same Server belong to the same
> single tenant system?  i.e., Tenant System A corresponding to Tenant A
> contains 2 VMs. Tenant System B corresponding to Tenant B contains 3 VMs.*
> ***
>
>
> LK3> No, that would be a really bad idea IMO.  The fact that two VMs
> happen to be running on the same server hardware is an ephemeral thing.
>  From one moment to the next a VM may migrate to a different server where
> it becomes separated from the other VMs.  Furthermore, I see no advantage
> in defining a Tenant System to be a grouping of VMs from a networking
> perspective since the thing that connects multiple VMs together is the
> network.****
>
> ** **
>
> [Qin]: Good point, I fully agree with you. Here is the update to
> http://www.ietf.org/internet-drafts/draft-wu-nvo3-nve2nve-03.txt, (NVO3
> control plane requirement mostly for NVE to NV Authority/Controller
> interface) which clears terminology confusing based on our discussion.****
>
> ** **
>
>
>
>
> ****
>
>  ****
>

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 16 Apr 2013 23:57:34 +0000

Hi Anoop,

I definitely see a TSI as belonging to a single tenant, since it is an interface on a single tenant's system (Tenant System).  My assumption is that each TSI is associated with only one NVE, so that there is a one to one correspondence between a TSI and a VAP on the NVE.  It is what is connected to the bottom lines in figure 4 of http://tools.ietf.org/html/draft-ietf-nvo3-framework-02

 - Larry

From: Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 16, 2013 4:47 PM
To: David Black <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

When I think of TSI, I think of something different than a vNIC (or, or in the case of non-virtualized systems, a NIC).  Of course, it's possible to create an implementation where a TSI is always a vNIC (or NIC), but I don't think it is always limited to that.

Anoop

On Tue, Apr 16, 2013 at 3:32 PM, Black, David <[email-masked]<mailto:[email-masked]>> wrote:
I think this may reflect some confusion around possible meanings of “interface”.  I believe that the notion of “interface” that is desired here corresponds to a vNIC, not a bonded set of vNICs (and likewise to a NIC, not a bonded set of NICs).  If that clarification is clear, I think TSI would be a reasonable term.

Thanks,
--David

From:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Reith, Lothar
Sent: Monday, April 15, 2013 7:18 AM
To: Qin Wu; Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Lothar

Von:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] Im Auftrag von Qin Wu
Gesendet: Montag, 15. April 2013 09:17
An: Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

“
”

[Qin]: Yes.

LK3> Yes

---

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 16 Apr 2013 17:32:31 -0700

Hi Larry,

That's how I would think about it as well.

The part about 1:1 correspondence with vNICs/NICs tends to break down when
we think of the case of a NIC or vNIC with multiple VLANs on it, each of
which in turn, maps to a different VAP.  So in this case, each of those
VLANs on the NIC would correspond to a different TSI, right?

Anoop

On Tue, Apr 16, 2013 at 4:57 PM, Larry Kreeger (kreeger)
<[email-masked]>wrote:

>  Hi Anoop,
>
>  I definitely see a TSI as belonging to a single tenant, since it is an
> interface on a single tenant's system (Tenant System).  My assumption is
> that each TSI is associated with only one NVE, so that there is a one to
> one correspondence between a TSI and a VAP on the NVE.  It is what is
> connected to the bottom lines in figure 4 of
> http://tools.ietf.org/html/draft-ietf-nvo3-framework-02
>
>   - Larry
>
>   From: Anoop Ghanwani <[email-masked]>
> To: David Black <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>, "Reith, Lothar" <
> [email-masked]>
>
>   When I think of TSI, I think of something different than a vNIC (or, or
> in the case of non-virtualized systems, a NIC).  Of course, it's possible
> to create an implementation where a TSI is always a vNIC (or NIC), but I
> don't think it is always limited to that.
>
>> *Von:[email-masked] [mailto:[email-masked]] *Im Auftrag
>> von *Qin Wu
>> >       Tenant System: A physical or virtual system that can play the
>> role****
>>
>> the more traditional definition of an internet host such as in RFC 1122.*
>> ***
>>
>>  ****
>>
>>
>> [Qin]: So “Host” in the definition of tenant system seems misleading
>> since we two have different interpretation to it. I agree Hypersor or
>> Server or Server blade can host multiple VMs, however in the framework
>> document, it also said, a host can be server or server blade in the
>> definition of End device.****
>>
>> ** **
>>
>> [Qin]: Are you saying that not all server are running hypervisor
>> software? If yes, I agree. If not, would you like to clarify a little bit?
>> ****
>>
>>  ****
>>
>> or multiple VMs of each tenant sharing the same Server belong to the
>> same single tenant system?  i.e., Tenant System A corresponding to Tenant A
>> contains 2 VMs. Tenant System B corresponding to Tenant B contains 3 VMs.
>> ****
>>
>>

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 17 Apr 2013 00:46:42 +0000

Hi Anoop,

Yes, you are right.  Since a VAP accesses a single VNI, that does imply there would be a TSI per VAP, which does not correspond well to a VNIC.  This is what Pat was saying also.  I was hoping for the TSI to be more analogous to a VNIC since the VNIC goes up/down/migrates at one time and all the VNs it may connect to must follow that state.  The Hypervisor-to-NVE protocol can be more optimized if that fact can be taken advantage of.

 - Larry

From: Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 16, 2013 5:32 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>
Cc: David Black <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Larry,

The part about 1:1 correspondence with vNICs/NICs tends to break down when we think of the case of a NIC or vNIC with multiple VLANs on it, each of which in turn, maps to a different VAP.  So in this case, each of those VLANs on the NIC would correspond to a different TSI, right?

Anoop

On Tue, Apr 16, 2013 at 4:57 PM, Larry Kreeger (kreeger) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Qin Wu <[email-masked]>
Date: Wed, 17 Apr 2013 02:14:52 +0000

Hi, Larry:
Is relation between vNIC and TSI one to one or one to more?
Let me interpret what you said below,
(a)If a VM has only one vNIC, then vNIC that belongs to the VM will be dealt with as one tenant system. Each vNIC corresponds to one TSI.
(b) If a VM has 5 vNICs and each vNIC is assigned with only one IP address, each vNIC that belongs to the same VM will be treated as one Tenant System. Each vNIC corresponds to one TSI. Then one VM have 5 Tenant system. Each Tenant System has one TSI.
(c) If a VM has 5 vNICs and each vNIC is assigned with 2 IP addresses, each vNIC that belongs to the same VM will be treated as one Tenant system, each vNIC with each IP address corresponds to one TSI. Then one VM has 5 tenant system. Each Tenant system has 2 TSIs.
That is to say if the relation between vNIC and TSi is one to more, then we can easy to guarantee that each TSI is associated with only one NVE.

However if one VM is simply regarded as one tenant system and each vNIC with multiple IP addresses belonging to one VM is treated as only one TSI in (b)(c), take (c) as an example. when one vNIC is assigned with multiple IP addresses and connect to multiple VNs through multiple NVE, then one TSI has multiple IP addresses and will be associated with multiple NVEs.
Here is two figure2, figure 1 shows one to one relation between vNIC and TSI, figure 2 shows one to more relation between vNIC and TSI.

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +
            vNIC1(TSI) vNIC2(TSI)  vNICx(TSI)
                  |
                  |
         |--------+------+
    (IP add1) (IP add2)  (IP addr3)
         |        |      |
        VN1      VN2    VN3

   Legend: TSI -- Tenant System Interface
           VN  -- Virtual Network
           IP add1
               -- IP address1
       Figure 1 One to One relation between vNIC and TSI

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |
          vNIC1(TS1)   vNIC2(TS2) vNIC3(TS3)
                 |
                 |
         +-------+------+
         |       |      |
         |       |      |
        TSI1    TSI2   TSI3

   (IP add1) (IP add2) (IP addr3)

    Legend: TSI1 -- Tenant System Interface1
            TS1_ Tenat System1

     Figure 2 One to more relation between vNIC and TSI

I think figure 2 well match what you said below. please confirm whether my understanding is correct.

Regards!
-Qin
发件人: [email-masked] [mailto:[email-masked]] 代表 Larry Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送: [email-masked]; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: John E Drake <[email-masked]>
Date: Wed, 17 Apr 2013 11:47:09 +0000

This thread reminds me of the lyrics to a Nicki Minaj song.

Irrespectively Yours,

John

From: [email-masked] [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Tuesday, April 16, 2013 7:15 PM
To: Larry Kreeger (kreeger); Anoop Ghanwani; Black, David
Cc: [email-masked]; Reith, Lothar
Subject: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin
发件人: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] 代表 Larry Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送: [email-masked]<mailto:[email-masked]>; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Truman Boyes <[email-masked]>
Date: Wed, 17 Apr 2013 10:15:43 -0400

Hilarious.

Btw, is this multiple vNIC scenario based on real world experience? I am
scared of more than 1 vNIC in a large scale deployment due to complexities
in routing at the host level and further upstream.

There may be corner cases the require a different set of reachable vectors
for a vm but there are many ways to provide that without more vNICs. Not
completely opposed to the idea, but when I see scenarios discussing VMs
with 5 vNICs I think the IETF may be off course here.

There is a lot of rope, it's our choice to make a raft or a noose.

Truman

On Wednesday, April 17, 2013, John E Drake wrote:

>  This thread reminds me of the lyrics to a Nicki Minaj song.****
>
> ** **
>
> Irrespectively Yours,****
>
> ** **
>
> John****
>
> ** **
>
> *From:* [email-masked] <javascript:_e({}, 'cvml',
> [email-masked]');> [mailto:[email-masked]<javascript:_e({}, 'cvml', [email-masked]');>]
> *On Behalf Of *Qin Wu
> *Sent:* Tuesday, April 16, 2013 7:15 PM
> *To:* Larry Kreeger (kreeger); Anoop Ghanwani; Black, David
> *Cc:* [email-masked] <javascript:_e({}, 'cvml', [email-masked]');>; Reith,
> Lothar
> *Subject:* [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes****
>
> ** **
>
> Hi, Larry:****
>
> Is relation between vNIC and TSI one to one or one to more?****
>
> Let me interpret what you said below, ****
>
> (a)If a VM has only one vNIC, then vNIC that belongs to the VM will be
> dealt with as one tenant system. Each vNIC corresponds to one TSI.****
>
> (b) If a VM has 5 vNICs and each vNIC is assigned with only one IP
> address, each vNIC that belongs to the same VM will be treated as one
> Tenant System. Each vNIC corresponds to one TSI. Then one VM have 5 Tenant
> system. Each Tenant System has one TSI.****
>
> (c) If a VM has 5 vNICs and each vNIC is assigned with 2 IP addresses,
> each vNIC that belongs to the same VM will be treated as one Tenant system,
> each vNIC with each IP address corresponds to one TSI. Then one VM has 5
> tenant system. Each Tenant system has 2 TSIs.****
>
> That is to say if the relation between vNIC and TSi is one to more, then
> we can easy to guarantee that each TSI is associated with only one NVE.***
> *
>
> ** **
>
> However if one VM is simply regarded as one tenant system and each vNIC
> with multiple IP addresses belonging to one VM is treated as only one TSI
> in (b)(c), take (c) as an example. when one vNIC is assigned with multiple
> IP addresses and connect to multiple VNs through multiple NVE, then one TSI
> has multiple IP addresses and will be associated with multiple NVEs.****
>
> Here is two figure2, figure 1 shows one to one relation between vNIC and
> TSI, figure 2 shows one to more relation between vNIC and TSI.****
>
>

---

From: Jon Hudson <[email-masked]>
Date: Wed, 17 Apr 2013 16:29:10 +0200

It's a worthwhile point. While I think it's always better to design for "N" and not "2" I don't think I have ever seen more that two vNICs on a VM. For the primary reason that it's usually 1:1 app to VM (if you can run multiple apps in the OS, then just keep running Linux, move on and forge this whole virtualization fad)

The only argument I can think of for running more than 2 is bandwidth, but vNICs are not NICs and can be bound to any speed interface the hypervisor is willing to support. So bandwidth is not a reason to bond 7 vNICs to a VM.

So while designing for N is logical, worrying too much about more than 2 vNICs per VM isn't terrible realistic.

And as you said many many customers just run one vNIC per VM.

J

On Apr 17, 2013, at 4:15 PM, Truman Boyes <[email-masked]> wrote:

> Hilarious. 
> 
> Btw, is this multiple vNIC scenario based on real world experience? I am scared of more than 1 vNIC in a large scale deployment due to complexities in routing at the host level and further upstream. 
> 
> There may be corner cases the require a different set of reachable vectors for a vm but there are many ways to provide that without more vNICs. Not completely opposed to the idea, but when I see scenarios discussing VMs with 5 vNICs I think the IETF may be off course here.
>

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Apr 2013 11:07:41 -0400

OTOH, if the vNIC is an 802.1Q VLAN-tagged interface, and different VLANs are connected to different virtual networks (VNIs, Virtual Network Instances), then the tenant side interface that corresponds to the VAP on the NVE is the VLAN within the vNIC.  At the other extreme, if the VNI is also 802.1Q tagged, and all the VLAN tags used by the vNIC are used with the same VNI, then the vNIC as a whole is what corresponds to the VAP.

We still need a term other than vNIC, as use of that term excludes physical network interfaces from participation in virtual networks.

Thanks,
--David

From: Jon Hudson [mailto:[email-masked]]
Sent: Wednesday, April 17, 2013 10:29 AM
To: Truman Boyes
Cc: John E Drake; [email-masked]; Reith, Lothar; Black, David; Anoop Ghanwani; Larry Kreeger (kreeger); Qin Wu
Subject: Re: [nvo3] NVO3 Terminology changes

J

On Apr 17, 2013, at 4:15 PM, Truman Boyes <[email-masked]<mailto:[email-masked]>> wrote:
Hilarious.

Truman

Irrespectively Yours,

John

From: [email-masked]<javascript:_e(%7b%7d,%20'cvml',[email-masked]');> [mailto:[email-masked]<javascript:_e(%7b%7d,%20'cvml',[email-masked]');>] On Behalf Of Qin Wu
Sent: Tuesday, April 16, 2013 7:15 PM
To: Larry Kreeger (kreeger); Anoop Ghanwani; Black, David
Cc: [email-masked]<javascript:_e(%7b%7d,%20'cvml',[email-masked]');>; Reith, Lothar
Subject: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi, Larry:

---

From: Eric Gray <[email-masked]>
Date: Wed, 17 Apr 2013 15:19:49 +0000

David,

                Not sure I agree with this.  While the term "virtual" has become (through usage)
a common term used to distinguish something (for example an interface,  network, or
private network) from the analog in reality – it should actually be the case that virtual
includes physical as a degenerate case.

                In other words, a NIC can be said to be a non-tagging vNIC that supports only
one virtual (untagged) interface.

                Also, can you clarify what it means to have distinct Virtual Local Area Networks
(VLANs) that are not "connected to different virtual networks"?

--
Eric

From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Wednesday, April 17, 2013 11:08 AM
To: Jon Hudson; Truman Boyes
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

Thanks,
--David

From: Jon Hudson [mailto:[email-masked]]
Sent: Wednesday, April 17, 2013 10:29 AM
To: Truman Boyes
Cc: John E Drake; [email-masked]<mailto:[email-masked]>; Reith, Lothar; Black, David; Anoop Ghanwani; Larry Kreeger (kreeger); Qin Wu
Subject: Re: [nvo3] NVO3 Terminology changes

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Apr 2013 11:45:48 -0400

Eric,

> one virtual (untagged) interface.

I disagree.  The term vNIC has become strongly associated with virtual machines,
and I don’t believe that this term is generally understood to encompass physical
NICs on physical servers or other systems - e.g., a middlebox, such as a firewall.

> (VLANs) that are not "connected to different virtual networks"?

Sure, consider a virtual network providing L2 VLAN-tagged service, and configured
so that any Ethernet frame arriving at the NVE is encapsulated to the same VNI,
regardless of VLAN.  The result is that the inner L2 Ethernet header beyond the NVE
includes the unchanged VLAN tag.  In this case, the virtual network instance (VNI)
supports multiple VLANs end-to-end, and those VLANs have no interactions with the
NVEs (the VLAN tag just passes through encap and decap).

Thanks,
--David

From: Eric Gray [mailto:[email-masked]]
Sent: Wednesday, April 17, 2013 11:20 AM
To: Black, David; Jon Hudson; Truman Boyes
Cc: [email-masked]
Subject: RE: [nvo3] NVO3 Terminology changes

David,

--
Eric

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: David Allan I <[email-masked]>
Date: Wed, 17 Apr 2013 15:52:08 +0000

Hi Dave

I have some sympathy with Eric's point but would suggest inverting the relationship to address your concern. The generic class is NIC, which may or may not be virtualizated....the virtual form being associated with VMs, and the generic class being associated with addressed end systems in general.

I think moving away from NIC to produce some gratuitous new term is not doing us any favors...

cheers
Dave

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Wednesday, April 17, 2013 8:46 AM
To: Eric Gray
Cc: [email-masked]
Subject: Re: [nvo3] NVO3 Terminology changes

Eric,

Thanks,
--David

David,

--
Eric

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: Eric Gray <[email-masked]>
Date: Wed, 17 Apr 2013 15:58:44 +0000

David,

                If you had a virtual machine consisting of a single virtual instance as part of a physical server,
connected via a single-instance vNIC associated with a physical interface – how would any other device
on the network be able to distinguish this "virtual machine" from a "non-virtual" system of the same
type?

                Thanks for the clarification on the VLAN/VNI case.  I would probably have understood your
meaning better had you said "connected via"  as opposed to "connected to."

--
Eric

From: Black, David [mailto:[email-masked]]
Sent: Wednesday, April 17, 2013 11:46 AM
To: Eric Gray
Cc: [email-masked]
Subject: RE: [nvo3] NVO3 Terminology changes
Importance: High

Eric,

Thanks,
--David

David,

--
Eric

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Black, David

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Apr 2013 12:46:31 -0400

Eric,

> connected via a single-instance vNIC associated with a physical interface – how would any
> other device on the network be able to distinguish this "virtual machine" from a "non-virtual"
> system of the same type?

Well, if the virtual switch is doing something other than discarding BPDUs, this
difference is likely to be visible.  Nonetheless, that BPDU discarding is likely
behavior, and moreover, this Q&A is a rathole ;-).

Overall, this discussion should be about effective use of terminology to
communicate concepts.  On that note, I’d be happy w/Dave Allan’s suggestion to
invert things and use NIC as the general term in nvo3 drafts with a discussion
that the virtual network may only be using one (or even possibly a subset of)
the VLANs on a VLAN-tagged NIC.  vNIC could then be a specialization of NIC to

Thanks,
--David

From: Eric Gray [mailto:[email-masked]]
Sent: Wednesday, April 17, 2013 11:59 AM
To: Black, David
Cc: [email-masked]
Subject: RE: [nvo3] NVO3 Terminology changes

David,

--
Eric

Eric,

Thanks,
--David

David,

--
Eric

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 17 Apr 2013 16:55:50 +0000

Hi Qin,

A VM is one possible implementation of a Tenant System. I have not seen anyone suggesting that a Tenant System get more granular than that.

Regarding what defines a TSI, someone could choose any branch in the tree below to define as the TSI.  I was assuming that the TSI would be at point A below in the tree.  However, point B would correspond the the VAP.  Pat had mentioned having the TSI at point D.  I think if we were to define the TSI to be either point C or point D, then it would depend on the service being given (L2 or L3).  I would prefer it to be at point A or B.

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

From: Qin Wu <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 16, 2013 7:14 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>
Subject: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Eric Gray <[email-masked]>
Date: Wed, 17 Apr 2013 19:23:49 +0000

And, of course, this is another (probably better) way to look at it.

From: David Allan I
Sent: Wednesday, April 17, 2013 11:52 AM
To: Black, David; Eric Gray
Cc: [email-masked]
Subject: RE: [nvo3] NVO3 Terminology changes
Importance: High

Hi Dave

cheers
Dave

Thanks,
--David

David,

--
Eric

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: Thomas Narten <[email-masked]>
Date: Wed, 17 Apr 2013 16:12:23 -0400

David Allan I <[email-masked]> writes:

> I have some sympathy with Eric's point but would suggest inverting the
> relationship to address your concern. The generic class is NIC, which
> may or may not be virtualizated....the virtual form being associated
> with VMs, and the generic class being associated with addressed end
> systems in general.
>  

I think a TSI is roughly equivalent to a NIC, but from the perspective
of a Tenant System. I view it as the entity that the Tenant System
sees. To the tenant, it looks like a NIC, and is how the Tenant System
interacts with the (virtual) network.

Using some other term than NIC or vNIC is (IMO) useful because this is
a Tenant System entity, whereas a NIC/vNIC applies when network
virtualization is not in use. It is good to have a term that
distinguishes those cases.

Maybe a "Tenant NIC" (or TNIC) would make a better name than TSI. (Not
sure it's a better term, but worth thinking about, especially if this
really is essentially a "NIC".)

Thomas

---

From: "Pat Thaler" <[email-masked]>
Date: Wed, 17 Apr 2013 20:59:38 +0000

I’m not sure how vNIC bonding works either. Most bonding I’m familiar with is at the level of bonding physical NICs.

Bonding (or teaming) is an ill-defined term. It is used to mean either Link Aggregation (LAG, originally part of IEEE 802.3 and now IEEE 802.1AX) or one of several non-standardized ways of using multiple physical NICs so that they appear as one NIC to the upper layer clients.

An IEEE 802.1AX LAG should be transparent to anything we are doing in NVO3. As currently defined, a LAG is two or more (physical) links that go between two systems (e.g. between an end system and a bridge or between two bridge) where each system divides the traffic amongst the links keeping a conversation (e.g. MAC address, VLAN and priority; some systems also include higher layer fields in making that determination) on a single (physical) link to minimize mis-ordering.  The existence of the LAG should be transparent to any client above LAG’s MAC Client interface.

Note:  There is an IEEE 802.1AX revision underway to add Dual Resilient Network Interconnect where the links in a LAG can be connecting a set of bridges in one provider network to a set of bridges in another provider network, but it isn’t clear that there is any intersection between that and NVO3 which would normally operate within a network administered by a single organization.

Regards,
Pat

From: [email-masked] [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Monday, April 15, 2013 6:14 PM
To: Reith, Lothar; Larry Kreeger (kreeger); [email-masked]
Subject: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Regards!
-Qin
发件人: Reith, Lothar [mailto:[email-masked]]
发送时间: 2013年4月15日 19:18
收件人: Qin Wu; Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
主题: AW: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Lothar

Von: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] Im Auftrag von Qin Wu
Gesendet: Montag, 15. April 2013 09:17
An: Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Wed, 17 Apr 2013 21:02:16 +0000

Hi Thomas:

Hmmm, so what you want is the notion of an attachment point to a particular layer, in this case to explicitly identify the overlay as the network attached to.... (?, just checking)

>From the tenants POV it cannot tell the difference, it just sees its network, so I'm not sure the distinction is as useful as it sounds...

Cheers
Dave

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Thomas Narten
Sent: Wednesday, April 17, 2013 1:12 PM
To: David Allan I
Cc: Black, David; [email-masked]; Eric Gray
Subject: Re: [nvo3] NVO3 Terminology changes

> not doing us any favors...

I think a TSI is roughly equivalent to a NIC, but from the perspective of a Tenant System. I view it as the entity that the Tenant System sees. To the tenant, it looks like a NIC, and is how the Tenant System interacts with the (virtual) network.

Using some other term than NIC or vNIC is (IMO) useful because this is a Tenant System entity, whereas a NIC/vNIC applies when network virtualization is not in use. It is good to have a term that distinguishes those cases.

Maybe a "Tenant NIC" (or TNIC) would make a better name than TSI. (Not sure it's a better term, but worth thinking about, especially if this really is essentially a "NIC".)

Thomas

---

From: Anoop Ghanwani <[email-masked]>
Date: Wed, 17 Apr 2013 20:01:50 -0700

Larry,

In reference to the figure below, I think all of them could be valid TSIs
depending on the application.

Anoop

On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger)
<[email-masked]>wrote:

>  Hi Qin,
>
>  A VM is one possible implementation of a Tenant System. I have not seen
> anyone suggesting that a Tenant System get more granular than that.
>
>  Regarding what defines a TSI, someone could choose any branch in the
> tree below to define as the TSI.  I was assuming that the TSI would be at
> point A below in the tree.  However, point B would correspond the the VAP.
>  Pat had mentioned having the TSI at point D.  I think if we were to define
> the TSI to be either point C or point D, then it would depend on the
> service being given (L2 or L3).  I would prefer it to be at point A or B.
>
>           VNIC-+-VN-+-MAC-+-IP
>            ^  |    |     +-IP ...
>            |  |    |
>            |  |    +-MAC-+-IP
>            |  |          +-IP ...
>            |  |
>            |  +-VN-+-MAC-+-IP
>            |     ^ |     +-IP ...
>            |     | |
>            |     | +-MAC-+-IP
>            |     |    ^  +-IP ...
>            |     |    |     ^
>            |     |    |     |
>            A     B    C     D
>
>   - Larry
>
>   From: Qin Wu <[email-masked]>
> To: Larry Kreeger <[email-masked]>, Anoop Ghanwani <
> [email-masked]>, David Black <[email-masked]>
>
>                 +---------------------+****
>
>                 |                     |****
>
>                 |  VM (Tenant System) |****
>
>                 |                     |****
>
>                 +-+--------+--------+-+****
>
>                   |        |        |****
>
>                   |        |        |****
>
>                   |        |        |****
>
>                   +        + ...... +****
>
>             vNIC1(TSI) vNIC2(TSI)  vNICx(TSI)****
>
>                   |****
>
>                   |****
>
>          |--------+------+****
>
>     (IP add1) (IP add2)  (IP addr3)****
>
>          |        |      |****
>
>         VN1      VN2    VN3****
>
> ** **
>
>    Legend: TSI -- Tenant System Interface****
>
>            VN  -- Virtual Network****
>
>            IP add1****
>
>                -- IP address1****
>
>        Figure 1 One to One relation between vNIC and TSI****
>
> ** **
>
> ** **
>
>                +---------------------+****
>
>                |                     |****
>
>                |  VM (Tenant Systems)|****
>
>                |                     |****
>
>                +-+--------+--------+-+****
>
>                  |        |        |****
>
>                  |        |        |****
>
>                  |        |        |****
>
>                  |                 |****
>
>           vNIC1(TS1)   vNIC2(TS2) vNIC3(TS3)****
>
>                  |****
>
>                  |****
>
>          +-------+------+****
>
>          |       |      |****
>
>          |       |      |****
>
>         TSI1    TSI2   TSI3****
>
> ** **
>
>    (IP add1) (IP add2) (IP addr3)****
>
> ** **
>
>     Legend: TSI1 -- Tenant System Interface1****
>
>             TS1_ Tenat System1****
>
> ** **
>
>      Figure 2 One to more relation between vNIC and TSI****
>
> ** **
>
> I think figure 2 well match what you said below. please confirm whether my
> understanding is correct.****
>
> ** **
>
> Regards!****
>
> -Qin****
>
> *发件人:* [email-masked] [mailto:[email-masked]<[email-masked]>]
> *代表 *Larry Kreeger (kreeger)
> *发送时间:* 2013年4月17日 7:58
> *收件人:* Anoop Ghanwani; Black, David
> *抄送:* [email-masked]; Reith, Lothar
> *主题:* Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes****
>
> ** **
>
> Hi Anoop,****
>
> ** **
>
> http://tools.ietf.org/html/draft-ietf-nvo3-framework-02****
>
> ** **
>
>  - Larry****
>
> ** **
>
> *From: *Anoop Ghanwani <[email-masked]>
> *Date: *Tuesday, April 16, 2013 4:47 PM
> *To: *David Black <[email-masked]>
> *Cc: *"[email-masked]" <[email-masked]>, "Reith, Lothar" <
> *Subject: *Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes****
>
> ** **
>
> think it is always limited to that. ****
>
> ** **
>
> It might help to clarify the following properties of a TSI:****
>
> - Can a TSI be associated with multiple NVEs?****
>
> - Does a TSI carry traffic from only one tenant or multiple tenants?****
>
> ** **
>
> Anoop****
>
> ** **
>
> *From:[email-masked] [mailto:[email-masked]] *On Behalf Of
> *Reith, Lothar
> https://www.ietf.org/mailman/listinfo/nvo3****
>
> ** **
>

---

From: Anoop Ghanwani <[email-masked]>
Date: Wed, 17 Apr 2013 20:16:20 -0700

Sometimes folks have redundant NIC cards in the server and they want each
individual VM to have a redundant connection via each card.

It may not be the most common use case, but it exists.

Anoop

On Wed, Apr 17, 2013 at 7:15 AM, Truman Boyes <[email-masked]> wrote:

>> Of *Qin Wu
>> *Cc:* [email-masked]; Reith, Lothar
>> we can easy to guarantee that each TSI is associated with only one NVE.**
>> **
>>
>> ** **
>>

---

From: Qin Wu <[email-masked]>
Date: Thu, 18 Apr 2013 03:28:45 +0000

WARNING: contains banned part

Subject: 答复: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes
From: Qin Wu <[email-masked]>
To: "Larry Kreeger (kreeger)" <[email-masked]>, Anoop Ghanwani <[email-masked]>, "Black, David" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "Reith, Lothar" <[email-masked]>
Date: Thu, 18 Apr 2013 03:28:45 +0000

Hi, Larry:

Please see my follow-up comment below.

Regards!

-Qin

发送时间: 2013年4月18日 0:56

收件人: Qin Wu; Anoop Ghanwani; Black, David

主题: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

>A VM is one possible implementation of a Tenant System. I have not seen anyone suggesting that a Tenant System get more granular than that.

[Qin]: Okay, I should agree with you since I didn’t see more granularity either and still doubt about that.

>Regarding what defines a TSI, someone could choose any branch in the tree below to define as the TSI.  I was assuming that the TSI would be at point A below in the tree.  However, point B would correspond the the

>VAP.  Pat had mentioned having the TSI at point D.  I think if we were to define the TSI to be either point C or point D, then it would depend on the service being given (L2 or L3).  I would prefer it to be at point A or B.

>           A     B    C     D

[Qin]:Firstly I agree with Pat about having TSI at point D, In the meanwhile I think we could also having TSI at point B.

Point C doesn't make sense to me since each VNIC only MUST be assigned one unique MAC address based on my understanding, when VNIC connects multiple VNs, you can not use one MAC address to distinct different TSI.

Point A is also having problem, as I mentioned earlier below, , if we regard vNIC as TSI, you may have case to associate one TSI with multiple NVEs when vNIC connect to multiple VNs.

Secondly, VNIC Containment Relationship in the figure you described above introduces a little bit confusing, since you didn't distinguish the entity or objects from identifiers since vNIC and VN are both objects while MAC address and IP address are both identifier. So I proposed to change the figure you described above as follows:

You don’t need to care about the relation between vNIC and TSI is 1:1 or 1: N.

Each TSI is identifier by combination of VNID, MAC address of vNIC, IP address of vNIC,

when going to pNIC, TSI can be identified by only combination of MAC address and IP address, not necessary adding VNID.

Note that for one vNIC, it uses the same MAC address for all the TSIs belonging to the same vNIC.

For L2 and L3 service distinction, you can add additional identifier to [VNID, MAC address, IP address], e.g., VLAN Tag to distinct L2 from L3.

I think this proposed change above also considers both NIC and vNIC and address the concerns what we discussed on this list and is also still getting in line with what pat observed.

Regards!

-Qin

---

From: Qin Wu <[email-masked]>
Date: Thu, 18 Apr 2013 04:01:26 +0000

Hi, :
Sorry, my figure is filtered out. So I send it again.

Regards!
-Qin

>Regarding what defines a TSI, someone could choose any branch in the tree below to define as the TSI.  I was assuming that the TSI would be at point A below in the tree.  However, point B would correspond the
         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

             +---------------------+
             |                     |
             |  VM (Tenant System) |
             |                     |
             +-+--------+--------+-+
               |        |        |
               |        |        |
               |        |        |
               +        + ...... +
         vNIC1/pNIC1  vNIC2     vNICx
               |
               |
      |--------+------+------+
      |        |      |      |
      |        |      |      |
     VN1      VN2    VN3    Non-VN

Tenant System Interfaces(TSI):
  TSIa [VNID1,MAC1,IP addr1]corresponding to vNIC
  TSIb [VNID2,MAC1,IP addr2]corresponding to vNIC
  TSIc [VNID3,MAC1,IP addr3]corresponding to vNIC
  TSId [     ,MAC1,IP add4] corresponding to pNIC

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 18 Apr 2013 20:02:57 +0000

+1 with Dave Allen‘s definition

I tend to oppose the view that the term “virtual” can degenerate into a meaning that also includes “physical”. This may confuse or make impossible a common understanding of the term “virtualization” in the contexts of  compute, store and network.  A common definition of “virtualization” could be along the lines that virtualization decouples a logical resource from a dedicated physical resource by introducing a virtualization layer running on the same or a different physical resource, such that  now the virtualization layer  provides the interfaces to the higher layers that previously the physical resource used to provide directly.  With physical resource I mean a storage, compute or network resource.  In case of storage, the storage in an X86 architecture server or PC gets emulated by some network attached storage, which may even be distributed over multiple locations or replicated. The architecture of the virtualization host does not have to be X86 based, however, the VMs architecture is.

In the case of network, I am intentionally silent, and would appreciate comment if the definition may fit or how it could be improved.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Eric Gray
Gesendet: Mittwoch, 17. April 2013 21:24
An: David Allan I; Black, David
Cc: [email-masked]
Betreff: Re: [nvo3] NVO3 Terminology changes

From: David Allan I

Hi Dave

cheers
Dave

Thanks,
--David

David,

--
Eric

Thanks,
--David

J

Truman

Irrespectively Yours,

John

Hi, Larry:

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 18 Apr 2013 20:41:10 +0000

Hi Anoop,

What benefit do you see in defining the TSI as point C or D?  Do you agree that the TSI should directly correspond to the VAP of the NVE?  If so, it seems like point B corresponds to it.

Thanks, Larry

From: Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 17, 2013 8:01 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>
Cc: Qin Wu <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Larry,

In reference to the figure below, I think all of them could be valid TSIs depending on the application.

Anoop

On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Qin,

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin
发件人:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] 代表 Larry Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送: [email-masked]<mailto:[email-masked]>; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 18 Apr 2013 21:38:25 +0000

Hi Qin,

Responses are below with LK>.

 - Larry

Date: Wednesday, April 17, 2013 9:01 PM

Hi, :

Regards!
-Qin

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

[Qin]:Firstly I agree with Pat about having TSI at point D

LK> There is a problem with this if the VN service is at layer 2 allowing any L3 protocol above it to be carried transparently.  The IP address must be optional.

In the meanwhile I think we could also having TSI at point B.

LK> Just because a NIC/vNIC has one "burned in" address, it doesn't mean the tenant system will not have traffic with more than one MAC address.  For example, if the TS is a router, it may use the burned-in MAC and also have a virtual MAC for supporting VRRP.  If the TS is a bridge, it will send/receive frames with the MAC address of all the entities it is bridging traffic for.  Some routers use different MAC addresses on different VLANs.

LK> Is this a problem because you are assuming different external NVEs will support different VNs?  My assumption is that any VN can be terminated by any external NVE.  If not, I'm not sure what the problem is.

Secondly, VNIC Containment Relationship in the figure you described above introduces a little bit confusing, since you didn't distinguish the entity or objects from identifiers since vNIC and VN are both objects while MAC address and IP address are both identifier.

LK> I don't see why a MAC or IP address cannot be considered an object as well as an identifier.  Yes, VNICs and VNs need identifiers, but I think they are separate from the addresses in frames/packets that may pass over them.

So I proposed to change the figure you described above as follows:

             +---------------------+
             |                     |
             |  VM (Tenant System) |
             |                     |
             +-+--------+--------+-+
               |        |        |
               |        |        |
               |        |        |
               +        + ...... +
         vNIC1/pNIC1  vNIC2     vNICx
               |
               |
      |--------+------+------+
      |        |      |      |
      |        |      |      |
     VN1      VN2    VN3    Non-VN

LK> I don't see any distinction between a TSI for a vNIC vs pNIC vs some other type of TSI.  They all need to at least one VN, otherwise what is the point of knowing their MAC or IP addresses?

LK> I would not assume that in all cases.

---

From: Anoop Ghanwani <[email-masked]>
Date: Thu, 18 Apr 2013 16:46:16 -0700

Hi Larry,

Maybe it's about the way I interpreted the picture and maybe that's not
accurate and if so you can correct me.

A - All traffic coming out of a vNIC is assumed to be from one TSI.
B - Multiple VLANs provisioned on the vNIC, each corresponds to a different
TSI.
C - (i) Multiple MAC addresses coming out a vNIC, and each corresponds to a
different TSI.
      (ii) Multiple MAC+VLAN coming out of a vNIC, and each corresponds to
a different TSI.
D - Multiple IP addresses provisioned with the same MAC/VLAN, each
corresponds to a different TSI. (Mulinetting case -- I guess the use case
isn't that strong.)

I think it's definitely simpler to design for each TSI mapping to exactly
one VAP on one NVE.

Anoop

On Thu, Apr 18, 2013 at 1:41 PM, Larry Kreeger (kreeger)
<[email-masked]>wrote:

>  Hi Anoop,
>
>  What benefit do you see in defining the TSI as point C or D?  Do you
> agree that the TSI should directly correspond to the VAP of the NVE?  If
> so, it seems like point B corresponds to it.
>
>  Thanks, Larry
>
> Cc: Qin Wu <[email-masked]>, David Black <[email-masked]>, "
> [email-masked]" <[email-masked]>, "Reith, Lothar" <[email-masked]>
> On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger) <
> [email-masked]> wrote:
>
>>  Hi Qin,
>>
>> I think figure 2 well match what you said below. please confirm whether
>> my understanding is correct.****
>>
>> ** **
>>
>> Regards!****
>>
>> -Qin****
>>
>> *发件人:[email-masked] [mailto:[email-masked]<[email-masked]>]
>> don't think it is always limited to that. ****
>>
>> ** **
>>
>> On Tue, Apr 16, 2013 at 3:32 PM, Black, David <[email-masked]>
>> wrote:****
>>

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Fri, 19 Apr 2013 00:31:53 +0000

Hi Anoop,

For C below, it is (ii).  For D, this can happen if the TS is acting as a load balancer with multiple VIPs, or if it is forwarding traffic for other TS (e.g. performing NAT).

I agree with you that keeping each TSI corresponding to one VAP on one NVE keeps everything simple and straight forward.

Thanks, Larry

Date: Thursday, April 18, 2013 4:46 PM

Hi Larry,

Maybe it's about the way I interpreted the picture and maybe that's not accurate and if so you can correct me.

A - All traffic coming out of a vNIC is assumed to be from one TSI.
B - Multiple VLANs provisioned on the vNIC, each corresponds to a different TSI.
C - (i) Multiple MAC addresses coming out a vNIC, and each corresponds to a different TSI.
      (ii) Multiple MAC+VLAN coming out of a vNIC, and each corresponds to a different TSI.
D - Multiple IP addresses provisioned with the same MAC/VLAN, each corresponds to a different TSI. (Mulinetting case -- I guess the use case isn't that strong.)

I think it's definitely simpler to design for each TSI mapping to exactly one VAP on one NVE.

Anoop

On Thu, Apr 18, 2013 at 1:41 PM, Larry Kreeger (kreeger) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“

”

[Qin]: Yes.

LK3> Yes

---

From: Anoop Ghanwani <[email-masked]>
Date: Thu, 18 Apr 2013 18:24:04 -0700

Hi Larry,

I think for C, (i) should also come into play.  Otherwise, how would we
cover the case of no VLAN, but just multiple MAC addresses being
used from the vNIC.  I agree (ii) would be the more common case.

Thanks,
Anoop

On Thu, Apr 18, 2013 at 5:31 PM, Larry Kreeger (kreeger)
<[email-masked]>wrote:

>  Hi Anoop,
>
>  For C below, it is (ii).  For D, this can happen if the TS is acting as
> a load balancer with multiple VIPs, or if it is forwarding traffic for
> other TS (e.g. performing NAT).
>
>  I agree with you that keeping each TSI corresponding to one VAP on one
> NVE keeps everything simple and straight forward.
>
>  Thanks, Larry
>
> On Thu, Apr 18, 2013 at 1:41 PM, Larry Kreeger (kreeger) <
>> [email-masked]" <[email-masked]>, "Reith, Lothar" <[email-masked]
>> >
>>>  A VM is one possible implementation of a Tenant System. I have not
>>> seen anyone suggesting that a Tenant System get more granular than that.
>>>
>>> we can easy to guarantee that each TSI is associated with only one NVE.*
>>> ***
>>>
>>> ** **
>>>
>>> >       firewall, etc. It belongs to a single tenant and connects to
>>> one or****
>>>
>>> the more traditional definition of an internet host such as in RFC 1122.
>>> ****
>>>
>>>  ****
>>>
>>>

---

From: Qin Wu <[email-masked]>
Date: Fri, 19 Apr 2013 07:09:58 +0000

Hi, Larry:

From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Friday, April 19, 2013 5:38 AM
To: Qin Wu; Anoop Ghanwani; Black, David
Cc: [email-masked]; Reith, Lothar
Subject: Re: 答复: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi Qin,

 - Larry

Hi, :

Regards!
-Qin

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

[Qin]: Sort of agree, but IP address still should be carried even for layer 2 service.

[Qin]: I agree with one tenant system may have one pNIC and one ore multiple vNICs, in this case, pNIC and vNIC will choose different MAC address, then one tenant system may use more than one MAC address. However for one vNIC or one pVNIC, each vNIC or pNIC still use one unique MAC address. vNIC may change its own MAC address, but at the same time, one vNIC will not use more than one MAC addresses, in my understanding.

[Qin]: It is up to whether one vNIC correspond to one TSI or multiple TSI when this vNIC is assigned with multiple IP addresses.
If it is former, when one vNIC connect to multiple VNs through multiple NVEs using each IP address, each address corresponds to the same TSI, then one TSI is associated with more than one NVE.
However if it is latter, then there is no problem, since each TSI correspond to only one NVE connecting to one VN.

[Qin]: Sorry for bad wording. I think I should say in the figure, you didn’t distinguish network or network element from identifier to network or network elment,
Examples of Network element identifier are MAC address, IP address, VLAN Tag, Example of network identifier is VNID.

Summarizing what we discussed above, I think it is better for use to choose TSI Identifier to identify each TSI or pNIC or vNIC. pNIC should be distinct from vNIC, althouth they may belong to the same Tenant System.
TSI Identifier could be combination of  VNID, MAC address, IP address. That will be consistent with we having TSI at either point B, Point C and point D. Does this make sense to you?
Here is my proposed change to the figure to distinct Network element/Network from Identifier of Network element/Network.

       +-------------------------+
       |                         |
       |    VM (Tenant System)   |
       |                         |
       +-+-----+-----+-------+---+
         |     |     |       |
         |     |     |       |
         |     |     |       |
         +     +     +   ... +
       pNIC1 vNIC1  vNIC2   vNICx
         |     |     |
      +--+     |     |
      |        |     |
      |        |     |
     PN        |    VN3
               |
         |-----+--+
         |        |
         |        |
        VN1      VN2

Tenant System Interfaces(TSI):
TSIa [     ,MAC addr1,IP addr1]corresponding to pNIC1

TSIb [VNID1,MAC addr2,IP addr2]corresponding to vNIC2
TSIc [VNID2,MAC addr2,IP addr3]corresponding to vNIC2

TSId [VNID3,MAC addr3,IP addr4]corresponding to vNIC3

Legend:
PN --  Physical Network
               Figure 2. VM information Hierarchy
For more details, see my updated http://tools.ietf.org/html/draft-wu-nvo3-nve2nve-04 ,section 4.

             +---------------------+
             |                     |
             |  VM (Tenant System) |
             |                     |
             +-+--------+--------+-+
               |        |        |
               |        |        |
               |        |        |
               +        + ...... +
         vNIC1/pNIC1  vNIC2     vNICx
               |
               |
      |--------+------+------+
      |        |      |      |
      |        |      |      |
     VN1      VN2    VN3    Non-VN

[Qin]:  VNID can be used to distinct vNIC from pNIC since vNIC will choose VNID as part of TSI Identifier while pNIC not. Maybe there are other information or parameter that can be used to distinct vNIC from pNIC.

[Qin]: I agree with you, see above. Suppose one tenant system has one pNIC and two vNIC, one tenant system can have three MAC address for each of them, but each vNIC or each pNIC should only have one MAC addresses.

---

From: Qin Wu <[email-masked]>
Date: Fri, 19 Apr 2013 07:17:14 +0000

+1,
Point B is a perfect reference point. But I am not sure we only use VNID to distinct one TSI from another, we may use combination of VNID, MAC address and IP address.
e.g., is there any case we use two TSIs belong to the same vNIC connect to the same VN, In this case, both TSIs use the same VNID.

Regards!
-Qin
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Friday, April 19, 2013 4:41 AM
To: Anoop Ghanwani
Cc: Qin Wu; Black, David; [email-masked]; Reith, Lothar
Subject: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Fri, 19 Apr 2013 09:55:18 -0400

I would put the TSI at point B (NIC + VN), but with one important change - IMHO, we should think in terms of VLANs at that interface, not virtual networks.  The TSI concept that we need attaches to one virtual network instance (end system analog to the VAP).  Here’s an attempt to walk through this.

The starting point is that a physical server with an untagged NIC attaches to exactly one virtual network instance at the NVE, therefore the TSI is the NIC.  Ditto for a virtual machine with an untagged vNIC - that untagged vNIC is the TSI (and I like Dave Allen’s approach of using NIC as the general term w/vNIC as a specialization).

Let’s call that vNIC-VAP communication path a “virtual link”; the next topic is what happens when the hypervisor uses VLANs or some other mechanism to trunk multiple untagged virtual links on a single physical link.  The TSI is still the far end of the virtual link and the VAP is still the other end, but there is now mux/demux functionality in both the NVE and End Device to handle this trunking.

Making VLAN tagging visible to the Tenant End System and providing it as part of the L2 virtual network service complicates things, and these need to be looked at independently.

When VLAN tagging is visible at the Tenant End system (i.e., NIC or vNIC is VLAN-tagged), but is *not* part of the L2 virtual network service, each VLAN is attached to a different virtual network instance at the NVE, so the TSI would be a <NIC, VID [VLAN ID]> pair.  This is an important case that will occur in practice.

Adding VLAN tagging to the L2 virtual network service results in at least a couple of possibilities.  The simplest one is that the VLAN tags from the NIC are end-to-end within the encapsulation, and we’re back to the TSI being the NIC which is associated with one virtual network instance via one VAP.

We might want to stop here, at least for the problem statement because things start to get complex beyond here.

The next step is to generalize the mux/demux of virtual links onto the End Device-NVE physical link to in effect allow VLAN tagging of those virtual links.  The result is that the VLANs used by the NIC in the Tenant End System get grouped by virtual network that they’re attached to (i.e., which VAP is used on the NVE), and the TSI would be a NIC plus a set of VIDs.  That’s the fully general definition with the explicit note that the set of VIDs is often empty or contains one VID.

For example, at a vNIC, VLANs 1-6 might attach to VNI A, VLAN 7 to VNI B, VLAN 8 to VNI C and VLANs 9-10 to VNI D.

The next wrinkle is that it’s quite possible for multiple virtual links that are trunked onto a common physical link to want to use the same VIDs, which will lead to having to remap VIDs as part of the mux/demux for trunking (at both ends in full generality).  I’d really prefer not to go there in the problem statement and framework drafts ;-).

Finally, I think the use of IP addresses or MAC addresses *to identify virtual network instances* is a mistake, and hence points C and D are bad ideas because the desired TSI-VAP virtual link is scoped to a virtual network instance.

Thanks,
--David

From: Qin Wu [mailto:[email-masked]]
Sent: Friday, April 19, 2013 3:17 AM
To: Larry Kreeger (kreeger); Anoop Ghanwani
Cc: Black, David; [email-masked]; Reith, Lothar
Subject: RE: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Carter Bullard <[email-masked]>
Date: Fri, 19 Apr 2013 11:03:03 -0400

My biggest issue is going to be TSI accountability.  If its a distinct thing, has a configuration, consumes resources, controls or modulates traffic dynamics or modifies content,  I believe that I need to know how to uniquely identify and reference it, to account for it, its function and its control.   The distinct thing should provide unambiguous, consistent, identifiability, at least within the administrative domain.

Hope all is most excellent,
Carter

On Apr 19, 2013, at 9:55 AM, "Black, David" <[email-masked]> wrote:

> Cc: Qin Wu <[email-masked]>, David Black <[email-masked]>, "[email-masked]" <[email-masked]>, "Reith, Lothar" <[email-masked]>
> On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger) <[email-masked]> wrote:
> Hi Qin,
>  
> To: Larry Kreeger <[email-masked]>, Anoop Ghanwani <[email-masked]>, David Black <[email-masked]>
> 
> 发件人:[email-masked] [mailto:[email-masked]] 代表 Larry Kreeger (kreeger)
> 发送时间: 2013年4月17日 7:58
> From:[email-masked] [mailto:[email-masked]] On Behalf Of Reith, Lothar
> Von:[email-masked] [mailto:[email-masked]] Im Auftrag von Qin Wu

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Fri, 19 Apr 2013 21:14:20 +0000

Hi David,

Thanks for your thoughts.  My responses are inline.  I think people may be getting confused because the TSI may look different depending on the point of view.  From the point of view of a TS that is a VM, the TSI may look like a NIC, or a NIC+VLAN ID; however, from the point of view of the hypervisor that is signaling this to an external NVE, it may look different (e.g. a different VLAN ID).  An external NVE should not care what the local VLAN ID is between the hypervisor vSwitch and the TS vNIC.

 - Larry

From: <Black>, David Black <[email-masked]<mailto:[email-masked]>>
Date: Friday, April 19, 2013 6:55 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [nvo3] What's a TSI? It depends ...

I would put the TSI at point B (NIC + VN),

LK> Based on most responses I've seen, I think we are converging on this.

but with one important change - IMHO, we should think in terms of VLANs at that interface, not virtual networks.

LK> I think it depends on the Tenant System implementation.  Internal to the TS, it can use whatever it likes to identify the VN.  However, when it comes to the Hypervisor-to-NVE signaling, I think the VN should be identified by the VN Name, not a VLAN ID.  Through the protocol negotiation, a VLAN ID for that VN can be temporarily assigned by the NVE.  My point is that there can be different VN identifiers for the control plane from the data plane.

The TSI concept that we need attaches to one virtual network instance (end system analog to the VAP).  Here’s an attempt to walk through this.

LK> You lost with with that last one (VLAN 9-10 to VNI D).  It seems that this would cause the two VLANs to be merged from the point of view of the TS.

LK> I don't know that TSI needs to be in the problem statement at all.  I think the framework should define the TSI, but you may be right that it doesn't need this level of detail in the framework if it ends up in the architecture document.  Anyway, the fact that VIDs become a very link local concept (whether local to the TS NIC to hypervisor vSwitch, or local between the hypervisor and the external NVE).  This is why VNs in the hypervisor-to-NVE protocol need to identify the VN by a Name and not a VID.

Thanks,
--David

Cc: Black, David; [email-masked]<mailto:[email-masked]>; Reith, Lothar

+1,

Regards!
-Qin
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Friday, April 19, 2013 4:41 AM
To: Anoop Ghanwani
Cc: Qin Wu; Black, David; [email-masked]<mailto:[email-masked]>; Reith, Lothar
Subject: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin
发件人:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] 代表Larry Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送:[email-masked]<mailto:[email-masked]>; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Qin Wu <[email-masked]>
Date: Sat, 20 Apr 2013 04:00:37 +0000

Hi, David:
My interpretation of VLAN ID is one logical interface identifier.
I agree VLAN ID or such logical interface identifier  is very useful element since it could be used to associate tenant system with NVE and associate tunnel from NVE to the final destination to the access link between tenant system and NVE.
In some cases, we need extra attribute to identify access link between tenant system and NVE, that could be VLAN ID, in additional  it provide more granularity to distinguish different access link and mapping different virtual link to the same access link.

Regards!
-Qin
From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Friday, April 19, 2013 9:55 PM
To: [email-masked]
Subject: [nvo3] What's a TSI? It depends ...

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Sun, 21 Apr 2013 11:15:40 +0000

Pat,

thanks for this clarification.

After a bit more thought I believe that

-          Yes, NIC bonding is fairly common in Virtualization host environments (Hypervisors such as XEN, KVM, VMware, Hyper-V), but we may have to differentiate two levels of NIC bonding – which I unfortunately did not do when I wrote below mail.

-          I believe we may have to differentiate between what the authority controlling the hypervisor can do when setting up the virtualization host (which applies to all tenants using that virtualization host, e.g. a possible policy that VMs get always loadshared across all available physical NICs for better availability  and what a tenant administrator interacting with the Virtualization environment management system can do (perhaps via a self service portal selecting that only a subset of the physical NICs on that virtualization host participate in the loadsharing), and what an application developer can do, for whom it must be transparent, if his application runs on a bare metal server or on a VM.

-          Because of this transparency requirement, a VNIC must look to the application developer like a NIC.

Depending on the final definition of TSI, it looks as if the loadsharing algorithms always lead that all ingress frames from a TSI always come out of  the same physical NIC.

However, I read that it may be possible that loadsharing across multiple IPs may be possible.

We may want to consider, that a LAG defined at the application layer from inside the VM SHOULD affect the next layer 2 hop, the  layer2 hop between the TSI and the Virtual Switch in the Hypervisor. This hop does not really implement multiple physical paths, nor do these multiple parallel layer 2 paths  have any failure risk which would not apply to all other parallel paths. Therefore, putting a LAG there does not make much sense, unless one would proxy the desired behavior in a non-standard way to the second layer 2 hop, between the v-Switch and the physical NIC.  In theory, this could be a proprietary implementation of the hypervisor vendor, or a policy decision of the hypervisor authority, or a policy decision of the tenant if and how such proxy works. I am not sure which hypervisor vendors, or which Cloud operators offer which options, and if they allow their tenants to establish different policies on a per VM basis.

But I know that the framework document would be well  advised to introduce some diagrams and some reference points, in order to facilitate more clarity and less confusion.

Along this line I recommend to change the definition of the term “tenant”system. I would not include routers and firewalls as tenant systems because they are either hosts (from the perspective of the datacenter or cloud system operator) or they are a VM (running a software application such as a firewall).

This would reduce the tenant system to two options (reference points), either a bare metal tenant system (a server connectet to the network like a host)  or a virtualized tenant system (a VM).

And it would help the discussions here – I find it hard to believe what discussions appear to be necessary about the nature of the TSI.

Lothar

MfG……Lothar Reith

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Pat Thaler
Gesendet: Mittwoch, 17. April 2013 23:00
An: Qin Wu; Reith, Lothar; Larry Kreeger (kreeger); [email-masked]
Betreff: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Regards,
Pat

Regards!
-Qin

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Tue, 23 Apr 2013 00:18:13 +0000

Hi There

Late to the party, I hope I have enough right to contribute somthing useful....

Now whether we are discussing a pNIC, a vNIC, VLAN tagged NIC teaming etc. should not matter (although I agree that NIC teaming with vNICs is a non-sequitur although it is perfectly valid in the general case).

As (and please correct me if I am wrong) a VAP binds to a single VNI, you would need an additional definition for the NIC to VAP link which if in the customer VLAN tagged case would permit multiple TSIs to be reached via a single NIC and to be demultiplexed to multiple VAPs. The NIC function being analogous to a UNI-C in MEF terms and the ethernet part of the VAP being analogous to the UNI-N function. Which also pulls through a number of useful service defintions...

Looking at the above and looking at Dave's message I think I'm in agreement with the proviso that the cardinalities of vNIC to VAP does not line up. It is potentially a 1:N relationship. So if we are actually discussing a TSI as an "interface" it would appear to be a dotted line between the VM stack binding to a NIC (or NIC+tag or LAG or LAG+tag) and the VAP although this could be painfully decomposed further ;-) key concept is the one end is how the VM telescopes the inferred binding to the VAP via choice of interface or tag/interface. All of the variations are simply deployment or implementation options.

I hope this helps
Dave

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Friday, April 19, 2013 6:55 AM
To: [email-masked]
Subject: [nvo3] What's a TSI? It depends ...

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Tue, 23 Apr 2013 00:56:24 +0000

Hi David,

My responses are inline.  - Larry

From: David Allan I <[email-masked]<mailto:[email-masked]>>
Date: Monday, April 22, 2013 5:18 PM
To: David Black <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] What's a TSI? It depends ...

Hi There

As (and please correct me if I am wrong) a VAP binds to a single VNI, you would need an additional definition for the NIC to VAP link which if in the customer VLAN tagged case would permit multiple TSIs to be reached via a single NIC and to be demultiplexed to multiple VAPs.

LK> I think I agree, but I'm not sure exactly what you mean by "an additional definition" - are you referring to the TSI as that definition?

The NIC function being analogous to a UNI-C in MEF terms and the ethernet part of the VAP being analogous to the UNI-N function. Which also pulls through a number of useful service defintions...

Looking at the above and looking at Dave's message I think I'm in agreement with the proviso that the cardinalities of vNIC to VAP does not line up. It is potentially a 1:N relationship.

LK> Yes, I agree.  It can be 1:N in the case where there is a hypervisor vSwitch in between the TSI and the VAP that is doing the mux/demux.

So if we are actually discussing a TSI as an "interface" it would appear to be a dotted line between the VM stack binding to a NIC (or NIC+tag or LAG or LAG+tag) and the VAP although this could be painfully decomposed further ;-) key concept is the one end is how the VM telescopes the inferred binding to the VAP via choice of interface or tag/interface. All of the variations are simply deployment or implementation options.

LK> We are assuming that the physical link connecting a hypervisor vSwitch to an external NVE is plain old 12 bit VLAN tags, so when a frame arrives at the external NVE, it doesn't necessarily know what TSI it originated from (unless there is some control plane signaling about the source address(es) in use on the TSI).

I hope this helps
Dave

To: [email-masked]<mailto:[email-masked]>

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Tue, 23 Apr 2013 11:33:37 +0200

Hi David,

Is there already a common understanding of the terms NIC, vNIC, TSI and
how these terms relate to each other?

I think, it would be good to summarize the statements received so far in
regard to this topic.

Thanks,

Florian

>>> "Black, David" <[email-masked]> 19.04.13 15.56 Uhr >>>

I would put the TSI at point B (NIC + VN), but with one important change
- IMHO, we should think in terms of VLANs at that interface, not virtual
networks.  The TSI concept that we need attaches to one virtual network
instance (end system analog to the VAP).  Here’s an attempt to walk
through this.

The starting point is that a physical server with an untagged NIC
attaches to exactly one virtual network instance at the NVE, therefore
the TSI is the NIC.  Ditto for a virtual machine with an untagged vNIC -
that untagged vNIC is the TSI (and I like Dave Allen’s approach of using
NIC as the general term w/vNIC as a specialization).

Let’s call that vNIC-VAP communication path a “virtual link”; the next
topic is what happens when the hypervisor uses VLANs or some other
mechanism to trunk multiple untagged virtual links on a single physical
link.  The TSI is still the far end of the virtual link and the VAP is
still the other end, but there is now mux/demux functionality in both
the NVE and End Device to handle this trunking.

Making VLAN tagging visible to the Tenant End System and providing it as
part of the L2 virtual network service complicates things, and these
need to be looked at independently.

When VLAN tagging is visible at the Tenant End system (i.e., NIC or vNIC
is VLAN-tagged), but is *not* part of the L2 virtual network service,
each VLAN is attached to a different virtual network instance at the
NVE, so the TSI would be a <NIC, VID [VLAN ID]> pair.  This is an
important case that will occur in practice.

Adding VLAN tagging to the L2 virtual network service results in at
least a couple of possibilities.  The simplest one is that the VLAN tags
from the NIC are end-to-end within the encapsulation, and we’re back to
the TSI being the NIC which is associated with one virtual network
instance via one VAP.

We might want to stop here, at least for the problem statement because
things start to get complex beyond here.

The next step is to generalize the mux/demux of virtual links onto the
End Device-NVE physical link to in effect allow VLAN tagging of those
virtual links.  The result is that the VLANs used by the NIC in the
Tenant End System get grouped by virtual network that they’re attached
to (i.e., which VAP is used on the NVE), and the TSI would be a NIC plus
a set of VIDs.  That’s the fully general definition with the explicit
note that the set of VIDs is often empty or contains one VID.

For example, at a vNIC, VLANs 1-6 might attach to VNI A, VLAN 7 to VNI
B, VLAN 8 to VNI C and VLANs 9-10 to VNI D.

The next wrinkle is that it’s quite possible for multiple virtual links
that are trunked onto a common physical link to want to use the same
VIDs, which will lead to having to remap VIDs as part of the mux/demux
for trunking (at both ends in full generality).  I’d really prefer not
to go there in the problem statement and framework drafts ;-).

Finally, I think the use of IP addresses or MAC addresses *to identify
virtual network instances* is a mistake, and hence points C and D are
bad ideas because the desired TSI-VAP virtual link is scoped to a
virtual network instance.

Thanks,
--David

 

+1,

Point B is a perfect reference point. But I am not sure we only use VNID
to distinct one TSI from another, we may use combination of VNID, MAC
address and IP address.
to the same VN, In this case, both TSIs use the same VNID.

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

From: Anoop Ghanwani <[email-masked]>
Date: Wednesday, April 17, 2013 8:01 PM
To: Larry Kreeger <[email-masked]>
Cc: Qin Wu <[email-masked]>, David Black <[email-masked]>,
"[email-masked]" <[email-masked]>, "Reith, Lothar"
<[email-masked]>
Subject: Re: [nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes

Larry, 

In reference to the figure below, I think all of them could be valid
TSIs depending on the application.

Anoop

On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger)
<[email-masked]> wrote:

Hi Qin,

 

Regarding what defines a TSI, someone could choose any branch in the
tree below to define as the TSI.  I was assuming that the TSI would be
at point A below in the tree.  However, point B would correspond the the
VAP.  Pat had mentioned having the TSI at point D.  I think if we were
to define the TSI to be either point C or point D, then it would depend
on the service being given (L2 or L3).  I would prefer it to be at point
A or B.

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

From: Qin Wu <[email-masked]>
Date: Tuesday, April 16, 2013 7:14 PM
To: Larry Kreeger <[email-masked]>, Anoop Ghanwani
<[email-masked]>, David Black <[email-masked]> 

Cc: "[email-masked]" <[email-masked]>, "Reith, Lothar"
<[email-masked]>

 

Hi, Larry:

(a)If a VM has only one vNIC, then vNIC that belongs to the VM will be
dealt with as one tenant system. Each vNIC corresponds to one TSI.

(b) If a VM has 5 vNICs and each vNIC is assigned with only one IP
address, each vNIC that belongs to the same VM will be treated as one
Tenant System. Each vNIC corresponds to one TSI. Then one VM have 5
Tenant system. Each Tenant System has one TSI.

(c) If a VM has 5 vNICs and each vNIC is assigned with 2 IP addresses,
each vNIC that belongs to the same VM will be treated as one Tenant
system, each vNIC with each IP address corresponds to one TSI. Then one
VM has 5 tenant system. Each Tenant system has 2 TSIs.

That is to say if the relation between vNIC and TSi is one to more, then
we can easy to guarantee that each TSI is associated with only one NVE.

However if one VM is simply regarded as one tenant system and each vNIC
with multiple IP addresses belonging to one VM is treated as only one
TSI in (b)(c), take (c) as an example. when one vNIC is assigned with
multiple IP addresses and connect to multiple VNs through multiple NVE,
then one TSI has multiple IP addresses and will be associated                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

I think figure 2 well match what you said below. please confirm whether
my understanding is correct.

Regards!

-Qin

发件人:[email-masked] [mailto:[email-masked]] 代表 Larry
Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送: [email-masked]; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

 

 - Larry

From: Anoop Ghanwani <[email-masked]>
Date: Tuesday, April 16, 2013 4:47 PM
To: David Black <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "Reith, Lothar"
<[email-masked]>
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

When I think of TSI, I think of something different than a vNIC (or, or
in the case of non-virtualized systems, a NIC).  Of course, it's
possible to create an implementation where a TSI is always a vNIC (or
NIC), but I don't think it is always limited to that. 

 

Anoop

wrote:

I think this may reflect some confusion around possible meanings of
“interface”.  I believe that the notion of “interface” that is
desired here corresponds to a vNIC, not a bonded set of vNICs (and
likewise to a NIC, not a bonded set of NICs).  If that clarification is
clear, I think TSI would be a reasonable term.

Thanks,
--David

From:[email-masked] [mailto:[email-masked]] On Behalf Of
Reith, Lothar
Sent: Monday, April 15, 2013 7:18 AM
To: Qin Wu; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

 AFAIK it is common practice to use vNIC bonof these 2 vNICs ( NIC-Bonding) creates a single Interface of the VM
which survives the outage of one of the both physical NICs.

Lothar

Von:[email-masked] [mailto:[email-masked]] Im Auftrag von
Qin Wu
Gesendet: Montag, 15. April 2013 09:17
An: Larry Kreeger (kreeger); [email-masked]
Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

and physical networks.

>[Qin]: Not sure about that, the definition of “Tenant system” in
Framework said:

“

>       Tenant System: A physical or virtual system that can play the
role

>       firewall, etc. It belongs to a single tenant and connects to one
or

”

>So tenant system can be a host and host one or multiple VMs on it. What
am I missing?

>LK2> I think that you are assuming that "host" is synonymous with
"Hypervisor".  In the definition above, I believe the term host relates
to the more traditional definition of an internet host such as in RFC
1122.

definition of End device.

LK3> Only servers or server blades that are running hypervisors can host
multiple VMs.  When we mean hypervisor we must say hypervisor (what form
factor of server it is running on is irrelevant).  All hypervisors run
on servers, 

[Qin]: Yes.

[Qin]: Are you saying that not all server are running hypervisor
software? If yes, I agree. If not, would you like to clarify a little
bit?

Suppose one tenant A have 2 VMs resided in the server1 or hypervisor1.
Tenant B have 3 VMs resided in the server 1. 

 

LK3> Yes

or multiple VMs of each tenant sharing the same Server belong to the
same single tenant system?  i.e., Tenant System A corresponding to
Tenant A contains 2 VMs. Tenant System B corresponding to Tenant B
contains 3 VMs.

>From one moment to the next a VM may migrate to a different server where
it becomes separated from the other VMs.  Furthermore, I see no
advantage in defining a Tenant System to be a grouping of VMs from a
networking perspective since the thing that connects multiple VMs
together is the network.

[Qin]: Good point, I fully agree with you. Here is the update to
http://www.ietf.org/internet-drafts/draft-wu-nvo3-nve2nve-03.txt, (NVO3
control plane requirement mostly for NVE to NV Authority/Controller
interface) which clears terminology confusing based on our discussion.

---

From: "Black, David" <[email-masked]>
Date: Tue, 23 Apr 2013 14:49:46 -0400

Hi Florian,

> Is there already a common understanding of the terms NIC, vNIC, TSI
> and how these terms relate to each other?  I think it would be good
> to summarize the statements received so far in regard to this topic.

IMHO, yes and no ☺.  I think the terms NIC, pNIC and vNIC are relatively
well understood:

- pNIC: Physical network interface (e.g., NIC card).
- vNIC: Virtualized version of a pNIC.  My usage of this term
      always refers to a virtual hardware interface presented by
      a hypervisor to virtual machines (VMs), not anything higher in
a VM’s OS stack (or lower in the hypervisor).
- NIC: General term encompassing both pNICs and vNICs.

OTOH, TSI is more difficult, IMHO, as indicated by the title of this
thread ;-).  Here’s my take on that, which I don’t promise to be much
more than IMHO ...

I believe the goal is that TSI:VAP cardinality be 1:1, so Dave Allen’s
observation on NIC cardinality is relevant (Dave is implicitly assuming
that the vNIC is VLAN-tagged):

> the cardinalities of vNIC to VAP does not line up. It is potentially a
> 1:N relationship.

That’s complemented by Larry Kreeger’s observation on how the mux/demux
works on the VAP side (based on VLAN tags, aka VIDs, no surprise there):

> LK> We are assuming that the physical link connecting a hypervisor
> vSwitch to an external NVE is plain old 12 bit VLAN tags, so when a
> frame arrives at the external NVE, it doesn't necessarily know what
> TSI it originated from

Beyond that, there are two complicating factors that I strongly prefer to
leave out of the definition of a TSI, based on the amount of confusion
they have caused in discussion (much heat, not much light):

- Addresses, both MAC and IP
- Link Aggregation, whether above or below the NICs (below NIC
      case covers vNIC usage of aggregated pNICs).

Hence my proposal for the definition of a TSI starts from:

> When VLAN tagging is visible at the Tenant End system (i.e., NIC or
> vNIC is VLAN-tagged), but is *not* part of the L2 virtual network
> service, each VLAN is attached to a different virtual network instance
> at the NVE, so the TSI would be a <NIC, VID [VLAN ID]> pair.  This is
> an important case that will occur in practice.

Restating that proposal:

- If the NIC is not VLAN-tagged, the NIC is a TSI.
- If the NIC is VLAN-tagged, the TSI is the NIC plus a VID.

I’d prefer to stop there, as that suffices when the L2 network service
provided by the nvo3 overlay is not VLAN-tagged.

If the L2 network service provided by the overlay is VLAN-tagged, the
VLAN-tagged NIC case above needs generalization.  For clarity “L2 network
service provided by the overlay is VLAN-tagged” means that the encapsulated
inner L2 frames exchanged between NVEs are VLAN-tagged.  At an NVE, a VAP
for such a network sends and receives frames to/from a corresponding NIC
on multiple VLANs, not just one. The upshot is that the generalization
of TSI for this case is that the TSI is a NIC plus a set of VIDs, namely
those VIDs that the VAP is using.

That generalization complicates explaining what’s going on - my first
choice is to leave it out of the general drafts, but allow solution drafts
to explain it.  Alternatively, if it needs to be explained in the general
drafts, I think it’s important to cover the “L2 network service provided
by the nvo3 overlay is not VLAN-tagged” case first (TSI is the NIC plus
a VID) before generalizing.

Does that make sense?

Thanks,
--David

p.s.  Of course, once this is settled, the next question will be how it
applies to L3 network services provided by an nvo3 overlay, if at all.
For my part, I hope the L3VPN people are watching this with mild bemusement
based on  the principle that IP addresses plus routing information sort
this all out at the NVE for L3VPNs ... I hope ;-).

From: Florian Mahr [mailto:[email-masked]]
Sent: Tuesday, April 23, 2013 5:34 AM
To: Black, David
Cc: [email-masked]
Subject: Antw: [nvo3] What's a TSI? It depends ...

Hi David,

Is there already a common understanding of the terms NIC, vNIC, TSI and how these terms relate to each other?
I think, it would be good to summarize the statements received so far in regard to this topic.

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Tue, 23 Apr 2013 20:59:05 +0000

If we can say a LAG is a form of NIC for the general case (e.g. bare metal servers or other directly attached non-virtualized devices), then I am happy with this.

thanks
Dave

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Tuesday, April 23, 2013 11:50 AM
To: Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Florian,

> to summarize the statements received so far in regard to this topic.

IMHO, yes and no :).  I think the terms NIC, pNIC and vNIC are relatively
well understood:

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Tue, 23 Apr 2013 23:24:03 +0200

Hi David,

Thank you very much for your summary. Please see my comments below
marked as "FM".

Regards,

Florian

>>> "Black, David" <[email-masked]> 23.04.13 20.50 Uhr >>>

Hi Florian,

> to summarize the statements received so far in regard to this topic.

IMHO, yes and no J.  I think the terms NIC, pNIC and vNIC are relatively

well understood:

FM> I totally agree with this definition.

more than IMHO ...

I believe the goal is that TSI:VAP cardinality be 1:1,

FM> I agree with you that the model will be much simpler if there is a

1:1 relationship between a TSI on the TS side and a corresponding VAP

on the NVE side.

so Dave Allen’s

> TSI it originated from

Beyond that, there are two complicating factors that I strongly prefer
to

> at the NVE, so the TSI would be a <NIC, VID [VLAN ID]> pair.

> This is an important case that will occur in practice.

Restating that proposal:

FM> I think, this definition makes sense.

Since there is a 1:1 relationship between the TSI and the VAP, this

definition should also apply to the VAP on the NVE side, although it

might be better to use the term "port" here instead of NIC?

VLAN-tagged NIC case above needs generalization.  For clarity “L2
network

service provided by the overlay is VLAN-tagged” means that the
encapsulated

inner L2 frames exchanged between NVEs are VLAN-tagged.  At an NVE, a
VAP

FM> Where do you see a necessity to support a VLAN-tagged L2 service?

That generalization complicato explain it.  Alternatively, if it needs to be explained in the
general

Does that make sense?

FM> I think so.

Thanks,
--David

For my part, I hope the L3VPN people are watching this with mild
bemusement

 

 

Hi David,

 

Thanks,

Florian

The next wrinkle is that it’s quite possible for multiple virtual links
that are trunked onto a common physical link to wantfor trunking (at both ends in full generality).  I’d really prefer not
to go there in the problem statement and framework drafts ;-).

Thanks,
--David

 

+1,

e.g., is there any case we use two TSIs belong to the same vNIC connect
to the same VN, In this case, both TSIs use the same VNID.

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

(b) If a VM has 5 vNICs and each vNIC is assigned with only one IP
address, each vNIC that belongs to the same VM will be treated as one
Tenant System. Each vNIC system, each vNIC with each IP address corresponds to one TSI. Then one
VM has 5 tenant system. Each Tenant system has 2 TSIs.

then one TSI has multiple IP addresses and will be associated with
multiple NVEs.

Here is two figure2, figure 1 shows one to one relation between vNIC and
TSI, figure 2 shows one to more relation between vNIC and TSI.

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

On Tue“interface”.  I believe that the notion of “interface” that is desired
here corresponds to a vNIC, not a bonded set of vNICs (and likewise to a
NIC, not a bonded set of NICs).  If that clarification is clear, I think
TSI would be a reasonable term.

Thanks,
--David

 

 AFAIK it is common practice to use vNIC bonding (LAG) for high
availability, where two vNICs of one VM are associated with different
physical NICs of the virtualization host running the VMs. Creating a LAG
of these 2 vNICs ( NIC-Bonding) creates a single Interface of the VM
which survives the outage of one of the both physical NICs.

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

[Qin]: Good point, I fully agree with you. interface) which clears terminology confusing based on our discussion.

---

From: "Black, David" <[email-masked]>
Date: Tue, 23 Apr 2013 18:45:24 -0400

Dave,

As long as the LAG is part of the physical NIC hardware (e.g., firmware on a dual-ported NIC card) or its virtual counterpart, I’m happy to consider it to be part of the NIC.

OTOH, I think LAG implemented in OS or hypervisor software above the NIC is on the “other side” of the network interface and hence is part of the OS or hypervisor, independent of whether LAG is implemented in a NIC device driver or in a device-independent fashion.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of David Allan I
Sent: Tuesday, April 23, 2013 4:59 PM
To: Black, David; Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

thanks
Dave

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Tue, 23 Apr 2013 22:59:50 +0000

Rather than sweat every corner case we can dream up (and there are a lot of 'em)...

Lets go with your TSI is a NIC or a NIC/VID combination...declare others as "out of scope".

I'm fine with that!

cheers
Dave

________________________________
From: Black, David [mailto:[email-masked]]
Sent: Tuesday, April 23, 2013 3:45 PM
To: David Allan I
Cc: [email-masked]
Subject: RE: [nvo3] Antw: What's a TSI? It depends ...

Dave,

Thanks,
--David

thanks
Dave

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: joel jaeggli <[email-masked]>
Date: Tue, 23 Apr 2013 16:00:05 -0700

On 4/23/13 3:45 PM, Black, David wrote:
>
> Dave,
>
> As long as the LAG is part of the physical NIC hardware (e.g., 
> firmware on a dual-ported NIC card) or its virtual counterpart, I’m 
> happy to consider it to be part of the NIC.
>
>
It isn't even that way on a router or switch...

802.1ax is a method for logical grouping of links.
>
> OTOH, I think LAG implemented in OS or hypervisor software above the 
> NIC is on the “other side” of the network interface and hence is part 
> of the OS or hypervisor, independent of whether LAG is implemented in 
> a NIC device driver or in a device-independent fashion.
>
> Thanks,
> --David
>
> Of *David Allan I
> *Sent:* Tuesday, April 23, 2013 4:59 PM
> *To:* Black, David; Florian Mahr
> *Cc:* [email-masked]
> *Subject:* Re: [nvo3] Antw: What's a TSI? It depends ...
>
> If we can say a LAG is a form of NIC for the general case (e.g. bare 
> metal servers or other directly attached non-virtualized devices), 
> then I am happy with this.
>
> thanks
>
> Dave
>
> ------------------------------------------------------------------------
>
> Of *Black, David
> *Sent:* Tuesday, April 23, 2013 11:50 AM
> *To:* Florian Mahr
> *Cc:* [email-masked]
> *Subject:* Re: [nvo3] Antw: What's a TSI? It depends ...
>
> Hi Florian,
>
> *From:*Florian Mahr [mailto:[email-masked]]
> *Sent:* Tuesday, April 23, 2013 5:34 AM
> *To:* Black, David
> *Cc:* [email-masked]
> *Subject:* Antw: [nvo3] What's a TSI? It depends ...
>
> Hi David,
>
> and how these terms relate to each other?
>
> I would put the TSI at point B (NIC + VN), but with one important 
> change - IMHO, we should think in terms of VLANs at that interface, 
> not virtual networks. The TSI concept that we need attaches to one 
> virtual network instance (end system analog to the VAP). Here’s an 
> attempt to walk through this.
>
> the TSI is the NIC. Ditto for a virtual machine with an untagged vNIC 
> - that untagged vNIC is the TSI (and I like Dave Allen’s approach of 
> using NIC as the general term w/vNIC as a specialization).
>
> mechanism to trunk multiple untagged virtual links on a single 
> physical link. The TSI is still the far end of the virtual link and 
> the VAP is still the other end, but there is now mux/demux 
> functionality in both the NVE and End Device to handle this trunking.
>
> Making VLAN tagging visible to the Tenant End System and providing it 
> as part of the L2 virtual network service complicates things, and 
> these need to be looked at independently.
>
> vNIC is VLAN-tagged), but is **not** part of the L2 virtual network 
> least a couple of possibilities. The simplest one is that the VLAN 
> tags from the NIC are end-to-end within the encapsulation, and we’re 
> back to the TSI being the NIC which is associated with one virtual 
> network instance via one VAP.
>
> to (i.e., which VAP is used on the NVE), and the TSI would be a NIC 
> plus a set of VIDs. That’s the fully general definition with the 
> explicit note that the set of VIDs is often empty or contains one VID.
>
> The next wrinkle is that it’s quite possible for multiple virtual 
> links that are trunked onto a common physical link to want to use the 
> same VIDs, which will lead to having to remap VIDs as part of the 
> mux/demux for trunking (at both ends in full generality). I’d really 
> prefer not to go there in the problem statement and framework drafts ;-).
>
> Finally, I think the use of IP addresses or MAC addresses **to 
> identify virtual network instances** is a mistake, and hence points C 
> and D are bad ideas because the desired TSI-VAP virtual link is scoped 
> to a virtual network instance.
>
> Thanks,
> --David
>
> *From:*Qin Wu [mailto:[email-masked]]
> *Sent:* Friday, April 19, 2013 3:17 AM
> *To:* Larry Kreeger (kreeger); Anoop Ghanwani
> *Cc:* Black, David; [email-masked]; Reith, Lothar
> *Subject:* RE: [nvo3] 答复: 答 复: 答 复: 答 复: NVO3 Terminology changes
>
> +1,
>
> Point B is a perfect reference point. But I am not sure we only use 
> VNID to distinct one TSI from another, we may use combination of VNID, 
> MAC address and IP address.
>
> e.g., is there any case we use two TSIs belong to the same vNIC 
> connect to the same VN, In this case, both TSIs use the same VNID.
>
> Regards!
>
> -Qin
>
> *From:*Larry Kreeger (kreeger) [mailto:[email-masked]]
> *Sent:* Friday, April 19, 2013 4:41 AM
> *To:* Anoop Ghanwani
> *Cc:* Qin Wu; Black, David; [email-masked]; Reith, Lothar
> *Subject:* Re: [nvo3] 答复: 答 复: 答 复: 答 复: NVO3 Terminology changes
>
> Hi Anoop,
>
> agree that the TSI should directly correspond to the VAP of the NVE? 
> If so, it seems like point B corresponds to it.
>
> Thanks, Larry
>
> *From: *Anoop Ghanwani <[email-masked] 
> <mailto:[email-masked]>>
> *Date: *Wednesday, April 17, 2013 8:01 PM
> *To: *Larry Kreeger <[email-masked] <mailto:[email-masked]>>
> *Cc: *Qin Wu <[email-masked] <mailto:[email-masked]>>, David 
> Black <[email-masked] <mailto:[email-masked]>>, 
> <mailto:[email-masked]>>, "Reith, Lothar" <[email-masked] 
> <mailto:[email-masked]>>
> *Subject: *Re: [nvo3] 答 复: 答复: 答复: 答复: NVO3 Terminology changes
>
> Larry,
>
> <[email-masked] <mailto:[email-masked]>> wrote:
>
> Hi Qin,
>
> at point A below in the tree. However, point B would correspond the 
> the VAP. Pat had mentioned having the TSI at point D. I think if we 
> were to define the TSI to be either point C or point D, then it would 
> depend on the service being given (L2 or L3). I would prefer it to be 
> at point A or B.
>
> VNIC-+-VN-+-MAC-+-IP
>
> ^ | | +-IP ...
>
> | | |
>
> | | +-MAC-+-IP
>
> | | +-IP ...
>
> | |
>
> | +-VN-+-MAC-+-IP
>
> | ^ | +-IP ...
>
> | | |
>
> | | +-MAC-+-IP
>
> | | ^ +-IP ...
>
> | | | ^
>
> | | | |
>
> A B C D
>
> - Larry
>
> *From: *Qin Wu <[email-masked] <mailto:[email-masked]>>
> *Date: *Tuesday, April 16, 2013 7:14 PM
> *To: *Larry Kreeger <[email-masked] <mailto:[email-masked]>>, 
> Anoop Ghanwani <[email-masked] <mailto:[email-masked]>>, 
> David Black <[email-masked] <mailto:[email-masked]>>
>
>
> *Cc: *"[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>, "Reith, Lothar" <[email-masked] 
> <mailto:[email-masked]>>
>
> *Subject: *[nvo3] 答复: 答复: 答复: 答复: NVO3 Terminology changes
>
> Hi, Larry:
>
> system, each vNIC with each IP address corresponds to one TSI. Then 
> one VM has 5 tenant system. Each Tenant system has 2 TSIs.
>
> That is to say if the relation between vNIC and TSi is one to more, 
> then we can easy to guarantee that each TSI is associated with only 
> one NVE.
>
> However if one VM is simply regarded as one tenant system and each 
> vNIC with multiple IP addresses belonging to one VM is treated as only 
> one TSI in (b)(c), take (c) as an example. when one vNIC is assigned 
> with multiple IP addresses and connect to multiple VNs through 
> multiple NVE, then one TSI has multiple IP addresses and will be 
> associated with multiple NVEs.
>
> Here is two figure2, figure 1 shows one to one relation between vNIC 
> and TSI, figure 2 shows one to more relation between vNIC and TSI.
>
> +---------------------+
>
> | |
>
> | VM (Tenant System) |
>
> | |
>
> +-+--------+--------+-+
>
> | | |
>
> | | |
>
> | | |
>
> + + ...... +
>
> *发件人**:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *代表 *Larry Kreeger (kreeger)
> *发送 时间:* 2013年4月17日 7:58
> *收件 人:* Anoop Ghanwani; Black, David
> *抄送:* [email-masked] <mailto:[email-masked]>; Reith, Lothar
> *主题:* Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes
>
> Hi Anoop,
>
> interface on a single tenant's system (Tenant System). My assumption 
> is that each TSI is associated with only one NVE, so that there is a 
> one to one correspondence between a TSI and a VAP on the NVE. It is 
> what is connected to the bottom lines in figure 4 of 
> *From: *Anoop Ghanwani <[email-masked] 
> <mailto:[email-masked]>>
> *To: *David Black <[email-masked] <mailto:[email-masked]>>
> *Cc: *"[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>, "Reith, Lothar" <[email-masked] 
> <mailto:[email-masked]>>
> *Subject: *Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes
>
> When I think of TSI, I think of something different than a vNIC (or, 
> or in the case of non-virtualized systems, a NIC). Of course, it's 
> “interface”. I believe that the notion of “interface” that is desired 
> here corresponds to a vNIC, not a bonded set of vNICs (and likewise to 
> a NIC, not a bonded set of NICs). If that clarification is clear, I 
> think TSI would be a reasonable term.
>
> Thanks,
> --David
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked] <mailto:[email-masked]>] *On 
> Behalf Of *Reith, Lothar
> <mailto:[email-masked]>
> *Subject:* Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes
>
> physical NICs of the virtualization host running the VMs. Creating a 
> LAG of these 2 vNICs ( NIC-Bonding) creates a single Interface of the 
> VM which survives the outage of one of the both physical NICs.
>
> Lothar
>
> *Von:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked] <mailto:[email-masked]>] *Im 
> Auftrag von *Qin Wu
> *An:* Larry Kreeger (kreeger); [email-masked] <mailto:[email-masked]>
> *Betreff:* [nvo3] 答复: 答复: 答复: NVO3 Terminology changes
>
> LK3> Only servers or server blades that are running hypervisors can 
> host multiple VMs. When we mean hypervisor we must say hypervisor 
> (what form factor of server it is running on is irrelevant). All 
> hypervisors run on servers,
>
> [Qin]: Yes.
>
> happen to be running on the same server hardware is an ephemeral 
> thing. From one moment to the next a VM may migrate to a different 
> server where it becomes separated from the other VMs. Furthermore, I 
> see no advantage in defining a Tenant System to be a grouping of VMs 
> from a networking perspective since the thing that connects multiple 
> VMs together is the network.
>
> http://www.ietf.org/internet-drafts/draft-wu-nvo3-nve2nve-03.txt, 
> (NVO3 control plane requirement mostly for NVE to NV 
> Authority/Controller interface) which clears terminology confusing 
> based on our discussion.
>
>

---

From: "Black, David" <[email-masked]>
Date: Tue, 23 Apr 2013 19:07:08 -0400

Hi Florian,

I hope other people agree that this looks like progress.  Here are a
couple of follow-ups:

[1]
> might be better to use the term "port" here instead of NIC?

I’m fine with using “port” as the name of the VLAN-tagged entity on
the NVE side that demuxes to non-VLAN-tagged VAPs.  I’d suggest
“NVE port” as a more specific term.

[2]
> FM> Where do you see a necessity to support a VLAN-tagged L2 service?

Well, both the VXLAN and NVGRE allow the inner VLAN tag that results
in an VLAN-tagged L2 service, although both drafts discourage its
use (the VXLAN draft says that “frames with the inner VLAN tag SHOULD
be discarded unless configured otherwise”, the NVGRE draft says that
the “inner Ethernet header of NVGRE SHOULD NOT contain inner VLAN Tag”).

In any case, I’ll leave it to other people and drafts to make the case
for where/how/why a VLAN-tagged L2 service would be useful.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Florian Mahr
Sent: Tuesday, April 23, 2013 5:24 PM
To: Black, David
Cc: [email-masked]
Subject: [nvo3] Antw: RE: Antw: What's a TSI? It depends ...

Hi David,

Thank you very much for your summary. Please see my comments below marked as "FM".

Regards,

Florian
Hi Florian,

so Dave Allen’s

Restating that proposal:

Does that make sense?

FM> I think so.

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 01:17:19 +0000

Hi David,

I agree that the discussion of “L2 network service provided by the overlay is VLAN-tagged” below is an added complication that I would like to avoid (not cover for now).

Regarding the TSI:VAP cardinality, I just want to make sure we are in sync that:

 - A TSI can only be associated with a single VAP, but
 - A single VAP can be associated with multiple TSI

I would also like to keep link aggregation / NIC Teaming out of this since there are multiple modes that teaming can work in.  Aggregation / Teaming can be in a mode where both sides are aware of it so they can coordinate its operation (e.g. LACP), or in a mode where only one side is aware of the team (NIC teaming modes such as Transmit Load Balancing, Receive Load Balancing, MAC pinning).

 - Larry

From: <Black>, David Black <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 23, 2013 11:49 AM
To: Florian Mahr <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian
>>> "Black, David" <[email-masked]<mailto:[email-masked]>> 19.04.13 15.56 Uhr >>>

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Yizhou Li <[email-masked]>
Date: Wed, 24 Apr 2013 10:11:19 +0800

Hi Larry,

A bit confused on the following.

If “L2 network service provided by the overlay is VLAN-tagged”  is left
out, in what case we can have

“ - A single VAP can be associated with multiple TSI”?

Yizhou

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Larry Kreeger (kreeger)
Sent: Wednesday, April 24, 2013 9:17 AM
To: Black, David; Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi David,

I agree that the discussion of “L2 network service provided by the overlay
is VLAN-tagged” below is an added complication that I would like to avoid
(not cover for now).

Regarding the TSI:VAP cardinality, I just want to make sure we are in sync
that:

I would also like to keep link aggregation / NIC Teaming out of this since
there are multiple modes that teaming can work in.  Aggregation / Teaming
can be in a mode where both sides are aware of it so they can coordinate its
operation (e.g. LACP), or in a mode where only one side is aware of the team
(NIC teaming modes such as Transmit Load Balancing, Receive Load Balancing,
MAC pinning).

 - Larry

From: <Black>, David Black <[email-masked]>
Date: Tuesday, April 23, 2013 11:49 AM
To: Florian Mahr <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Florian,

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

Hi David,

Is there already a common understanding of the terms NIC, vNIC, TSI and how
these terms relate to each other?

Thanks,

Florian

>>> "Black, David" <[email-masked]> 19.04.13 15.56 Uhr >>>

I would put the TSI at point B (NIC + VN), but with one important change -
IMHO, we should think in terms of VLANs at that interface, not virtual
networks.  The TSI concept that we need attaches to one virtual network
instance (end system analog to the VAP).  Here’s an attempt to walk through
this.

The starting point is that a physical server with an untagged NIC attaches
to exactly one virtual network instance at the NVE, therefore the TSI is the
NIC.  Ditto for a virtual machine with an untagged vNIC - that untagged vNIC
is the TSI (and I like Dave Allen’s approach of using NIC as the general
term w/vNIC as a specialization).

Let’s call that vNIC-VAP communication path a “virtual link”; the next
topic is what happens when the hypervisor uses VLANs or some other mechanism
to trunk multiple untagged virtual links on a single physical link.  The TSI
is still the far end of the virtual link and the VAP is still the other end,
but there is now mux/demux functionality in both the NVE and End Device to
handle this trunking.

Making VLAN tagging visible to the Tenant End System and providing it as
part of the L2 virtual network service complicates things, and these need to
be looked at independently.

When VLAN tagging is visible at the Tenant End system (i.e., NIC or vNIC is
VLAN-tagged), but is *not* part of the L2 virtual network service, each VLAN
is attached to a different virtual network instance at the NVE, so the TSI
would be a <NIC, VID [VLAN ID]> pair.  This is an important case that will
occur in practice.

Adding VLAN tagging to the L2 virtual network service results in at least a
couple of possibilities.  The simplest one is that the VLAN tags from the
NIC are end-to-end within the encapsulation, and we’re back to the TSI
being the NIC which is associated with one virtual network instance via one
VAP.

The next step is to generalize the mux/demux of virtual links onto the End
Device-NVE physical link to in effect allow VLAN tagging of those virtual
links.  The result is that the VLANs used by the NIC in the Tenant End
System get grouped by virtual network that they’re attached to (i.e., which
VAP is used on the NVE), and the TSI would be a NIC plus a set of VIDs.
That’s the fully general definition with the explicit note that the set of
VIDs is often empty or contains one VID.

For example, at a vNIC, VLANs 1-6 might attach to VNI A, VLAN 7 to VNI B,
VLAN 8 to VNI C and VLANs 9-10 to VNI D.

The next wrinkle is that it’s quite possible for multiple virtual links
that are trunked onto a common physical link to want to use the same VIDs,
which will lead to having to remap VIDs as part of the mux/demux for
trunking (at both ends in full generality).  I’d really prefer not to go
there in the problem statement and framework drafts ;-).

Finally, I think the use of IP addresses or MAC addresses *to identify
virtual network instances* is a mistake, and hence points C and D are bad
ideas because the desired TSI-VAP virtual link is scoped to a virtual
network instance.

Thanks,
--David

+1,

Point B is a perfect reference point. But I am not sure we only use VNID to
distinct one TSI from another, we may use combination of VNID, MAC address
and IP address.

e.g., is there any case we use two TSIs belong to the same vNIC connect to
the same VN, In this case, both TSIs use the same VNID.

Regards!

-Qin

Hi Anoop,

What benefit do you see in defining the TSI as point C or D?  Do you agree
that the TSI should directly correspond to the VAP of the NVE?  If so, it
seems like point B corresponds to it.

Thanks, Larry

"[email-masked]" <[email-masked]>, "Reith, Lothar" <[email-masked]>

Larry, 

Anoop

On Wed, Apr 17, 2013 at 9:55 AM, Larry Kreeger (kreeger) <[email-masked]>
wrote:

Hi Qin,

Regarding what defines a TSI, someone could choose any branch in the tree
below to define as the TSI.  I was assuming that the TSI would be at point A
below in the tree.  However, point B would correspond the the VAP.  Pat had
mentioned having the TSI at point D.  I think if we were to define the TSI
to be either point C or point D, then it would depend on the service being
given (L2 or L3).  I would prefer it to be at point A or B.

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

Hi, Larry:

(a)If a VM has only one vNIC, then vNIC that belongs to the VM will be dealt
with as one tenant system. Each vNIC corresponds to one TSI.

(b) If a VM has 5 vNICs and each vNIC is assigned with only one IP address,
each vNIC that belongs to the same VM will be treated as one Tenant System.
Each vNIC corresponds to one TSI. Then one VM have 5 Tenant system. Each
Tenant System has one TSI.

(c) If a VM has 5 vNICs and each vNIC is assigned with 2 IP addresses, each
vNIC that belongs to the same VM will be treated as one Tenant system, each
vNIC with each IP address corresponds to one TSI. Then one VM has 5 tenant
system. Each Tenant system has 2 TSIs.

That is to say if the relation between vNIC and TSi is one to more, then we
can easy to guarantee that each TSI is associated with only one NVE.

However if one VM is simply regarded as one tenant system and each vNIC with
multiple IP addresses belonging to one VM is treated as only one TSI in
(b)(c), take (c) as an example. when one vNIC is assigned with multiple IP
addresses and connect to multiple VNs through multiple NVE, then one TSI has
multiple IP addresses and will be associated with multiple NVEs.

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

I think figure 2 well match what you said below. please confirm whether my
understanding is correct.

Regards!

-Qin

发件人:[email-masked] [mailto:[email-masked]] 代表 Larry
Kreeger (kreeger)
发送时间: 2013年4月17日 7:58
收件人: Anoop Ghanwani; Black, David
抄送:[email-masked]; Reith, Lothar
主题: Re: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

Hi Anoop,

I definitely see a TSI as belonging to a single tenant, since it is an
interface on a single tenant's system (Tenant System).  My assumption is
that each TSI is associated with only one NVE, so that there is a one to one
correspondence between a TSI and a VAP on the NVE.  It is what is connected
to the bottom lines in figure 4 of
http://tools.ietf.org/html/draft-ietf-nvo3-framework-02

 - Larry

Anoop

I think this may reflect some confusion around possible meanings of
“interface”.  I believe that the notion of “interface” that is desired
here corresponds to a vNIC, not a bonded set of vNICs (and likewise to a
NIC, not a bonded set of NICs).  If that clarification is clear, I think TSI
would be a reasonable term.

Thanks,
--David

survives the outage of one of the both physical NICs.

Lothar

Von:[email-masked] [mailto:[email-masked]] Im Auftrag von Qin
Wu
Gesendet: Montag, 15. April 2013 09:17
An: Larry Kreeger (kreeger); [email-masked]
Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

>>I believe one tenant system can host multiple VMs, each VM may have
multiple vNIC adapters that it uses to communicate with both the virtual and
physical networks.

Framework said:

“

”

device.

servers, 

[Qin]: Yes.

[Qin]: Are you saying that not all server are running hypervisor software?
If yes, I agree. If not, would you like to clarify a little bit?

LK3> Yes

LK3> No, that would be a really bad idea IMO.  The fact that two VMs happen
to be running on the same server hardware is an ephemeral thing.  From one
moment to the next a VM may migrate to a different server where it becomes
separated from the other VMs.  Furthermore, I see no advantage in defining a
Tenant System to be a grouping of VMs from a networking perspective since
the thing that connects multiple VMs together is the network.

[Qin]: Good point, I fully agree with you. Here is the update to http://www.
ietf.org/internet-drafts/draft-wu-nvo3-nve2nve-03.txt, (NVO3 control plane
requirement mostly for NVE to NV Authority/Controller interface) which
clears terminology confusing based on our discussion.

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 03:08:52 +0000

Hi Yizhou,

If understand what David wrote correctly, the below quoted phrase means a case where VLAN tagged frames are carried inside the encapsulation.  So, if there is a VLAN tagged link between a vSwitch and the NVE, then it would carry a double-tagged (Q-in-Q) frame. I would prefer not to go there.

If that was not your understanding of what David wrote, then please help me to better understand.

Thanks, Larry

From: Yizhou Li <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 23, 2013 7:11 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>, 'Florian Mahr' <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Larry,

If “L2 network service provided by the overlay is VLAN-tagged”  is left out, in what case we can have
“ - A single VAP can be associated with multiple TSI”?

Yizhou

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Yizhou Li <[email-masked]>
Date: Wed, 24 Apr 2013 14:43:07 +0800

Hi Larry,

My original assumption was if we do want to support “L2 network service
provided by the overlay is VLAN-tagged” , then it is more likely to have
(NIC+ a group of VIDs) associating to a single VAP. If every member VID of
that group is considered as a TSI, that would be a good example of  “ - A
single VAP can be associated with multiple TSI”. Since we do not want to go
that far, it would be not so important to discuss what should be the best
definition of TSI or whether Q-in-Q should be used between vSwitch and NVE
in this particular scenario anyway.  

My confusion part was how could single VAP associating multiple TSI? Are you
referring the following case, TSI1 and TSI2 from different servers(VMs)
connecting to the same VN via the same (external) NVE, so they are accessing
the same VAP? 

I had thought TSI was interface facing NVE located on tenant system and VAP
was the corresponding interface facing tenant system located on (external)
NVE, so they were 1:1 mapping. But I might be wrong.

Thanks.

Yizhou

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Larry Kreeger (kreeger)
Sent: Wednesday, April 24, 2013 11:09 AM
To: Yizhou Li; 'Black, David'; 'Florian Mahr'
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Yizhou,

If understand what David wrote correctly, the below quoted phrase means a
case where VLAN tagged frames are carried inside the encapsulation.  So, if
there is a VLAN tagged link between a vSwitch and the NVE, then it would
carry a double-tagged (Q-in-Q) frame. I would prefer not to go there.

If that was not your understanding of what David wrote, then please help me
to better understand.

Thanks, Larry

From: Yizhou Li <[email-masked]>
Date: Tuesday, April 23, 2013 7:11 PM
To: Larry Kreeger <[email-masked]>, David Black <[email-masked]>,
'Florian Mahr' <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Larry,

Yizhou

Hi David,

 - Larry

Hi Florian,

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!

-Qin

Hi Anoop,

Thanks, Larry

Larry, 

Anoop

Hi Qin,

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

 

Lothar

Framework said:

“

”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Wed, 24 Apr 2013 12:31:56 +0200

Hi David,

Please see my answers below marked as "FM".

Thanks,

Florian

>>> "Black, David" <[email-masked]> 24.04.13 1.08 Uhr >>>

Hi Florian,

couple of follow-ups:

[1]

FM> You are right. "NVE port" would be the better term here as it is

more precise.

[2]

FM> Correct. I know these two drafts as well, and that they both
discourage

the use of an inner VLAN tag. At the moment I personally do not see a
use case

demanding a VLAN-tagged L2 service either. But maybe others have one
handy.

Thanks,
--David

Florian Mahr

Hi David,

 

Regards,

Florian

Hi Florian,

well understood:

 

more than IMHO ...

on the NVE side.

so Dave Allen’s

> vSwitch to an external NVE is plain old 12 bit VLANleave out of the definition of a TSI, based on the amount of confusion

Restating that proposal:

 

choice is to leave it out of the general drafts, but allow solution
drafts

to explain it.  Alternatively, if it needs to be explained in the
general

Does that make sense?

FM> I think so.

Thanks,
--David

 

 

Hi David,

 

Thanks,

Florian

Let’s call that vNIC-VAP communication path a link.  The TSI is still the far end of the virtual link and the VAP is
still the other end, but there is now mux/demux functionality in both
the NVE and End Device to handle this trunking.

Thanks,
--David

 

+1,

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |          TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

Thanks,
--David

 

 

Lothar

 

 

Framework said:

“

role

or

”

>So tenant system can be a host and host one or m"Hypervisor".  In the definition above, I believe the term host relates

 

[Qin]: Yes.

 

 

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Wed, 24 Apr 2013 13:54:21 +0200

Hi Larry,

I think, you are right in your first point. When the L2 network service
is VLAN tagged, the frames travelling across the trunk link between the
vSwitch and the external NVE need to be double-tagged. The inner VLAN
tag (Customer Tag [C-TAG]) would be the one added by the TS (VM), while
the
second VLAN tag (Service Tag [S-TAG]) would be the one imposed by the
vSwitch/NVE. This certainly adds to the complexity of the solution,
since you need

an additional entity (an S-VLAN component) within the hypervisor as well
as within the external NVE.

However, I do not really understand why you think the double-tagging
will lead to a situation where a single VAP is associated with several
TSIs?
In my opinion, the TSI on the TS side would be a (v)NIC plus a set of
C-VIDs. The C-tagged frames sent by this TSI are then mapped to a common
S-VLAN
on the vSwitch, tagged with an additional S-TAG carrying the S-VID for
that S-VLAN, and then sent to the external NVE across the trunk link.
On the NVE side the NVE port receiving a double-tagged frame needs to
strip off the outer S-TAG and forward the resulting C-tagged frame to
the
corresponding VAP which would be identified by the S-VID within the
S-TAG. The only difference I see here, is the fact that the VAP is now
receiving/sending C-tagged frames, i.e. the VAP is now C-VLAN-tagged.
The VAP itself is defined by the NVE port sending/receiving the S-tagged

frame plus the S-VID that is mapped to the C-VLANs used within the L2
VNI the VAP is attached to. Nevertheless, there is still a bidirectional
1:1

relationship between the VAP on the NVE side and the TSI on the TS side,
I think. Or did I get something wrong?

Thanks,

Florian

>>> "Larry Kreeger (kreeger)" <[email-masked]> 24.04.13 5.09 Uhr >>>

Hi Yizhou,

If understand what David wrote correctly, the below quoted phrase means
a case where VLAN tagged frames are carried inside the encapsulation. 
So, if there is a VLAN tagged link between a vSwitch and the NVE, then
it would carry a double-tagged (Q-in-Q) frame. I would prefer not to go
there.

If that was not your understanding of what David wrote, then please help
me to better understand.

Thanks, Larry

From: Yizhou Li <[email-masked]>
Date: Tuesday, April 23, 2013 7:11 PM
To: Larry Kreeger <[email-masked]>, David Black
<[email-masked]>, 'Florian Mahr'
<[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Larry,

Yizhou

 

Hi David,

I agree that the discussion of “L2 network service provided by the
overlay is VLAN-tagged” below is an added complication that I would like
to avoid (not cover for now).

Regarding the TSI:VAP cardinality, I just want to make sure we are in
sync that:

 

I would also like to keep link aggregation / NIC Teaming out of this
since there are multiple modes that teaming can work in.  Aggregation /
Teaming can be in a mode where both sides are aware of it so they can
coordinate its operation (e.g. LACP), or in a mode where only one side
is aware of the team (NIC teaming modes such as Transmit Load Balancing,
Receive Load Balancing, MAC pinning).

 - Larry

From: <Black>, David Black <[email-masked]>
Date: Tuesday, April 23, 2013 11:49> Is there already a common understanding of the terms NIC, vNIC, TSI

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

Hi David,

 

Thanks,

Florian

Thanks,
--David

 

+1,

Point B is a perfect reference point. But I am not sure we only use VNID
to dise.g., is there any case we use two TSIs belong to the same vNIC connect
to the same VN, In this case, both TSIs use the same VNID.

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

Thanks,
--David

 

 

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

---

From: Lucy yong <[email-masked]>
Date: Wed, 24 Apr 2013 13:57:38 +0000

Here is my 2 cent below.

 - A TSI can only be associated with a single VAP, but
 - A single VAP can be associated with multiple TSI
[Lucy] This is my understanding too.

[Lucy] In fact, a single VAP may be associated with one or multiple TSIs. For the latter case, TSs first attach to a L2 network that attaches to the VAP. The L2 network is identified by the vlan tag on the frame. Thus, TSI is the NIC plus vlan id;  multiple TSIs associated with a single VAP are distinguished by the NIC, not the vlan id. NVE receives the frames from one port. The frames may contain different vlan ids. One vlan id maps to one VAP or vise versa.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 11:11:49 -0400

Hi Larry,

> - A single VAP can be associated with multiple TSI

Yes, and the framework draft is clear that multiple end systems can be associated with a VAP.

> I would also like to keep link aggregation / NIC Teaming out of this ...

Good.

Thanks,
--David

From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Tuesday, April 23, 2013 9:17 PM
To: Black, David; Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 11:24:11 -0400

Hi Yizhou,

> If “L2 network service provided by the overlay is VLAN-tagged” is left out,
> in what case we can have “ - A single VAP can be associated with multiple TSI”?

Here’s an example:

a) The VAP (part of the NVE) is located in the ToR switch.  This VAP is a virtual
port for a single VNI (virtual network instance).
b) The ToR switch is connected to a server running a hypervisor.  There’s an
L2 softswitch in the hypervisor, and VLANs are used between the ToR and
the server.
c) There are multiple VMs running on that hypervisor, all of which access that
      same VNI via untagged vNICs.
d) The L2 softswitch in the hypervisor multiplexes traffic to/from those
      multiple vNICs onto one VLAN to reach the VAP in the ToR switch.

In this case the multiple TSIs are the vNICs in the VMs, and the mux/demux that
enables them to access the same VAP (virtual port in the ToR switch) for the VNI
occurs in the hypervisor’s L2 softswitch.

Thanks,
--David

From: Yizhou Li [mailto:[email-masked]]
Sent: Tuesday, April 23, 2013 10:11 PM
To: 'Larry Kreeger (kreeger)'; Black, David; 'Florian Mahr'
Cc: [email-masked]
Subject: RE: [nvo3] Antw: What's a TSI? It depends ...

Hi Larry,

Yizhou

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 11:38:09 -0400

Hi Lucy,

In the example from your message, “The L2 network is identified by the vlan tag on the frame”, so I would have said that the TSI is the vNIC + that VLAN tag value, so while the vNIC attaches to multiple VAPs, the TSI does not.

OTOH, if there’s link aggregation in a hypervisor softswitch across pNICs that are connected to two different ToR switches, then each TSI in a VM may be associated with two VAPs, one in each ToR, depending on how the link aggregation is configured.  I think we need to allow this possibility, as it’s used in practice.  Link aggregation is still not part of the TSI in this case, because it’s being done by the hypervisor softswitch, not the vNIC - the TSI is still the vNIC + VID (VLAN tag value).

Link aggregation ... why is it always link aggregation? (paraphrasing Indiana Jones on the subject of snakes ...) :-).

Thanks,
--David

From: Lucy yong [mailto:[email-masked]]
Sent: Wednesday, April 24, 2013 9:58 AM
To: Larry Kreeger (kreeger); Black, David; Florian Mahr
Cc: [email-masked]
Subject: RE: [nvo3] Antw: What's a TSI? It depends ...

Here is my 2 cent below.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Wed, 24 Apr 2013 15:52:07 +0000

If TSI-VAP cardinality is not 1:1, then at least for the L2'oFoo case it implies a bridging component between the VAP and the NIC. Which is not aligned with the VNI definition which my read implies any bridging component is located there....

So TSI:VAP is 1:1 and the VNI bridges. Or we assume a bridge between the VAP and the NIC.

cheers
Dave

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Wednesday, April 24, 2013 8:12 AM
To: Larry Kreeger (kreeger); Florian Mahr
Cc: [email-masked]
Subject: [nvo3] TSI: VAP cardinality (was RE: Antw: What's a TSI? It depends ...)

Hi Larry,

Good.

Thanks,
--David

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 11:59:51 -0400

> Or we assume a bridge between the VAP and the NIC.

Right, and I think that’s an important configuration that needs to be in scope.

When the VAP is in the ToR switch and the NIC is a vNIC in a VM, the hypervisor softswitch is a “bridging component” between the two (may not be a full bridge, see EVB), and that’s an important configuration for nvo3, IMHO.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of David Allan I
Sent: Wednesday, April 24, 2013 11:52 AM
To: Black, David; Larry Kreeger (kreeger); Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] TSI: VAP cardinality (was RE: Antw: What's a TSI? It depends ...)

cheers
Dave

Good.

Thanks,
--David

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Lucy yong <[email-masked]>
Date: Wed, 24 Apr 2013 16:11:43 +0000

Hi David,

Please see below.

From: Black, David [mailto:[email-masked]]
Sent: Wednesday, April 24, 2013 10:38 AM
To: Lucy yong
Cc: [email-masked]
Subject: RE: [nvo3] Antw: What's a TSI? It depends ...

Hi Lucy,

In the example from your message, “The L2 network is identified by the vlan tag on the frame”, so I would have said that the TSI is the vNIC + that VLAN tag value, so while the vNIC attaches to multiple VAPs, the TSI does not.
[Lucy] The former is what I mean. The last part, do you mean, vNIC attaches to multiple VAPs by using different addresses?

[Lucy] Agree.

Lucy

Thanks,
--David

Here is my 2 cent below.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Wed, 24 Apr 2013 16:13:15 +0000

That has to be true even if the VAP is in the Vswitch of N:1 is to be achieved. Hence the VAP is purely an adaptation function onto the overlay.

cheers
Dave

________________________________
From: Black, David [mailto:[email-masked]]
Sent: Wednesday, April 24, 2013 9:00 AM
To: David Allan I
Cc: [email-masked]
Subject: RE: [nvo3] TSI: VAP cardinality (was RE: Antw: What's a TSI? It depends ...)

Thanks,
--David

cheers
Dave

Good.

Thanks,
--David

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Lucy yong <[email-masked]>
Date: Wed, 24 Apr 2013 16:15:51 +0000

My understanding that NVO3 allows  L2 bridge between NIC and VAP.  That is the local access. The framework is written that way including the L2NVO.

Lucy

From: [email-masked] [mailto:[email-masked]] On Behalf Of David Allan I
Sent: Wednesday, April 24, 2013 10:52 AM
To: Black, David; Larry Kreeger (kreeger); Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] TSI: VAP cardinality (was RE: Antw: What's a TSI? It depends ...)

cheers
Dave

Good.

Thanks,
--David

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 12:24:55 -0400

One more inline ...

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Lucy yong
Sent: Wednesday, April 24, 2013 12:12 PM
To: Black, David
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi David,

Please see below.

Hi Lucy,

In the example from your message, “The L2 network is identified by the vlan tag on the frame”, so I would have said that the TSI is the vNIC + that VLAN tag value, so while the vNIC attaches to multiple VAPs, the TSI does not.
[Lucy] The former is what I mean. The last part, do you mean, vNIC attaches to multiple VAPs by using different addresses?
[DLB] Not exactly - I was thinking of multiple VLANs ... the VLAN-tagged NVE port muxes/demuxes on VLAN to/from multiple VAPs.

Lucy

Thanks,
--David

Here is my 2 cent below.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Wed, 24 Apr 2013 17:11:58 +0000

David,

Something is disturbing me with your definitions of the term NIC and vNIC.  So please forgive me that I go back to this definition and make a proposal how some of my perceived ambiguity could be eliminated.

- NIC: Physical network interface (e.g., NIC card), presented as network-side-facing interface by a server to a physical cable.

- vNIC: Virtualized version of a NIC.  Presented by a hypervisor as network-side-facing interface to a virtual yellow cable. Said virtual yellow cable (aka virtual link that may be a multipoint link)  being emulated by a V-Switch that is under the same administrative control as said hypervisor. The virtual yellow cables emulated by the V-Switch connect one vNIC to one NIC, or one vNIC to multiple NICs (NIC teaming/LAG).  In addition, said hypervisor presents a virtual hardware interface as an associated compute-side-facing interface to a Virtual Machine (VM) which is also under the administrative control of said hypervisor.

I understand that many on the list would prefer not having to consider NIC-teaming with LAG, but the market realities may not allow such a simplistic solutions that unfortunately lacks any resilience.

Pre-emptively, I would further argue, that there may be a couple of more objects, such as:

A L3-interface (network side facing) of the NIC (as well as of the VNIC)

-          As client of an underlying server layer network service: Interfacing to a service access point of an underlying layer 2 server layer network service (such as a virtual station Interface on a virtual yellow cable)

A L2-interface (network side facing) of the NIC (as well as of the VNIC)

-          As client of an underlying server layer network service: Interfacing to a service access point of an underlying layer1 server layer network service

The nice part of this definition is, that it fits both cases, when the VAP and NIC are not in the same device.e. with the VAP in the ToRswitch , and when the VAP and NIC are collocated back to back  in the hypervisor (in which case the yellow cable may be considered a collapsed yellow cable).

It is also extensible in multiple dimensions, i.e. in space when a MEF Ethernet EVC may be located in  between the VAP and the NIC, because a MEF EVC is a virtual yellow cable.

It is also extensible from point to point to multipoint, just by adding a station to the yellow cable, even if is a  collapsed virtual yellow cable inside the hypervisor.

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 17:36:57 +0000

See inline, marked with LK>. - Larry

From: Florian Mahr <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 24, 2013 4:54 AM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Antw: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Larry,

I think, you are right in your first point. When the L2 network service is VLAN tagged, the frames travelling across the trunk link between the
vSwitch and the external NVE need to be double-tagged. The inner VLAN tag (Customer Tag [C-TAG]) would be the one added by the TS (VM), while the
second VLAN tag (Service Tag [S-TAG]) would be the one imposed by the vSwitch/NVE. This certainly adds to the complexity of the solution, since you need
an additional entity (an S-VLAN component) within the hypervisor as well as within the external NVE.

However, I do not really understand why you think the double-tagging will lead to a situation where a single VAP is associated with several TSIs?

LK> I don't think the multiple TSI to one VAP is related to double-tagging.  It will exist even in the simple case.  The vSwitch multiplexes the traffic from multiple TSI onto the same physical link to the external NVE.  If two TSI on the same vSwitch are connected to the same VN, then the traffic will be tagged with the same VID on the link to the NVE, so they will share a common VAP on the NVE.

In my opinion, the TSI on the TS side would be a (v)NIC plus a set of C-VIDs. The C-tagged frames sent by this TSI are then mapped to a common S-VLAN
on the vSwitch, tagged with an additional S-TAG carrying the S-VID for that S-VLAN, and then sent to the external NVE across the trunk link.
On the NVE side the NVE port receiving a double-tagged frame needs to strip off the outer S-TAG and forward the resulting C-tagged frame to the
corresponding VAP which would be identified by the S-VID within the S-TAG. The only difference I see here, is the fact that the VAP is now
receiving/sending C-tagged frames, i.e. the VAP is now C-VLAN-tagged. The VAP itself is defined by the NVE port sending/receiving the S-tagged
frame plus the S-VID that is mapped to the C-VLANs used within the L2 VNI the VAP is attached to. Nevertheless, there is still a bidirectional 1:1
relationship between the VAP on the NVE side and the TSI on the TS side, I think. Or did I get something wrong?

Thanks,

Florian

>>> "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>> 24.04.13 5.09 Uhr >>>
Hi Yizhou,

Thanks, Larry

Hi Larry,

Yizhou

From:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 17:50:15 +0000

Hi Yizhou,

Please see below, marked with LK>.  - Larry

Date: Tuesday, April 23, 2013 11:43 PM

Hi Larry,

My original assumption was if we do want to support “L2 network service provided by the overlay is VLAN-tagged” , then it is more likely to have (NIC+ a group of VIDs) associating to a single VAP. If every member VID of that group is considered as a TSI, that would be a good example of  “ - A single VAP can be associated with multiple TSI”. Since we do not want to go that far, it would be not so important to discuss what should be the best definition of TSI or whether Q-in-Q should be used between vSwitch and NVE in this particular scenario anyway.

LK> I see what you mean…and this is one of the reasons I don't want to go there, it really complicates things and I don't see a real use case for it.

My confusion part was how could single VAP associating multiple TSI? Are you referring the following case, TSI1 and TSI2 from different servers(VMs) connecting to the same VN via the same (external) NVE, so they are accessing the same VAP?

LK> Yes.

I had thought TSI was interface facing NVE located on tenant system

LK> Yes.

and VAP was the corresponding interface facing tenant system located on (external) NVE,

LK> Yes.

so they were 1:1 mapping. But I might be wrong.

LK> I think you are missing the fact there there can be a bridge (e.g. a vSwitch) multiplexing the traffic from TSI1 and TSI2 onto a common link (sharing the same VID) to get to the external NVE.

Thanks.

Yizhou

Hi Yizhou,

Thanks, Larry

Hi Larry,

Yizhou

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 14:11:45 -0400

Lothar,

> Something is disturbing me with your definitions of the term NIC and vNIC.

Ok ...

[... snip ...]

IMHO, it’s important to distinguish definitions of concepts/objects from
the assembly/construction of solutions:

> I understand that many on the list would prefer not having to consider
> NIC-teaming with LAG, but the market realities may not allow such a
> simplistic solutions that unfortunately lacks any resilience.

Not exactly ;-).  To be precise, my position is that I would prefer
“not having to consider NIC-teaming with LAG” *to be part of the TSI*.

I agree that LAG will need to be part of a lot of solutions (there’s a
lot of it deployed out there and its functionality is important), but
I expect that LAG functionality to be part of a solution component other
than the NIC, vNIC, pNIC or TSI.  Possible locations for the LAG
functionality include the hypervisor softswitch and in an operating
system (e.g.,  device driver) running on bare metal or in a VM.

> A L2-interface (network side facing) of the NIC (as well as of the VNIC)

I’m inclined to agree in principle, in part because I’ve taken a strong
position that L2 and L3 addresses should not be part of the TSI object,
so some other objects appear to be needed.  At the level of abstraction
of this discussion, the objects you’ve proposed seem plausible.  To go
further, I’d suggest taking a look at the IF-MIB (RFC 2863) for some
relevant concepts, although nvo3 surely does not need everything that’s
in that MIB.

Returning to LAG, for LAG implemented in an OS, it would make sense for
LAG to be below the L2 interface but above the TSI.

Thanks,
--David

From: Reith, Lothar [mailto:[email-masked]]
Sent: Wednesday, April 24, 2013 1:12 PM
To: Black, David; Florian Mahr; David Allan I
Cc: [email-masked]
Subject: AW: [nvo3] Antw: What's a TSI? It depends ...

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: David Allan I <[email-masked]>
Date: Wed, 24 Apr 2013 20:54:20 +0000

I cannot envision a scenario where there would be a double tagged frame, nor where the circumstances that would result in one have utility...

cheers
Dave
________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Wednesday, April 24, 2013 10:37 AM
To: Florian Mahr; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi Larry,

Thanks,

Florian

Hi Yizhou,

Thanks, Larry

Hi Larry,
Restating that proposal:

“
”

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 23:03:47 +0000

Replies inline. - Larry

From: <Black>, David Black <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 24, 2013 11:11 AM
To: "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Lothar,

Ok ...

[... snip ...]

LK> Yes.

I’m inclined to agree in principle, in part because I’ve taken a strong
position that L2 and L3 addresses should not be part of the TSI object,
so some other objects appear to be needed.

LK> I would like to hear more about how you see L2 and/or L3 addresses relating to the TSI, if not directly, then through what other objects and what are their relationships with the TSI.  Do you see these other objects as required to be signaled between a hypervisor and an external NVE?

At the level of abstraction
of this discussion, the objects you’ve proposed seem plausible.  To go
further, I’d suggest taking a look at the IF-MIB (RFC 2863) for some
relevant concepts, although nvo3 surely does not need everything that’s
in that MIB.

LK> I think this depends on the teaming mode.  If the teaming is using 802.1ax/802.3ad then I would consider the TSI to be above the team.  If the teaming is performed by using a modes such as transmit load balancing, receive load balancing, MAC pinning (modes where the switch side of the team is really unaware that the team exists in the server), then I would expect the TSI to be below the team.

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

>LK2> I think that you are assuming that "host" is synonymous with "Hypervisor".  In the definition above, I believe the term host relates to the more traditional definition of an internet host such…

[Qin]: So “Host”in the definition of tenant system seems misleading since we two have different interpretation to it. I agree Hypersor or Server or Server blade can host multiple VMs, however in the framework document, it also said, a host can be server or server blade in the definition of End device.

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Wed, 24 Apr 2013 19:21:52 -0400

Larry,

> LK> I would like to hear more about how you see L2 and/or L3 addresses
> relating to the TSI, if not directly, then through what other objects
> and what are their relationships with the TSI.  Do you see these
> other objects as required to be signaled between a hypervisor and an
> external NVE?

I believe that the addresses that correspond to the nvo3 service do need
to be signaled - that would be MAC addresses for an L2 service and IP addresses
for an L3 service.  Some of this is in draft-kompella-nvo3-server2nve, for
which a revised version should be coming later this week (mostly to finish
off the text that I should have written before Atlanta so that it’s available
for other drafts).  I don’t think it’s necessary to signal IP addresses for
an L2 service, although I suppose one could view ARP as a signaling protocol.

Lothar’s proposal for interface objects makes sense to me in principle, as
does associating addresses with those sort of interface objects, but I
think it suffices to signal the addresses to the NVE, not the objects.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Wednesday, April 24, 2013 7:04 PM
To: Black, David; Reith, Lothar
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Replies inline. - Larry

Lothar,

Ok ...

[... snip ...]

LK> Yes.

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Apr 2013 23:28:07 +0000

Hi David,

OK, Good.  I'll look forward to reading the new server2nve.

Sometimes I have a hard time separating which objects are needed in the framework vs which are needed to be exposed in the protocols.  As you saw in the hypervisor-to-nve draft, I felt it was useful to associate the address(es) with a TSI, so I considered them to be attributes of the TSI…and I'm not sure how that is different from saying that the addresses are part of the TSI object.

 - Larry

Date: Wednesday, April 24, 2013 4:21 PM

Larry,

Thanks,
--David

Replies inline. - Larry

Lothar,

Ok ...

[... snip ...]

LK> Yes.

Thanks,
--David

David,

Lothar

From:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Black, David

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Qin Wu <[email-masked]>
Date: Thu, 25 Apr 2013 07:50:27 +0000

Hi,Larry:

From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Thursday, April 25, 2013 1:37 AM
To: Florian Mahr; [email-masked]; Liyizhou
Cc: [email-masked]
Subject: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi Larry,

[Qin]: Why using VID rather than VNID to associate multiple TSI to one VAP? Is it possible to tag each traffic from different TSIs with different VID? Who tags traffic? Tenant system or NVE or vswitch in between?

Thanks,

Florian
Hi Yizhou,

Thanks, Larry

Hi Larry,

Yizhou

Hi David,

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: Qin Wu <[email-masked]>
Date: Thu, 25 Apr 2013 08:16:56 +0000

Hi, David:

From: [email-masked] [mailto:[email-masked]] On Behalf Of Black, David
Sent: Wednesday, April 24, 2013 11:38 PM
To: Lucy yong
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Lucy,

OTOH, if there’s link aggregation in a hypervisor softswitch across pNICs that are connected to two different ToR switches, then each TSI in a VM may be associated with two VAPs, one in each ToR, depending on how the link aggregation is configured.  I think we need to allow this possibility, as it’s used in practice.  Link aggregation is still not part of the TSI in this case,

[Qin]: Agree since you may have two switches between Tenant system and NVE, one is softswitch in a hypervisor connecting to vNIC, the other is ToR switch connecting to NVE. Many VLAN can be transported between two switch. It is also called trunk link.

because it’s being done by the hypervisor softswitch, not the vNIC - the TSI is still the vNIC + VID (VLAN tag value).

[Qin]: Who assigns VID to TSI? Is VID added by either a VM or a NVE? Is VID assigned by NVA or other entity? Is the same VID used at NVE to identify VAP?

Thanks,
--David

Here is my 2 cent below.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Thu, 25 Apr 2013 10:18:42 +0200

Hi David,

Please see my comment below marked as "FM>".

Thanks,

Florian

>>> "Black, David" <[email-masked]> 24.04.13 20.13 Uhr >>>

Lothar,

Ok ...

[... snip ...]

> Pre-emptively, I would further argue, that there may be a couple of

> more objects, such as:

FM> Could you please give me more details why you think it might be
better not to treat the L2/L3

addresses as attributes of a TSI object and signal them as such in a
hypervisor-to-NVE control protocol?

in that MIB.

Thanks,
--David

 

David,

Something is disturbing me with your definitions of the term NIC and
vNIC.  So please forgive me that I go back to this definition and make a
proposal how some of my perceived ambiguity could be eliminated.

- NIC: Physical network interface (e.g., NIC card), presented as
network-side-facing interface by a server to a physical cable.

- vNIC: Virtualized version of a NIC.  Presented by a hypervisor as
network-side-facing interface to a virtual yellow cable. Said virtual
yellow cable (aka virtual link that may be a multipoint link)  being
emulated by a V-Switch that is under the same administrative control as
said hypervisor. The virtual yellow cables emulated by the V-Switch
connect one vNIC to one NIC, or one vNIC to multiple NICs (NIC
teaming/LAG).  In addition, said hypervisor presents a virtual hardware
interface as an associated compute-side-facing interface to a Virtual
Machine (VM) which is also under the administrative control of said
hypervisor.

Pre-emptively, I would further argue, that there may be a couple of more
objects, such as:

-          As client of an underlying server layer A L2-interface (network side facing) of the NIC (as well as of the VNIC)

-          As client of an underlying server layer network service:
Interfacing to a service access point of an underlying layer1 server
layer network service

The nice part of this definition is, that it fits both cases, when the
VAP and NIC are not in the same device.e. with the VAP in the ToRswitch
, and when the VAP and NIC are collocated back to back  in the
hypervisor (in which case the yellow cable may be considered a collapsed
yellow cable).

It is also extensible in multiple dimensions, i.e. in space when a MEF
Ethernet EVC may be located in  between the VAP and the NIC, because a
MEF EVC is a virtual yellow cable.

It is also extensible from point to point to multipoint, just by adding
a station to the yellow cable, even if is a  collapsed virtual yellow
cable inside the hypervisor.

Lothar

Black, David

Hi Florian,

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

 

Hi David,

 

Thanks,

Florian

nB, VLAN 8 to VNI C and VLANs 9-10 to VNI D.

Thanks,
--David

 

+1,

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

Let (b) If a VM has 5 vNICs and each vNIC is assigned with only one IP
address, each vNIC that belongs to the same VM will be treated as one
Tenant System. Each vNIC corresponds to one TSI. Then one VM have 5
Tenant system. Each Tenant System has one TSI.

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

When I think of TSI, I think of something different than a vNIC (or, or
in the case of non-virtualized systems, a NIC).  Of course, it's
possible to create an implementation where - Can a TSI be associated with multiple NVEs?

 

Anoop

Thanks,
--David

 

 

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

LK3> No, that would be a really bad idea IMO.  The fact that two VMs
happen to be running on the same advantage in defining a Tenant System to be a grouping of VMs from a
networking perspective since the thing that connects multiple VMs
together is the network.

---

From: "Florian Mahr" <[email-masked]>
Date: Thu, 25 Apr 2013 10:46:19 +0200

Hi David,

Thanks, Florian

>>> "Black, David" <[email-masked]> 25.04.13 1.23 Uhr >>>

Larry,

to be signaled - that would be MAC addresses for an L2 service and IP
addresses

for an L3 service.  Some of this is in draft-kompella-nvo3-server2nve,
for

which a revised version should be coming later this week (mostly to
finish

off the text that I should have written before Atlanta so that it’s
available

for other drafts).  I don’t think it’s necessary to signal IP addresses
for

an L2 service, although I suppose one could view ARP as a signaling
protocol.

FM> I am not really sure what you mean with "one could view ARP as a
signaling protocol", but I personally see an

advantage in signalling IP addresses also for an L2 service, because
this would provide the possibility to implement

a proxy-ARP function within the NVE. Therefore, I think we should
optionally allow this within the control protocol

whatever this protocol might look like. What do you think?

Lothar’s proposal for interface objects makes sense to me in principle,
as

 

Thanks,
--David

 

Replies inline. - Larry

To: "Reith, Lothar" <[email-masked]>

Lothar,

> Something is disturbing me with your definitions of the term NIC and
vNIC.

Ok ...

[... snip ...]

 

LK> Yes.

 

LK> I would like to hear more about how you see L2 and/or L3 addresses
relating to the TSI, if not directly, then through what other objects
and what are their relatioof this discussion, the objects you’ve proposed seem plausible.  To go

in that MIB.

 

LK> I think this depends on the teaming mode.  If the teaming is using
802.1ax/802.3ad then I would consider the TSI to be above the team.  If
the teaming is performed by using a modes such as transmit load
balancing, receive load balancing, MAC pinning (modes where the switch
side of the team is really unaware that the team exists in the server),
then I would expect the TSI to be below the team.

Thanks,
--David

 

David,

-          As client of an underlying server layer network service:
Interfacing to a service access point of an underlying layer 2 server
layer network service (such as a virtual station Interface on a virtual
yellow cable)

Lothar

Hi Florian,

> to summarize the statements r- pNIC: Physical network interface (e.g., NIC card).

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

 

Hi David,

I think, it woI would put the TSI at point B (NIC + VN), but with one important change
- IMHO, we should think in terms of VLANs at that interface, not virtual
networks.  The TSI concept that we need attaches to one virtual network
instance (end system analog to the VAP).  Here’s an attempt to walk
through this.

Thanks,
--David

 

+1,

Regards!

-Qin

SubjecWhat benefit do you see in defining the TSI as point C or D?  Do you

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

Thanks,
--David

 

 

Lothar

Von:[email-masked] [mailto:nvo3Betreff: [nvo3] 答复: 答复: 答复: NVO3 Terminology changes

 

Framework said:

“

role

or

”

am I missing?

 

[Qin]: So “Host”in the definition of tenant system seems misleading

[Qin]: Yes.

 

 

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 25 Apr 2013 09:19:02 +0000

For clarification regarding the below excerpted statements regarding the L3-Interface and L2-Interface, I like to add some clarification:

> A L2-interface (network side facing) of the NIC (as well as of the VNIC)

Clarification:
If a VNIC is present, then there is no L3-Interface present at the NIC. However, if no VNIC is present (i.e. in case of a bare metal server without hypervisor) then the L3-Interface is obviously at the NIC.

If a VNIC is present, then there is a L2-Interface present at the VNIC, and a bridge at the NIC (not sure if the bridge could be regarded as back to back L2-Interface, it does not matter as bridge is well understood).
If no VNIC is present, then there is obviously a L2 interface present at the NIC.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Florian Mahr
Gesendet: Donnerstag, 25. April 2013 10:46
An: [email-masked]; [email-masked]
Cc: [email-masked]
Betreff: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi David,

Thanks, Florian
Larry,

FM> I am not really sure what you mean with "one could view ARP as a signaling protocol", but I personally see an
advantage in signalling IP addresses also for an L2 service, because this would provide the possibility to implement
a proxy-ARP function within the NVE. Therefore, I think we should optionally allow this within the control protocol
whatever this protocol might look like. What do you think?

Thanks,
--David

Replies inline. - Larry

Lothar,

Ok ...

[... snip ...]

LK> Yes.

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 25 Apr 2013 09:34:25 +0000

I forgot to mention:

In case the NIC being a firewall that is intentionally not visible at layer 3, then there may be no  L-3 Interface present at the NIC.

Same applies to the VNIC in case the associated VM hosts a software based firewall application that is invisible at L3.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Reith, Lothar
Gesendet: Donnerstag, 25. April 2013 11:19
An: Florian Mahr; [email-masked]; [email-masked]
Cc: [email-masked]
Betreff: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Clarification:

Lothar

Hi David,

Thanks, Florian
Larry,

Thanks,
--David

Replies inline. - Larry

Lothar,

Ok ...

[... snip ...]

LK> Yes.

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Thu, 25 Apr 2013 13:29:06 +0200

Lothar,

When I interpret your message correctly, you are trying to define two
new additional objects.

Could you please give me some more details on what you are trying to
achieve with these objects,

why you are thinking that they are necessary, and how these new objects
relate to the TSI?

Thanks, Florian

>>> "Reith, Lothar" <[email-masked]> 25.04.13 11.19 Uhr >>>

For clarification regarding the below excerpted statements regarding the
L3-Interface and L2-Interface, I like to add some clarification:

Clarification:

If a VNIC is present, then there is no L3-Interface present at the NIC.
However, if no VNIC is present (i.e. in case of a bare metal server
without hypervisor) then the L3-Interface is obviously at the NIC.

If a VNIC is present, then there is a L2-Interface present at the VNIC,
and a bridge at the NIC (not sure if the bridge could be regarded as
back to back L2-Interface, it does not matter as bridge is well
understood). 

If no VNIC is present, then there is obviously a L2 interface present at
the NIC.

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von
Florian Mahr
Gesendet: Donnerstag, 25. April 2013 10:46
An: [email-masked]; [email-masked]
Cc: [email-masked]
Betreff: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi David,

 

Thanks, Florian

Larry,

 

Thanks,
--David

 

Replies inline. - Larry

 

Lothar,

vNIC.

Ok ...

[... snip ...]

IMHO, it’s important to distinguis> NIC-teaming with LAG, but the market realities may not allow such a

 

LK> Yes.

 

LK> I would like to hear more about how you see L2 and/or L3 addresses
relating to the TSI, if not directly, then through what other objects
and what are their relationships with the TSI.  Do you see these other
objects as required to be signaled between a hypervisor and an external
NVE?

in that MIB.

 

 

Thanks,
--David

 

David,

-     layer network service (such as a virtual station Interface on a virtual
yellow cable)

Lothar

Hi Florian,

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

 

Hi David,

 

Thanks,

Florian

noB, VLAN 8 to VNI C and VLANs 9-10 to VNI D.

Thanks,
--David

 

+1,

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

(a)If a address, each vNIC that belongs to the same VM will be treated as one
Tenant System. Each vNIC corresponds to one TSI. Then one VM have 5
Tenant system. Each Tenant System has one TSI.

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

Anoop

Thanks,
--David

 

 

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

>From one momnetworking perspective since the thing that connects multiple VMs
together is the network.

---

From: "Black, David" <[email-masked]>
Date: Thu, 25 Apr 2013 09:06:55 -0400

> [Qin]: Who assigns VID to TSI? Is VID added by either a VM or a NVE?
> Is VID assigned by NVA or other entity?

I think VID assignment responsibility depends on the solution design.  The VID could be assigned by the hypervisor (e.g., for an untagged vNIC whose traffic is tagged by the hypervisor softswitch), by the NVE (see draft-kompella), the VM (e.g., for a tagged vNIC), the NVA or other orchestration software.

On the server side, the VID could be added by the VM (e.g., if the vNIC is tagged) or by the hypervisor softswitch.  At the NVE, assuming that the VAP interface is (logically) untagged, the VID is added by the NVE port.

> Is the same VID used at NVE to identify VAP?

I’m not sure about “identify”, but all parties using the VID need to have a common understanding of its meaning, and for a (logically) untagged VAPs, the NVE port uses the VID to mux/demux traffic from a VLAN-tagged link to the VAPs.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Qin Wu
Sent: Thursday, April 25, 2013 4:17 AM
To: Black, David; Lucy yong
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi, David:

Hi Lucy,

Thanks,
--David

Here is my 2 cent below.

Lucy

 - Larry

Hi Florian,

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Thu, 25 Apr 2013 09:24:08 -0400

> FM> Could you please give me more details why you think it might be better not
> to treat the L2/L3 addresses as attributes of a TSI object and signal them as
> such in a hypervisor-to-NVE control protocol?

Sure, but let me start with a caveat that some of this may seem like hair-splitting,
because we’re getting into object-oriented design where the nature of relationships
is crucial.  From that perspective “attribute of” is a “part of” relationship, whereas
I view addresses as “associated with” TSI objects.  To make this distinction clear by
example:

      - My car is strongly associated with me.
      - Nonetheless, my car is not part of me. :-) :-) :-)

Regarding signaling, I believe that the addresses should be signaled, not TSIs, and
I’m a co-author of draft-kompella, which spells out some of the details of how that
could work.  That draft just expired (mea culpa - my day job’s been limiting the
time that I can put into IETF), I expect to submit a new version either today or
tomorrow (it’s written, just needs some final checks).

OTOH, I do not believe that the TSIs need to be signaled as TSI objects.  To make
this a bit more concrete, I don’t think the following two cases need to be
distinguished at the VAP or NVE for a single VAP and VNI:

- 6 VMs, one untagged vNIC each, one MAC/vNIC, total of 6 TSIs
- 3 VMs, one untagged vNIC each, two MACs/vNIC, total of 3 TSIs

VM events may implicitly distinguish these cases (e.g., one MAC vs. two MACs
affected by VM migration or VM power on/off events), but (IMHO) the signaling
protocol doesn’t have to care - the NVE and VAP just have to know that those
6 MACs on VLAN 72 are for VNI 348.

Thanks,
--David

From: [email-masked] [mailto:[email-masked]] On Behalf Of Florian Mahr
Sent: Thursday, April 25, 2013 4:19 AM
To: [email-masked]; Black, David
Cc: [email-masked]
Subject: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

FM> Could you please give me more details why you think it might be better not to treat the L2/L3
addresses as attributes of a TSI object and signal them as such in a hypervisor-to-NVE control protocol?

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 25 Apr 2013 13:57:02 +0000

Florian,

indeed, I do beleive that these objects are necessary, in actual fact even more for the complete picture, even more.

To make it simple: These Objects L2-IF and L3-IF may have different lifecycles and states, such as enabled but not in service, in service, out of service, planned to be shutdown, suspended, down, up, degraded, just to name a few candidates from which you may want to take what you need for your operational requirements. Please not, I am not saying that these all make sense, but some do.

How do you test if a network facing L2-Interface is up?   Of course with L2 OAM mechanisms, such as CFM or Service OAM.

How do you test if a network facing L3-Interface is up? I guess using Ping from the network side.

If you can test if something is up independently, then these are independent objects.  I am not trying to introduce them, I want to remind people, that they do exist.

These may be needed for assurance, for example for alarm correlation to find out, if an alarm issued by the interface  is relevant, which may depend on such state (even if the state is not held in the Network element, but in  an associated resource inventory of the operator)

Lothar

Von: Florian Mahr [mailto:[email-masked]]
Gesendet: Donnerstag, 25. April 2013 13:29
An: [email-masked]; Reith, Lothar; [email-masked]
Cc: [email-masked]
Betreff: Antw: AW: [nvo3] Antw: Re: Antw: What's a TSI? It depends...

Lothar,

When I interpret your message correctly, you are trying to define two new additional objects.
Could you please give me some more details on what you are trying to achieve with these objects,
why you are thinking that they are necessary, and how these new objects relate to the TSI?

Thanks, Florian

Clarification:

Lothar

Hi David,

Thanks, Florian
Larry,

Thanks,
--David

Replies inline. - Larry

Lothar,

Ok ...

[... snip ...]

LK> Yes.

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Thu, 25 Apr 2013 17:56:23 +0200

>> FM> Could you please give me more details why you think it might be
better not

>       - Nonetheless, my car is not part of me. :-) :-) :-)

I am not really sure if I understand what you mean. In object-oriented
design the

relationship between two classes is called "association".  In addition
to this "simple"

association there are two special kinds of association which describe a
relationship

between a whole and its parts. These two forms of associations are
called "aggregation"

and "composition". When I interpret your statement correctly, you want
to model the

addresses as separate classes and put them in a "simple" association
with the TSI class.

Is this what you mean?

> OTOH, I do not believe that the TSIs need to be signaled as TSI
objects.

What do you exactly mean with "signaled as TSI objects"?

> To make this a bit more concrete, I don’t think the following two
cases need to be

> 6 MACs on VLAN 72 are for VNI 348.

Don't you think it might be useful to signal some form of TSI Instance
Identifier

together with its associated addresses?

Thanks,

Florian

 

 

Hi David,

 

Thanks,

Florian

Lothar,

vNIC.

Ok ...

[... snip ...]

than the NIC, vNIC, pNIC or TSI.  Possible locations for the L> Pre-emptively, I would further argue, that there may be a couple of

 

in that MIB.

Thanks,
--David

 

David,

Lothar

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Black, David
Sent: Tuesday, April 23, 2013 11:> Is there already a common understanding of the terms NIC, vNIC, TSI

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

 

From: Florian Mahr [mailto:[email-masked] there already a common understanding of the terms NIC, vNIC, TSI and
how these terms relate to each other?

 

Thanks,

Florian

Thanks,
--David

 

+1,

e.g., is to the same VN, In this case, both TSIs use the same VNID.

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

Here is two figure2, figure 1 shows one to one relation between vN                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

Thanks,
--David

 

 AFAIK it is common practice to use vNIC bonding (LAG) for high
availability, where two vNICs of one VM are associated with diffewhich survives the outage of one of the both physical NICs.

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

---

From: "Black, David" <[email-masked]>
Date: Thu, 25 Apr 2013 12:25:29 -0400

Selective responses:

> When I interpret your statement correctly, you want to model the
> addresses as separate classes and put them in a "simple" association with the TSI class.
> Is this what you mean?

The addresses could be separate classes or possibly of interface classes - in both cases
a “simple” association with the TSI class would be involved.

> What do you exactly mean with "signaled as TSI objects"?

My initial answer to the following question ...

> Don't you think it might be useful to signal some form of TSI Instance Identifier

... is “no” (obviously, IMHO).

I think signaling the addresses suffices to get a lot done, and I’d want to see a
strong case made for adding functionality to signal TSIs.

Thanks,
--David

From: Florian Mahr [mailto:[email-masked]]
Sent: Thursday, April 25, 2013 11:56 AM
To: Black, David
Cc: [email-masked]
Subject: RE: RE: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

>       - Nonetheless, my car is not part of me. :-) :-) :-)

I am not really sure if I understand what you mean. In object-oriented design the
relationship between two classes is called "association".  In addition to this "simple"
association there are two special kinds of association which describe a relationship
between a whole and its parts. These two forms of associations are called "aggregation"
and "composition". When I interpret your statement correctly, you want to model the
addresses as separate classes and put them in a "simple" association with the TSI class.
Is this what you mean?

> OTOH, I do not believe that the TSIs need to be signaled as TSI objects.

> To make this a bit more concrete, I don’t think the following two cases need to be

> 6 MACs on VLAN 72 are for VNI 348.

Don't you think it might be useful to signal some form of TSI Instance Identifier
together with its associated addresses?

Thanks,
Florian

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 25 Apr 2013 18:16:45 +0000

David,

I fully agree with your statement: “Because we’re getting into object-oriented design where the nature of relationships is crucial”

Therefore I would like to ask: Are we 100% sure, that an association of multiple MAC-addresses directly with the vNIC is a good idea?

Something is disturbing me with that idea. Looking at Bob Metcalfe’s 40 year old design, it becomes obvious, that the object “station” may be missing.

Requirement1: The object “station” needs a station-name, that is not the MAC address.
Reason: To support use cases where an associated MAC-address or an associated MEP-ID has to be changed, e.g. due to a hardware failure of a NIC, but without having to tell all communication partners, which may even be in other administrative domains.

Requirement2: There needs to be a maximum of one MAC-address assigned per station, or none during times when a station does not have a MAC address assigned.
Reason: To be able to leverage existing standards for endpoint identification and authentication and to ease the IT support for that. I am thinking of DHCP and of DHCP intermediate agent (where the station-id is the identifier that identifies the object that tries to get the IP address assigned to it – I conclude therefore that the object is the station) or the PPPoE intermediate agent (where the station-ID is the identifier that identifies the object that tries to get an IP address dynamically assigned – obviously I conclude that this object is the station).

To relate that to the below: The MAC-address is strongly associated with the station, but the MAC-address is not the station.

And when an IP address gets associated to a MAC-address with DHCP, that does not mean that the IP-address identifies the MAC-address, it does also not mean that the IP address identifies the host or VM, rather one could argue the IP-address always identifies a station on a physical or virtual yellow cable, that is located at the network facing side of the NIC (or vNIC in case of VM).  On the associated compute facing side the corresponding compute facing interface object may be called “port” (at least OpenStack Quantum API2.0 calls this compute-side-facing interface “port”, and assigns a MAC-address to the port, which in reality does not get assigned to the port, but to the station on the “other side” of the interface, i.e. on the network facing side.

As a side note: One could argue, that a loopback address in a router could be considered as a degenerate case of a collapsed virtual yellow cable that is not bound to a physical NIC, but has a single station as one endpoint of a virtual yellow cable, a single interface to a gateway as another endpoint, and the length of the virtual yellow cable being zero – such that the loopback network and the loopback station and the loopback interface are collapsed into one singularity requiring only a single IP address to identify all 3 objects.

Now, what does all have to do with the title of this thread “what is a TSI?”. When looking with a compute side paradigm at this, the answer may be: a port (or in Quantum API: a Quantum network port).

When looking at this with a network side paradigm, the answer may be: a station

Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Black, David
Gesendet: Donnerstag, 25. April 2013 15:24
An: Florian Mahr
Cc: [email-masked]
Betreff: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 25 Apr 2013 19:19:38 +0000

Hi Lothar,

In the simplest case of a VM acting as a host with a vNIC, the station has one MAC address and one IP address using that MAC.  I assume this is the most common case, but I think we must also support the cases where the VM is:

 (a) acting as a bridge (perhaps while performing a service such as IDS or firewall, or acting as a L2 VPN gateway),
 (b) acting as a router (perhaps while performing a service such as IDS or firewall, or acting as an L3 VPN gateway),
 (c) acting as load balancer terminating different services on different IP addresses

In case (a), the VM's vNIC may have a MAC address that it uses for an L3 interface used for management, but there will also be frames entering and leaving the vNIC with MAC addresses that belong to other stations for which the VM is performing the bridging function.  The VM could be bridging these frames between two vNICs, or between two VNs represented as VIDs on the same vNIC.

In case (b), the VM always sends/receives frames using the vNIC MAC address, but these frames may be carrying packets with the IP addresses of stations for which the VM is performing the routing function, in addition to the VM's own IP address used for management or control plane traffic.

In case (c), the VM always sends/receives frames using the vNIC MAC address, but these frames may be carrying multiple IP addresses which are terminated on the VM.

I don't think these cases are all covered in the model you describe below.  If they are, can you help describe the relationships you are thinking of?

In the thread below, David is asserting that the NVE does not need to know what TSI any of the addresses are associated with (only the VAP).  I agree that the TSI is not strictly needed, but I think it could be useful for the following reasons.   In the cases above where there are multiple addresses associated with a TSI.  If we look at case (a), the NVE will need to forward frames towards the vNIC for all the MAC addresses for which the VM is performing the bridging function (let's say 100 addresses).  When that VM migrates to another hypervisor, the NVE will need to disassociate the 100 addresses from the NVE port leading towards the vNIC.  This could be signaled from the hypervisor to the NVE as 100 individual addresses, or it could be signaled as one TSI disassociate (if the relationship of the addresses to the TSI are known).

Thanks, Larry

From: <Reith>, Lothar <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 25, 2013 11:16 AM
To: David Black <[email-masked]<mailto:[email-masked]>>, Florian Mahr <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

David,

Lothar

Von: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] Im Auftrag von Black, David
Gesendet: Donnerstag, 25. April 2013 15:24
An: Florian Mahr
Cc: [email-masked]<mailto:[email-masked]>
Betreff: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Thanks,
--David

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Florian Mahr
Sent: Thursday, April 25, 2013 4:19 AM
To: [email-masked]<mailto:[email-masked]>; Black, David
Cc: [email-masked]<mailto:[email-masked]>
Subject: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi David,

Thanks,

Florian
>>> "Black, David" <[email-masked]<mailto:[email-masked]>> 24.04.13 20.13 Uhr >>>
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Thu, 25 Apr 2013 20:20:41 +0000

Larry,

thinking about it the following may be a solution how we could model this without violating the critical one to one relation between station and MAC-address.

In case where the station needs to be receiver for more than a single unicast MAC address (e.g. if the station is a bridge port), then we need to model it as if the wildcard MAC-address (or broadcast MAC address with the value 0xFFFFFF is assigned to the station.

This would model the behavior of a transparent bridge correctly, I hope at least.

I believe in all the cases you mention, the station is configured with the Station-ID 0xFFFFFF, which is a valid MAC-Address, but not a L2-endpoint address.

The functions you mention such as bridging, IDS, firewall may be considered filtering of relayed frames  while bridging them from the ingress station to a predetermined egress station of the bridge.  L3-Forwarding using a routing table in a router  may be considered bridging from an ingress station to  dynamically determined egress station of the bridge, where said dynamic determining of said egress bridge port happens  on the basis of looking at L3 address information (IP-address) encapsulated in that bridged layer 2 frame.

Loadbalancing may be considered hunting an available egress station of the bridge, while perhaps at the same time also hunting one of multiple end stations present on the virtual or real yellow cable attached to the dynamically selected egress bridge port. Said hunting may for example be using round robin loadbalancing among a group of stations referred to as “hunt group”. There may be DNS lookup involved when terminating different services on different stations (identified each by a MAC-address within the local Layer2 network and also identified by a globally reachable IP address, that not only identifies the end point, but also the virtual yellow cable by the CIDR that identifies the subnet associated with the virtual yellow cable on which the destination station is present.

Lothar

Von: Larry Kreeger (kreeger) [mailto:[email-masked]]
Gesendet: Donnerstag, 25. April 2013 21:20
An: Reith, Lothar; Black, David; Florian Mahr
Cc: [email-masked]
Betreff: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Hi Lothar,

Thanks, Larry

David,

Lothar

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 25 Apr 2013 21:54:58 +0000

Lothar,

While I agree that a NIC would be programmed to accept frames with any MAC address, the reason for me to discuss addresses is more related to how the addresses are relayed to the NVE for inner-to-outer address mapping for delivering frames/packets across the overlay VN to the correct NVE.  It seems like a bad idea for any station that can receive more than one address to request the NVE to deliver all frames/packets to it.  If there are 100 stations on the other side of a VM performing bridging, the overlay should only deliver  those 100 addresses to the NVE the VM is connected to, not any address.

This discussion does make me wonder if we should have a way for an NVE to deliver "unknown" addresses to a TS.  There are applications that (such as one mode of Microsoft Network Load Balancing) that desire unknown frames to be delivered to them.  A VM acting as a bridge may be one of these applications because the bridge may not know exactly what station addresses are available its interfaces if the stations are quiet until spoken to.

- Larry

From: <Reith>, Lothar <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 25, 2013 1:20 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>, Florian Mahr <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: AW: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Larry,

The functions you mention such as bridging, IDS, firewall may be considered filtering of relayed frames  while bridging them from the ingress station to a predetermined egress station of the bridge.

L3-Forwarding using a routing table in a router  may be considered bridging from an ingress station to  dynamically determined egress station of the bridge, where said dynamic determining of said egress bridge port happens  on the basis of looking at L3 address information (IP-address) encapsulated in that bridged layer 2 frame.

Lothar

Hi Lothar,

Thanks, Larry

David,

Lothar

Von:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] Im Auftrag von Black, David
Gesendet: Donnerstag, 25. April 2013 15:24
An: Florian Mahr
Cc: [email-masked]<mailto:[email-masked]>
Betreff: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Thanks,
--David

From:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Florian Mahr

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Pat Thaler" <[email-masked]>
Date: Fri, 26 Apr 2013 22:29:39 +0000

We found similar use cases when we defined VDP in IEEE 802.1Qbg.
To allow a switch to classify traffic as belonging to a VSI (Virtual Service Interface – similar to a TSI), we defined several “Filter Info Formats” – information in the frame that would identify it as belonging to that end point.

To support a system that was providing a service across many addresses (e.g. a firewall or load balancer), the filter info can be a VID (VLAN ID) or a list of VIDs. That allows for a device that receives and sends on any MAC addresses within the VID(s). Another filter info format carries one or more MAC/VID pairs to allow for a VSI that is associated with a small number of MAC/VID pairs.

For the case where the network was managed on the basis of a longer tenant identifier, e.g. a 24-bit I-SID, there are formats that add a 32-bit Group ID to the VID or MAC/VID. We created this format because of the use case where the data center is using PBB, but the concept could also apply to tenant identifiers for NVO.  When this format is used, the profile for the VSI provides the Group ID. A locally-scoped VID is assigned to map to the Group ID.

I agree with Larry that one should allow for cases where there are multiple addresses associated with a VSI without having all addresses going to that VSI though I’m not convinced that the number for that should be as high as 100.

Regards,
Pat

From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Thursday, April 25, 2013 2:55 PM
To: Reith, Lothar; Black, David; Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Lothar,

- Larry

Larry,

Lothar

Hi Lothar,

Thanks, Larry

David,

Lothar

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Fri, 26 Apr 2013 23:10:20 +0000

Hi Pat,

How do we decide what the right number of addresses to support on a TSI is?  If the TS is bridging, it could have a lot.  I have no idea if 100 is the right number either, but it doesn't seem far fetched.  How many is the "small number of MAC/VID pairs" supported by VDP?

Thanks, Larry

From: Pat Thaler <[email-masked]<mailto:[email-masked]>>
Date: Friday, April 26, 2013 3:29 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, "Reith, Lothar" <[email-masked]<mailto:[email-masked]>>, David Black <[email-masked]<mailto:[email-masked]>>, Florian Mahr <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Regards,
Pat

Lothar,

- Larry

Larry,

Lothar

Hi Lothar,

Thanks, Larry

David,

Lothar

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Florian Mahr" <[email-masked]>
Date: Sat, 27 Apr 2013 14:04:50 +0200

AFAIK, the number of MAC/VID pairs in one VDP association TLV is
specified by the Number of Filter Info entries field, and this field has
a length of 16 bits.

Nevertheless, in regard to the case of a TSI de-associate with 100
addresses associated with that TSI, I think it is not quite efficient to
signal them as 100 indivual addresses to the NVE. In such a situation I
would prefer to signal only a single TSI Instance ID (TSIID) which
represents all the addresses associated with that TSI. However, in order
to achieve that, the NVE needs to know the exact relationship between
the addresses and the TSI. In VDP this is quite easy to achieve, as an
128-bit VSIID is always part of an VDP association TLV. I personally do
not see any disadvantage when each TSI would be assigned a 128-bit UUID
at its creation time, which is then signaled to the NVE as part of an
associate and de-associate request.

Thanks, Florian

>>> "Larry Kreeger (kreeger)" <[email-masked]> 27.04.13 1.11 Uhr >>>

Hi Pat,

How do we decide what the right number of addresses to support on a TSI
is?  If the TS is bridging, it could have a lot.  I have no idea if 100
is the right number either, but it doesn't seem far fetched.  How many
is the "small number of MAC/VID pairs" supported by VDP?

Thanks, Larry

From: Pat Thaler <[email-masked]>
Date: Friday, April 26, 2013 3:29 PM
To: Larry Kreeger <[email-masked]>, "Reith, Lothar"
<[email-masked]>, David Black <[email-masked]>, Florian
Mahr <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: RE: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

To allow a switch to classify traffic as belonging to a VSI (Virtual
Service Interface – similar to a TSI), we defined several “Filter Info
Formats” – information in the frame that would identify it as belonging
to that end point. 

To support a system that was providing a service across many addresses
(e.g. a firewall or load balancer), the filter info can be a VID (VLAN
ID) or a list of VIDs. That allows for a device that receives and sends
on any MAC addresses within the VID(s). Another filter info format
carries one or more MAC/VID pairs to allow for a VSI that is associated
with a small number of MAC/VID pairs. 

For the case where the network was managed on the basis of a longer
tenant identifier, e.g. a 24-bit I-SID, there are formats that add a
32-bit Group ID to the VID or MAC/VID. We created this format because of
the use case where the data center is using PBB, but the concept could
also apply to tenant identifiers for NVO.  When this format is used, the
profile for the VSI provides the Group ID. A locally-scoped VID is
assigned to map to the Group ID.

I agree with Larry that one should allow for cases where there are
multiple addresses associated with a VSI without having all addresses
going to that VSI though I’m not convinced that the number for that
should be as high as 100.

Regards,

Pat

 

Lothar,

While I agree that a NIC would be programmed to accept frames with any
MAC address, the reason for me to discuss addresses is more related to
how the addresses are relayed to the NVE for inner-to-outer address
mapping for delivering frames/packets across the overlay VN to the
correct NVE.  It seems like a bad idea for any station that can receive
more than one address to request the NVE to deliver all frames/packets
to it.  If there are 100 stations on the other side of a VM performing
bridging, the overlay should only deliver  those 100 addresses to the
NVE the VM is connected to, not to deliver "unknown" addresses to a TS.  There are applications that
(such as one mode of Microsoft Network Load Balancing) that desire
unknown frames to be delivered to them.  A VM acting as a bridge may be
one of these applications because the bridge may not know exactly what
station addresses are available its interfaces if the stations are quiet
until spoken to.

- Larry

From: <Reith>, Lothar <[email-masked]>
Date: Thursday, April 25, 2013 1:20 PM
To: Larry Kreeger <[email-masked]>, David Black
<[email-masked]>, Florian Mahr
<[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: AW: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

Larry,

thinking about it the following may be a solution how we could model
this without violating the critical one to one relation between station
and MAC-address.

In case where the station needs to be receiver for more than a single
unicast MAC address (e.g. if the station is a bridge port), then we need
to model it as if the wildcard MAC-address (or broadcast MAC address
with the value 0xFFFFFF is assigned to the station.

This would model the behavior of a transparent bridge correctly, I hope
at least.

I believe in all the cases you mention, the station is configured with
the Station-ID 0xFFFFFF, which is a valid MAC-Address, but not a
L2-endpoint address.

The functions you mention such as bridging, IDS, firewall may be
considered filtering of relayed frames  while bridging them from the
ingress station to a predetermined egress station of the bridge. 

L3-Forwarding using a routing table in a router  may be considered
bridging from an ingress station to  dynamically determined egress
station of the bridge, where said dynamic determining of said egress
bridge port happens  on the basis of looking at L3 address information
(IP-address) encapsulated in that bridged layer 2 frame.

Loadbalancing may be considered hunting an available egress station of
the bridge, while perhaps at the same time also hunting one of multiple
end stations present on the virtual or real yellow cable attached to the
dynamically selected egress bridge port. Said hunting may for example be
using round robin loadbalancing among a group of stations referred to as
“hunt group”. There may be DNS lookup involved when terminating
different services on different stations (identified each by a
MAC-address within the local Layer2 network and also identified by a
globally reachable IP address, that not only identifies the end point,
but also the virtual yellow cable by the CIDR that identifies the subnet
associated with the virtual yellow cable on which the destination
station is present.

Lothar

 

 

Hi Lothar,

In the simplest case of a VM acting as a host with a vNIC, the station
has one MAC address and one IP address using that MAC.  I assume this is
the most common case, but I think we must also support the cases where
the VM is:

 (a) acting as a bridge (perhaps while performing a service such as IDS
or firewall, or acting as a L2 VPN gateway), 

 (b) acting as a router (perhaps while performing a service such as IDS
or firewall, or acting as an L3 VPN gateway),

 (c) acting as load balancer terminating different services on different
IP addresses

In case (a), the VM's vNIC may have a MAC address that it uses for an L3
interface used for management, but there will also be frames entering
and leaving the vNIC with MAC addresses that belong to other stations
for which the VM is performing the bridging function.  The VM could be
bridging these frames between two vNIaddress, but these frames may be carrying packets with the IP addresses
of stations for which the VM is performing the routing function, in
addition to the VM's own IP address used for management or control plane
traffic.

In case (c), the VM always sends/receives frames using the vNIC MAC
address, but these frames may be carrying multiple IP addresses which
are terminated on the VM.

I don't think these cases are all covered in the model you describe
below.  If they are, can you help describe the relationships you are
thinking of?

In the thread below, David is asserting that the NVE does not need to
know what TSI any of the addresses are associated with (only the VAP). 
I agree that the TSI is not strictly needed, but I think it could be
useful for the following reasons.   In the cases above where there are
multiple addresses associated with a TSI.  If we look at case (a), the
NVE will need to forward frames towards the vNIC for all the MAC
addresses for which the VM is performing the bridging function (let's
say 100 addresses).  When that VM migrates to another hypervisor, the
NVE will need to disassociate the 100 addresses from the NVE port
leading towards the vNIC.  This could be signaled from the hypervisor to
the NVE as 100 individual addresses, or it could be signaled as one TSI
disassociate (if the relationship of the addresses to the TSI are
known).

Thanks, Larry

From: <Reith>, Lothar <[email-masked]>
Date: Thursday, April 25, 2013 11:16 AM
To: David Black <[email-masked]>, Florian Mahr
<[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends ...

David,

I fully agree with your statement: “Because we’re getting into
object-oriented design where the nature of relationships is crucial”

Therefore I would like to ask: Are we 100% sure, that an association of
multiple MAC-addresses directly with the vNIC is a good idea?

Something is disturbing me with that idea. Looking at Bob Metcalfe’s 40
year old design, it becomes obvious, that the object “station” may be
missing.

Requirement1: The object “station” needs a station-name, that is not the
MAC address.

Reason: To support use cases where an associated MAC-address or an
associated MEP-ID has to be changed, e.g. due to a hardware failure of a
NIC, but without having to tell all communication partners, which may
even be in other administrative domains.

Requirement2: There needs to be a maximum of one MAC-address assigned
per station, or none during times when a station does not have a MAC
address assigned.

Reason: To be able to leverage existing standards for endpoint
identification and authentication and to ease the IT support for that. I
am thinking of DHCP and of DHCP intermediate agent (where the station-id
is the identifier that identifies the object that tries to get the IP
address assigned to it – I conclude therefore that the object is the
station) or the PPPoE intermediate agent (where the station-ID is the
identifier that identifies the object that tries to get an IP address
dynamically assigned – obviously I conclude that this object is the
station).

To relate that to the below: The MAC-address is strongly associated with
the station, but the MAC-address is not the station.

And when an IP address gets associated to a MAC-address with DHCP, that
does not mean that the IP-address identifies the MAC-address, it does
also not mean that the IP address identifies the host or VM, rather one
could argue the IP-address always identifies a station on a physical or
virtual yellow cable, that is located at the network facing side of the
NIC (or vNIC in case of VM).  On the associated compute facing side the
corresponding compute facing interface object may be called “port” (at
least OpenStack Quantum API2.0 calls this compute-side-facing interface
not get assigned to the port, but to the station on the “other side” of
the interface, i.e. on the network facing side.

As a side note: One could argue, that a loopback address in a router
could be considered as a degenerate case of a collapsed virtual yellow
cable that is not bound to a physical NIC, but has a single station as
one endpoint of a virtual yellow cable, a single interface to a gateway
as another endpoint, and the length of the virtual yellow cable being
zero – such that the loopback network and the loopback station and the
loopback interface are collapsed into one singularity requiring only a
single IP address to identify all 3 objects.

Now, what does all have to do with the title of this thread “what is a
TSI?”. When looking with a compute side paradigm at this, the answer may
be: a port (or in Quantum API: a Quantum network port). 

When looking at this with a network side paradigm, the answer may be: a
station

Lothar 

 

better not

> such in a hypervisor-to-NVE control protocol?

Sure, but let me start with a caveat that some of this may seem like
hair-splitting,

because we’re getting into object-oriented design where the nature of
relationships

is crucial.  From that perspective “attribute of” is a “part of”
relationship, whereas

I view addresses as “associated with” TSI objects.  To make this
distinction clear by

example:

Regarding signaling, I believe that the addresses should be signaled,
not TSIs, and

I’m a co-author of draft-kompella, which spells out some of the details
of how that

could work.  That draft just expired (mea culpa - my day job’s been
limiting the

time that I can put into IETF), I expect to submit a new version either
today or

 To make

this a bit more concrete, I don’t think the following two cases need to
be

VM events may implicitly distinguish these cases (e.g., one MAC vs. two
MACs

affected by VM migration or VM power on/off events), but (IMHO) the
signaling

protocol doesn’t have to care - the NVE and VAP just have to know that
those

Thanks,
--David

 

Hi David,

 

Thanks,

Florian

Lothar,

vNIC.

Ok ...

[... snip ...]

“not having to consider NIC-teaming with LAG” *tI expect that LAG functionality to be part of a solution component other

 

in that MIB.

Thanks,
--David

 

David,

It is also extensible from point to point toFrom:[email-masked] [mailto:[email-masked]] On Behalf Of
Black, David
Sent: Tuesday, April 23, 2013 11:50 AM
To: Florian Mahr
Cc: [email-masked]
Subject: Re: [nvo3] Antw: What's a TSI? It depends ...

Hi Florian,

well understood:

more than IMHO ...

Restating that proposal:

Does that make sense?

Thanks,
--David

applies to L3 network services provided by an based on  the principle that IP addresses plus routing information sort

 

 

Hi David,

 

Thanks,

Florian

Thanks,
--David

Cc: Black, David;Point B is a perfect reference point. But I am not sure we only use VNID

Regards!

-Qin

 

Hi Anoop,

 

Thanks, Larry

 

Larry, 

 

Anoop

Hi Qin,

 

 

         VNIC-+-VN-+-MAC-+-IP

           ^  |    |     +-IP ...

           |  |    |

           |  |    +-MAC-+-IP

           |  |          +-IP ...

           |  |

           |  +-VN-+-MAC-+-IP

           |     ^ |     +-IP ...

           |     | |

           |     | +-MAC-+-IP

           |     |    ^  +-IP ...

           |     |    |     ^

           |     |    |     |

           A     B    C     D

 - Larry

 

Hi, Larry:

However if one VM is simply regarded as one tenant system and each vNIC
with multiple IP addresses belthen one TSI has multiple IP addresses and will be associated with
multiple NVEs.

                +---------------------+

                |                     |

                |  VM (Tenant System) |

                |                     |

                +-+--------+--------+-+

                  |        |        |

                  |        |        |

                  |        |        |

                  +        + ...... +

                  |

                  |

         |--------+------+

         |        |      |

        VN1      VN2    VN3

           VN  -- Virtual Network

           IP add1

               -- IP address1

 

               +---------------------+

               |                     |

               |  VM (Tenant Systems)|

               |                     |

               +-+--------+--------+-+

                 |        |        |

                 |        |        |

                 |        |        |

                 |                 |

                 |

                 |

         +-------+------+

         |       |      |

         |       |      |

        TSI1    TSI2   TSI3

            TS1_ Tenat System1

Regards!

-Qin

 

Hi Anoop,

 

 - Larry

 

 

 

Anoop

Thanks,
--David

Subject: Re: [nvo3] 答复: 答 AFAIK it is common practice to use vNIC bonding (LAG) for high

Lothar

 

 

Framework said:

“

role

or

”

am I missing?

 

 

[Qin]: Yes.

 

 

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Mon, 29 Apr 2013 08:43:16 +0000

Dear all,

I would like to apologize upfront that the following is going to be a fairly long email response, which I wrote over the weekend in response to the latest 3 mails on this thread (from Florian, Pat, and Larry).

Writing this felt like doing some historic reverse re-engineering (or post-mortem forensic analysis). Many thanks to Larry for challenging me to explain my model for the cases a) b) c) of bridging,routing, loadbalancing. While I wrote down the answer relatively quickly in a creative moment, I must say I did not know the answer before Larry had challenged me. And I now understand better what it was that was disturbing me. It was fear that my thought building was going to crash, because of the corner stone one to one relation between one (unicast) MAC-address and one yellow cable station-interface appeared to be jeopardized. I know how it feels when thought buildings crash, in which you have invested many years and effort to build. Such a personal paradigm change is a painful experience, because you have to rewire your brain.

I am afraid, that I may be doing this to some of you in the below text. If it feels like having to re-wire your brain, then this is a good indication that you understood, and that you were also a victim of the missing object problem which I am going to describe below.

@ Florian: to better understand my response, you should also read my below response to Pat.  I think what you call a MAC/VID pair is in my below revised model a L2-SAP instance (and in my prior modeling language a yellow cable station where “station“ more precisely refers to “station-interface” or even more precisely refers to the compute side facing interface of a layer 2 layer network endpoint  to the rest of the computer referred to as station, even more precisely the “missing object” which may be missing in your paradigm is the L2-SAP (Layer 2 Service Access Point which is the compute side facing interface which interfaces from an instance of a L2 network endpoint upwards in the stack to the next higher layer, i.e. to an entity in the layer 3 layer network which interfaces downwards in the stack towards the layer 2 and therefore towards the lower layers and therefore ultimately towards the network side.  My experience is, that some layer 3 paradigm centric colleagues have some kind of confusion about the meaning of the term interface, because due the lack of a corresponding object L2-SAP they have modeled stuff belonging to lower layers into their own interpretation of the term interface. So depending on context, their perception of “interface” may include a reference to VLAN, Portchannel/LAG down to and including the port. This may be a  source of confusion, in particular when you have to deal for example with 2 vendor specific implementations of Multi-Chassis LAG where one vendor on one side of the LAG virtualizes two chassis into one virtual switch and the other on the other side of the LAG does not. I had to get my head around this real world implementation not so long ago, trying to enhance the associated inventory and IT solutions for assurance purpose under operator conditions including use cases such as a port change due to a defect port. Another indication for the existence of this “missing object problem” and the overloading of the object “interface” is the fact, that in the SID (shared information and data model of the TMF) the object “interface” has two attributes: “customerInterfaceNumber” and “vendorInterfaceNumber”.

@Larry: considering the use case of a hybrid cloud migration, it may be necessary to move all stations in a subnet over to a different location in another administrative domain. Therefore my recommendation for the size would be to engineer for the worst case scenario, where the number you are looking for is the size of the subnet aka the maximum number of station interfaces on the virtual yellow cable when an IP address range has been bound to a virtual yellow cable instance, where said subnet was created in the moment where said binding was done (probably by a planer in a service and resource lifecycle management process according to eTOM).

@ Pat: I believe you used the term „tenant“ a bit loosely. I believe what you meant in your last mail was not “tenant” but  tenant system interface instance, characterized by said interface instance of said tenant system being an interface of a layer 3 layer network endpoint instance  interfacing downwards in the stack (i.e. network side facing) in a one to one relation to a L2-SAP (Service Access Point) of a corresponding layer 2 layer network endpoint instance, where said L2-SAP is a layer 2 layer network Service Access Point instance interfacing upwards in a one to one relation to said layer 3 layer network interface instance, further characterized by said L2-SAP instance residing on top of (or in?) a MAC-layer instance, further characterized by said L2-SAP instance having a one to one relation to said MAC-layer instance, further characterized by said MAC-Layer instance being a uniquely addressable network endpoint in said layer 2 layer network instance, and further characterized by all said instances being instantiated inside a tenant station, further characterized by said tenant station being a VM or a physical machine that is associated with a tenant, further characterized by said  uniquely addressable network endpoint instance unfortunately – perhaps due to a historic standardization accident, and perhaps due to object oriented design being invented only a couple of years later – may in the data models of some legacy routers not have been properly modeled.
This historic standardization accident may be the reason for the confusion that has puzzled me over some time, and has been evident on this mailing list. I believe this may be an explanation for the situation at hand – as evident on this thread - that a number of leading experts in the field of routing and switching appeared to have been confused about the true nature of an object “interface”, or “tenant system interface”.
I had been confused myself and only had the feeling, that something may be wrong with the definition of the object “interface”, due to the fact that it may have been overloaded with attributes of another object, which the original designers of the first routers may have failed to specify as a separate object.  Given the historic situation at the time when these decisions were made, and the concepts available to the designers, this failure is fully understandable. I would probably have made the same mistake in the same historic situation.
This design accident (and associated subsequent standardization accidents) probably happened at a time when the yellow cable (10BASE5)  and perhaps the Cheapernet (10BAS2) were state of the art of networking. Therefore, I looked at Bob Metcalfes original design, and identified the term “station” as a candidate for this missing object. However, as I have meanwhile analyzed the situation a bit deeper, I found that the term “station” in Bob Metcalfe’s original design is depicted at a position which later was standardized as MAU (Media Access Unit) in IEEE802.3. Bob Metcalfes original design also had the term “interface”, but at a position which later became the PMA (Physical Media Attachment) terminating the AUI-cable.  To make a long story a little bit shorter: When I used the term “station” in my below emails of this thread, I was trying to refer to a layer 2 network endpoint or L2-SAP. So I found that I did not mean “station” in the sense of “computer” but more in the sense of “station interface” even more precisely specified above in may response to Florian as L2-SAP.  So when you expand the word “station” in my below mails to “station interface” (in the sense of the L2-SAP being the interface of layer 2 to the station in the sense of the next higher layer in the station), then my model using the word station (or better station interface) should make perfect sense.
As I had explained, my proposal to name the missing object “station” was founded based on the use of the term “station-ID” in the DHCP and PPPoE standards.
The  term station-ID refers in DHCP and PPPoE standards to the hardware address/ unicast MAC-address of the NIC issuing the PADI or DHCP request Ethernet frames. In my view, this is another standardization accident, that the MAC-address is referred to as “station-ID”. There should have been another entity specified, to which the MAC address will be assigned. And this other entity should have been called “station-ID” or even more precisely “station-interface-ID”.  This other entity could have been called with whatever name, but “station interface” or  “L2-SAP” would be have been good names for this entity. The MAC-address should not have been called Station-ID, nor station-interface-ID, because an address is not the same thing as the interface to which it is assigned, even if that assignment is created by burning the MAC-address into the hardware of the NIC. My preferred candidate would have been Unicast-MAC-address (instead of station-id).
When the original router designer made their implicit object oriented design decisions, they may have taken the following unspoken assumptions for granted:

  1.  A  computer (or station) always has only  a single interface to the network
  2.  The term “station-ID” identifies the single NIC of a computer, and because of assumption 1 it is no problem to use the term station-ID, even though it refers to the NIC being a station-Interface.
  3.  The term station-ID indeed refers to an identifier of a station. It identifies a station or more precisely a NIC, and not the unicast-MAC-address to be used when trying to reach said station (or more precisely said station interface or said L2-SAP).
  4.  It is not necessary to introduce the concept of a layer 2 layer network explicitly, as there will always be a one to one relation between “Interface” and “port”. (Portchannels, LAGs, Ethernet Switches, and VLANs did not exist yet).

Given the above historic situation, it is no wonder that the original router designers did not see a need to specify a L2-SAP as a separate object. They thought that the object “interface” is enough. The above wrong assumptions may have led to the side effect,  that the host-part of the IP-address does not identify the host, but an interface of a host.  Well, a workaround was found by inventing the loopback interface.
In any case, the lack of a L2-SAP object in some router architectures appears to be a root cause for much complexity and many problems.
Let me compare this situation with a fictitious situation in the automotive industry and a fictitious “intelligent street” industry. Can you imagine the automotive industry would not specify the object “tyre”, and always call this entity “wheel”, regardless if you meant the axis facing wheel or the street facing tyre? And the management system of the intelligent street trying to interface to the tyre, but his counterparts always talking about wheel, because they have no concept of “tyre”?
That is the kind of confusion I was referring to, when I stated that I have personal alarm bells ringing when someone talks about an “interface” and expects his counterpart to understand the same as him.
Why is this important?
Because the early day router designers and early day standardization experts made more mistakes further complicating the situation and increasing the complexity: Not only did they not specify separate objects for a L3-interface and a L2-SAP and a L2-Interface to a L1-SAP of a physical port. To make things worse, they used a “talking name” as identifier for the  object “interface” and made the interface-ID expose topology information of the router hardware, e.g. in which slot, subslot, port the associated L1 SAP can be located.
This was a terrible design choice, leading to hundreds of millions, if not billions of wasted IT dollars and carrier operations expenses which could have been saved in case the interface-ID would have been kept free of any semantic. For example, a random number (like a combined with a  table to the locator information Slot/Subslot/Port would have decoupled the interface-ID from interface locator identifying the location of the L2-SAP (which in turn interfaces to a L1-SAP of the physical Port terminating the physical medium in the physical layer.
The talking name nature of the interface identifier lead to “interface ID” being vendor specific and having an implicit locator function allowing to locate the L1-SAP “Port” (implicitly with the unspoken but wrong assumption, that there is a one to one relation between L3 interface and L1 SAP “Port”. To make things even worse, the DSL-Forum TR-69 based 1:1 architecture makes this “interface” (or its derivative NAS-Port) the leading identifier of the DSL access service, with terrible consequences for carriers having adopted this 1:1 architecture.
Trust me, I believe carriers around the world have collectively spent hundreds of millions of dollars trying to manage the resulting complexity in their IT systems.
So, whatever you do when designing a tenant system “router”. If possible at all, try to not repeat these old mistakes.
At least OpenStack based routers are immune against this mistake, because they first create a UUID when a new Quantum network port gets created in the compute side of the world.
Perhaps this is the hidden secret of the SDN paradigm change, that the compute side paradigm centric part of the world in future creates these network entities (instead of these entities being created in the network paradigm centric side of the world and later being “discovered” and “managed” by the compute side paradigm centric part of the world).

Best Regards, Lothar  (full disclosure: my employer is Deutsche Telekom, more precisely Detecon International GmbH which is a 100% subsidiary of Deutsche Telekom offering consultancy in ICT.

Von: Florian Mahr [mailto:[email-masked]]
Gesendet: Samstag, 27. April 2013 14:05
An: [email-masked]; [email-masked]; Reith, Lothar; [email-masked]
Cc: [email-masked]
Betreff: Antw: Re: [nvo3] Antw: Re: Antw: What's a TSI? It depends...

AFAIK, the number of MAC/VID pairs in one VDP association TLV is specified by the Number of Filter Info entries field, and this field has a length of 16 bits.
Nevertheless, in regard to the case of a TSI de-associate with 100 addresses associated with that TSI, I think it is not quite efficient to signal them as 100 indivual addresses to the NVE. In such a situation I would prefer to signal only a single TSI Instance ID (TSIID) which represents all the addresses associated with that TSI. However, in order to achieve that, the NVE needs to know the exact relationship between the addresses and the TSI. In VDP this is quite easy to achieve, as an 128-bit VSIID is always part of an VDP association TLV. I personally do not see any disadvantage when each TSI would be assigned a 128-bit UUID at its creation time, which is then signaled to the NVE as part of an associate and de-associate request.

Thanks, Florian

Hi Pat,

Thanks, Larry

Regards,
Pat

Lothar,

- Larry

Larry,

Lothar

Hi Lothar,

Thanks, Larry

David,

Lothar

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes

---

From: "Reith, Lothar" <[email-masked]>
Date: Mon, 29 Apr 2013 09:08:01 +0000

Correction: I looked again at the 40 year old diagram from Bob Metcalfe, and I found that my statement regarding the position of the station in Bob’s diagram is wrong. Indeed, the diagram is closer to my current view than I thought. The interface in Bob’s diagram seems to refer to the L2-interface towards Layer1. The station is not the transceiver (my bad), rather the term station seems to refer to the computer or to the computer interface being attached. That is not quite clear from the diagram.

Sorry for this little confusion. Lothar

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Reith, Lothar
Gesendet: Montag, 29. April 2013 10:43
An: Florian Mahr; [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]
Betreff: Re: [nvo3] Antw: Re: Antw: Re: Antw: What's a TSI? It depends...

Dear all,

Thanks, Florian

Hi Pat,

Thanks, Larry

Regards,
Pat

Lothar,

- Larry

Larry,

Lothar

Hi Lothar,

Thanks, Larry

David,

Lothar

Thanks,
--David

Hi David,

Thanks,

Florian
Lothar,

Ok ...

[... snip ...]

Thanks,
--David

David,

Lothar

Restating that proposal:

Does that make sense?

Thanks,
--David

Hi David,

Thanks,

Florian

Thanks,
--David

+1,

Regards!
-Qin

Hi Anoop,

Thanks, Larry

Larry,

Anoop

         VNIC-+-VN-+-MAC-+-IP
           ^  |    |     +-IP ...
           |  |    |
           |  |    +-MAC-+-IP
           |  |          +-IP ...
           |  |
           |  +-VN-+-MAC-+-IP
           |     ^ |     +-IP ...
           |     | |
           |     | +-MAC-+-IP
           |     |    ^  +-IP ...
           |     |    |     ^
           |     |    |     |
           A     B    C     D

 - Larry

Hi, Larry:

                +---------------------+
                |                     |
                |  VM (Tenant System) |
                |                     |
                +-+--------+--------+-+
                  |        |        |
                  |        |        |
                  |        |        |
                  +        + ...... +

               +---------------------+
               |                     |
               |  VM (Tenant Systems)|
               |                     |
               +-+--------+--------+-+
                 |        |        |
                 |        |        |
                 |        |        |
                 |                 |

Regards!
-Qin

Hi Anoop,

 - Larry

Anoop

Thanks,
--David

Lothar

“
”

[Qin]: Yes.

LK3> Yes