Subject: [alto] How Data Center Virtualization influence ALTO mechanism.

From: "Y.J. GU" <[email-masked]>
Date: Sat, 09 Oct 2010 15:18:29 +0800

Hi all, 
I was thinking about how Data Center Virtualization and Virtual Machine(VM) Migration will influence ALTO mechanism.

Current ALTO Protocol defines clustering of peers according to their IP Addresses. E.g. peers in same subnet will be classified into same PID, and path cost will indicate the cost within and between PIDs, which is also actually based on IP Addresses.

In the current world, peers are partitioned by IP subnet. While considering virtual machines migration, there might be more interesting things to think of.

In Data Center operation, one basic consensus is 'When Virtual Machines move from one site to another, the IP Addresses will not change, so that the existing service connection will not be broken'.  VMs can migrate to arbitrary site, not under the control and knowledge of ISP. For example, some VMs in Data Center A(IP subnet 198.1.1.0) move to Data Center B (IP subnet 210.1.1.0). IP-based, Vms are closer to DC-A. Physically, these VMs are much closer to hosts in DC-B. However things are not so easy, especially considering how these VMs are routed. Current ALTO may give wrong cost ranking.

VMs may migrate under, but not limited to, these situations: 1) to save electricity power, 2) disaster recovery, 3) customer prefer another Data Center, 4) company extension, etc. In the end, the internet will not be a regular world partitioned by IP Addresses.

Does anyone think this is an interesting aspect to study?

---

From: Tao Ma <[email-masked]>
Date: Tue, 12 Oct 2010 10:44:44 +0800

Hi,
    I think it is an interesting aspect for ALTO protocol to be considered.
The current ALTO uses IP address as the endpoint adress for grouping and
locating. For the VM migration, this would fail, especailly without the
contol of the ISP or ALTO service provider. Can ALTO mechanism make some
changes to reflect this migration in some way? By using other alternatives
such as geolocations or PoP to group, can we avoid this and how?
PS: I have a question personally. How the routing would succeed after VM are
migrated without changing IP adress? Can you tell me some current mechanism?
Thanks:)
Tao Ma
Beijing University of Posts and Telecommunications

From: Y.J. GU <[email-masked]>
> Date: Sat, Oct 9, 2010 at 3:18 AM
> Subject: [alto] How Data Center Virtualization influence ALTO mechanism.
> To: [email-masked]
>
>
> Hi all,
> Current ALTO Protocol defines clustering of peers according to their IP
> Addresses. E.g. peers in same subnet will be classified into same PID, and
> path cost will indicate the cost within and between PIDs, which is also
> actually based on IP Addresses.
>
> In the current world, peers are partitioned by IP subnet. While considering
> virtual machines migration, there might be more interesting things to think
> of.
>
> In Data Center operation, one basic consensus is 'When Virtual Machines
> move from one site to another, the IP Addresses will not change, so that the
> existing service connection will not be broken'.  VMs can migrate to
> arbitrary site, not under the control and knowledge of ISP. For example,
> some VMs in Data Center A(IP subnet 198.1.1.0) move to Data Center B (IP
> subnet 210.1.1.0). IP-based, Vms are closer to DC-A. Physically, these VMs
> are much closer to hosts in DC-B. However things are not so easy, especially
> considering how these VMs are routed. Current ALTO may give wrong cost
> ranking.
>
> VMs may migrate under, but not limited to, these situations: 1) to save
> electricity power, 2) disaster recovery, 3) customer prefer another Data
> Center, 4) company extension, etc. In the end, the internet will not be a
> regular world partitioned by IP Addresses.
>
> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto
>
>

---

From: Lars Eggert <[email-masked]>
Date: Tue, 12 Oct 2010 10:17:13 +0300

Hi,

On 2010-10-9, at 10:18, Y.J. GU wrote:
> In Data Center operation, one basic consensus is 'When Virtual Machines move from one site to another, the IP Addresses will not change, so that the existing service connection will not be broken'.

inside one data center, sure. Maybe even across data centers, with PI space and a coordinated route update. But I don't think that'd common practice.

>  VMs can migrate to arbitrary site, not under the control and knowledge of ISP. For example, some VMs in Data Center A(IP subnet 198.1.1.0) move to Data Center B (IP subnet 210.1.1.0).

Huh? Didn't you just say above that the IP addresses wouldn't change?

> IP-based, Vms are closer to DC-A. Physically, these VMs are much closer to hosts in DC-B. However things are not so easy, especially considering how these VMs are routed. Current ALTO may give wrong cost ranking.

If you're moving VMs around in the topology, you'll have to update the announced ALTO information as you do that. 

But since ALTO merely "provide(s) applications with information to perform better-than-random initial peer selection" - a direct quote from the charter - inaccurate or incorrect ALTO information is not catastrophic anyway.

Lars

---

From: Enrico Marocco <[email-masked]>
Date: Tue, 12 Oct 2010 10:26:03 +0200

On 10/12/10 9:17 AM, Lars Eggert wrote:
>> IP-based, Vms are closer to DC-A. Physically, these VMs are much
>> closer to hosts in DC-B. However things are not so easy, especially
>> considering how these VMs are routed. Current ALTO may give wrong
>> cost ranking.
> 
> If you're moving VMs around in the topology, you'll have to update
> the announced ALTO information as you do that.

According to a presentation I saw last year, this is actually what
happens in Facebook's server farms, where BitTorrent and a customized
locality-aware tracker are used to distribute software updates
minimizing inter-rack traffic. Whenever the topology changes -- and I
guess that happens quite often -- the updated information is directly
loaded into the tracker. I don't remember how the update mechanism works
in that specific case, but I guess that could be quite effectively
automated with some kind of integration between the ALTO-like service
and some IGP database.

I can't find a pointer to that presentation, but an article and a video
explaining how software update works in Facebook is available on
TorrentFreak:
http://torrentfreak.com/facebook-uses-bittorrent-and-they-love-it-100625/

-- 
Ciao,
Enrico

---

From: "Y.J. GU" <[email-masked]>
Date: Wed, 13 Oct 2010 11:29:41 +0800

Hi,

Good question. I don’t know how this is guaranteed in real world. But there are some discussions on this. 

HSRP is one way. IP mobility is another example.  

  _____  

From: Tao Ma [mailto:[email-masked]] 
Sent: Tuesday, October 12, 2010 10:45 AM
To: [email-masked]
Cc: [email-masked]; [email-masked]; chunhong zhang; [email-masked]; [email-masked]
Subject: Re: [alto] How Data Center Virtualization influence ALTO mechanism.

Hi,
    I think it is an interesting aspect for ALTO protocol to be considered. The current ALTO uses IP address as the endpoint adress for grouping and locating. For the VM migration, this would fail, especailly without the contol of the ISP or ALTO service provider. Can ALTO mechanism make some changes to reflect this migration in some way? By using other alternatives such as geolocations or PoP to group, can we avoid this and how?
PS: I have a question personally. How the routing would succeed after VM are migrated without changing IP adress? Can you tell me some current mechanism? Thanks:)
Tao Ma
Beijing University of Posts and Telecommunications 

Hi all,

---

From: stefano previdi <[email-masked]>
Date: Wed, 13 Oct 2010 13:22:13 +0200

On Oct 9, 2010, at 9:18 AM, Y.J. GU wrote:

> Hi all,
> I was thinking about how Data Center Virtualization and Virtual  
> Machine(VM) Migration will influence ALTO mechanism.
>
> Current ALTO Protocol defines clustering of peers according to their  
> IP Addresses. E.g. peers in same subnet will be classified into same  
> PID, and path cost will indicate the cost within and between PIDs,  
> which is also actually based on IP Addresses.

the methods of grouping are orthogonal to ALTO protocol specification.

There are ALTO implementations that allow address/prefixes grouping
relaxed from pure IP aggregation. The fact that you use IP addresses
in the protocol doesn't mean that locality is solely based on
address/mask pairs.

It is mostly a policy definition by the ALTO and infrastructure
operator.

> In the current world, peers are partitioned by IP subnet. While  
> considering virtual machines migration, there might be more  
> interesting things to think of.
>
> In Data Center operation, one basic consensus is 'When Virtual  
> Machines move from one site to another, the IP Addresses will not  
> change, so that the existing service connection will not be  
> broken'.  VMs can migrate to arbitrary site, not under the control  
> and knowledge of ISP. For example, some VMs in Data Center A(IP  
> subnet 198.1.1.0) move to Data Center B (IP subnet 210.1.1.0). IP- 
> based, Vms are closer to DC-A. Physically, these VMs are much closer  
> to hosts in DC-B. However things are not so easy, especially  

that is true. ALTO relies on accurate infrastructure/topology
information. It can be derived from lower layers (routing and below)
or inferred by policy DBs.

> VMs may migrate under, but not limited to, these situations: 1) to  
> save electricity power, 2) disaster recovery, 3) customer prefer  
> another Data Center, 4) company extension, etc. In the end, the  
> internet will not be a regular world partitioned by IP Addresses.

the "swamp" already validated the theory...

> Does anyone think this is an interesting aspect to study?

probably yes but I'm not sure I see the protocol implication.

s.

---

From: "Y.J. GU" <[email-masked]>
Date: Fri, 15 Oct 2010 16:17:57 +0800

I don't see the protocol implications either for now. But I think this is an implementation issue that ALTO needs to consider. ALTO doesn't care about how the information is collected, however ALTO need to define how the network map should be like in this scenario, will the cost map change as well? 
And VMs migration might be a strong motivation for ALTO Server to update the ALTO information that has been distributed among the network.

> -----Original Message-----
> From: stefano previdi [mailto:[email-masked]]
> Sent: Wednesday, October 13, 2010 7:22 PM
> To: Y.J. GU
> Cc: [email-masked]

---

From: Tao Ma <[email-masked]>
Date: Tue, 19 Oct 2010 13:10:02 +0800

Maybe the key question is how the change or update is possible if "VMs can
migrate to arbitrary site, not under the control and knowledge of ISP"(or
unknown to ALTO service provider). Like the secion 9.2 in "
draft-ietf-alto-protocol-05",  "In particular, a particular ALTO Service
provider may not be able to determine if connectivity with a particular
endhost will succeed over IPv4 or IPv6, as this may depend upon information
unknown to the ISP such as particular application implementations." I think
VM migration has the similar challenge and should be considered.

2010/10/15 Y.J. GU <[email-masked]>

> > From: stefano previdi [mailto:sprevidi <sprevidi> at cisco.com]
> > Cc: alto at ietf.org
>   ------------------------------
>
> *From:* Tao Ma [mailto:[email-masked]]
> *Sent:* Tuesday, October 12, 2010 10:45 AM
> *To:* [email-masked]
> *Cc:* [email-masked]; [email-masked]; chunhong zhang;
> [email-masked]; [email-masked]
> *Subject:* Re: [alto] How Data Center Virtualization influence ALTO
> mechanism.
>
>
>
> Hi,
> PS: I have a question personally. How the routing would succeed after VM
> are migrated without changing IP adress? Can you tell me some current
> mechanism? Thanks:)
> Tao Ma
>  From: *Y.J. GU* <[email-masked]>