Subject: [Idr] What do you think about this description of Path Selection added to draft-dunbar-idr-5g-edge-compute-app-meta-data?

From: Linda Dunbar <[email-masked]>
Date: Thu, 11 Feb 2021 03:32:55 +0000

Stephane, Acee and IDR WG:

During the IETF109 session on draft-dunbar-idr-5g-edge-compute-app-meta-data, both of you pointed out that there should be a section describing the forwarding plane behavior, such as how does the route selection decision is made or changed with the additional information carried in  the AppMetaData.  AiJun also pointed out needing this description in the IDR mailing list.

We plan to add  this subsection. Is it good enough?

3.5   BGP Path Selection upon receiving AppMetaData
UEs anchored to PSA1 (5G UPF) and PSA2 (5G UPF) all need to communicate with S1:aa08::4450, which is the ANYCAST address with the servers attached to R1, R2, R3 and R5.
Packets from the UEs anchored to PSA1 will ingress to R-PSA1 (Ingress router); Packets from the UEs anchored to PSA2 will ingress R-PSA2.
Assume that both R-PSA1 and R-PSA2 have BGP Multipath enabled. As a result, the routing table in either of them would look like: Dst Address: S1:aa08::4450   ---- resolved via NH: R1, R2, R3, R5
Based on BGP AppMetaData TLV, let’s say local BGP Compute Application Plugin for AppMetaData finds R1 is the best for the flow S1:aa08::4450. Then this Compute Application Plugin can insert higher weight for the path R1 so that BGP Best Path is locally influenced by the weight parameter based on the local decision.

Just to refresh what we discussed in IETF109:

Thank you very much.

Linda Dunbar

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 11 Feb 2021 11:26:18 -0500

Given the importance of managing all of selectivity, proximity 
(sometimes), and stickiness (so you don't move users from one service 
instance to another unless you also move the application state),...

WHy would we put this information in routing?
Yes, the information is needed.  But the full set of rotuers seem the 
wrong place to manage this behavior.  It might belong in edge routers, 
or in end devices, but not throughout the system.  So again, why put all 
this dynamic overhead into the routing system, burdening all rotuers.

Yours,
Joel

On 2/10/2021 10:32 PM, Linda Dunbar wrote:
> During the IETF109 session on 
> draft-dunbar-idr-5g-edge-compute-app-meta-data, both of you pointed out 
> that there should be a section describing the forwarding plane behavior, 
> such as how does the route selection decision is made or changed with 
> the additional information carried in  the AppMetaData.  AiJun also 
> pointed out needing this description in the IDR mailing list.
> 
>     3.5BGP Path Selection upon receiving AppMetaData
> 
> UEs anchored to PSA1 (5G UPF) and PSA2 (5G UPF) all need to communicate 
> with S1:aa08::4450, which is the ANYCAST address with the servers 
> attached to R1, R2, R3 and R5.
> 
> Packets from the UEs anchored to PSA1 will ingress to R-PSA1 (Ingress 
> router); Packets from the UEs anchored to PSA2 will ingress R-PSA2.
> 
> Assume that both R-PSA1 and R-PSA2 have BGP Multipath enabled. As a 
> result, the routing table in either of them would look like: Dst 
> Address: S1:aa08::4450   ---- resolved via NH: R1, R2, R3, R5
> 
> Based on BGP AppMetaData TLV, let’s say local BGP Compute Application 
> Plugin for AppMetaData finds R1 is the best for the flow S1:aa08::4450. 
> Then this Compute Application Plugin can insert higher weight for the 
> path R1 so that BGP Best Path is locally influenced by the weight 
> parameter based on the local decision.
> 
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 11 Feb 2021 17:39:58 +0000

Joel, 

Thanks for your comments. Replies are inserted below:

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Thursday, February 11, 2021 10:26 AM
To: Linda Dunbar <[email-masked]>; [email-masked]; Aijun Wang <[email-masked]>; Acee Lindem (acee) <[email-masked]>; Stephane Litkowski <[email-masked]>
Cc: [email-masked]
Subject: Re: [Idr] What do you think about this description of Path Selection added to draft-dunbar-idr-5g-edge-compute-app-meta-data?

Given the importance of managing all of selectivity, proximity (sometimes), and stickiness (so you don't move users from one service instance to another unless you also move the application state),...

[Linda] The Stickiness of binding one UE to the same Server when the UE moves to a different Cell Tower (i.e. anchored to a different UPF/PSA) is NOT conveyed by routing protocol. 
https://datatracker.ietf.org/doc/draft-dunbar-6man-5g-edge-compute-sticky-service/  describes approach to achieve stickiness , either by Hop by Hop or Destination Optional header.

WHy would we put this information in routing?
Yes, the information is needed.  But the full set of rotuers seem the wrong place to manage this behavior.  It might belong in edge routers, or in end devices, but not throughout the system.  So again, why put all this dynamic overhead into the routing system, burdening all rotuers.
[Linda] The AppMetaData is only sent to the Edge Nodes which are interested in the service. 

Yours,
Joel

On 2/10/2021 10:32 PM, Linda Dunbar wrote:
> draft-dunbar-idr-5g-edge-compute-app-meta-data, both of you pointed 
> out that there should be a section describing the forwarding plane 
> behavior, such as how does the route selection decision is made or 
> changed with the additional information carried in  the AppMetaData.  
> AiJun also pointed out needing this description in the IDR mailing list.
> 
> UEs anchored to PSA1 (5G UPF) and PSA2 (5G UPF) all need to 
> communicate with S1:aa08::4450, which is the ANYCAST address with the 
> servers attached to R1, R2, R3 and R5.
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fidr&amp;data=04%7C01%7Clinda.dunbar%40
> futurewei.com%7C47fa57e5844749e05e3f08d8cea9c541%7C0fee8ff2a3b240189c7
> 53a1d5591fedc%7C1%7C0%7C637486575848772996%7CUnknown%7CTWFpbGZsb3d8eyJ
> WIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000
> &amp;sdata=75pkqbg6z75Ji3MOtZhBdz8wAMrm9FNQg0i7Pwk8IEo%3D&amp;reserved
> =0
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 11 Feb 2021 12:47:34 -0500

Linda, changing towers is not the only case which needs stickiness. 
Given that Routing can change, and the attributes of servers can change, 
the whole system needs stickiness.
And once you have solved that level of stickiness I believe it is almost 
inherent that one no longer needs to put the application attributes or 
any enhancements to anycast into routing.

Yours,
Joel

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 11 Feb 2021 18:27:29 +0000

Joel, 

Now I understand where our communication misaligned. 

There are 3 separate things:
1. When a new flow comes to an ingress node, how to select the optimal egress router to reach an ANYCAST server.
2. How Ingress node keeps the packets from one flow to the same ANYCAST server, 
3. When a UE moves to a new Cell Tower, method to stick the flow to the same ANYCAST server, which is required by 5G Edge Computing: 3GPP TR 23.748. 

You are talking about Item 2 above, aren’t you? 
The Item 2 above, a.k.a. Flow Affinity, or Flow-based load balancing, is supported by many commercial routers. There is no need for additional attributes. For example, https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/lanswitch/configuration/xe-3s/lanswitch-xe-3s-book/lnsw-flow-portchannel-load.html 
https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/data-flow-affinity-edit-chassis.html

draft-dunbar-idr-5g-edge-compute-app-meta-data states that the ingress node, i.e. the router Ra/Rb which are adjacent to 5G UPF (or Packet Session anchor point-PSA), can use Flow ID (in IPv6 header) or UDP/TCP port number combined with Source Address to enforce packets in one flow being placed in one tunnel to one Egress router.  All other nodes in the network don’t need to take any extra action.  
Does it address your concern?

The Path Selection Behavior added is for the Item 1 above. 

The draft-dunbar-6man-5g-edge-compute-sticky-service is for Item 3 above. 

Thanks, Linda

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Thursday, February 11, 2021 11:48 AM
To: Linda Dunbar <[email-masked]>; [email-masked]; Aijun Wang <[email-masked]>; Acee Lindem (acee) <[email-masked]>; Stephane Litkowski <[email-masked]>
Subject: Re: [Idr] What do you think about this description of Path Selection added to draft-dunbar-idr-5g-edge-compute-app-meta-data?

Linda, changing towers is not the only case which needs stickiness. 
Given that Routing can change, and the attributes of servers can change, the whole system needs stickiness.
And once you have solved that level of stickiness I believe it is almost inherent that one no longer needs to put the application attributes or any enhancements to anycast into routing.

Yours,
Joel

On 2/11/2021 12:39 PM, Linda Dunbar wrote:
> To: Linda Dunbar <[email-masked]>; [email-masked]; Aijun 
> Wang <[email-masked]>; Acee Lindem (acee) <[email-masked]>; 
> Stephane Litkowski <[email-masked]>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-dunbar-6man-5g-edge-compute-sticky-service%2F&amp;data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4692baaedc1944adbf6808d8ceb51eb4%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637486624592889060%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=ZbV%2FiRJRpgPRxRkizYxeKLJilBj9U%2FtEGkF38aEwM7Q%3D&amp;reserved=0  describes approach to achieve stickiness , either by Hop by Hop or Destination Optional header.
> 
>