Subject: [Apn] Issues to be closed #33-#38

From: "Pengshuping (Peng Shuping)" <[email-masked]>
Date: Wed, 9 Feb 2022 02:32:05 +0000

Hi Adrian, all,

All the issues to be closed are going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted before, we post our responses to the issues #32-#38 this week. These issues are the last set of issues summarized in the Chairs' Summary section at the end of the BoF. Thanks to the Chairs for their sharp and clear summary which have captured almost all the key issues over the BoF.

Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

33. Why APN is needed when there are multiple existing mechanisms, just as DetNet and Network Slicing? #33
https://github.com/APN-Community/Issues/issues/33

In the latest version of this gap analysis draft https://datatracker.ietf.org/doc/html/draft-peng-apn-scope-gap-analysis-03, a few new subsections have been added to further clarify the differences between APN and the existing mechanisms such as DetNet and Network Slicing which are both resource partitioning techniques.

34. How privacy affects APN, especially about accidental breach of privacy and subversion of decapsulation? #34
https://github.com/APN-Community/Issues/issues/34

This draft https://datatracker.ietf.org/doc/draft-li-apn-framework/ specifies that the APN marking will be encapsulated in the outer tunnel encapsulation and removed together with the tunnel encapsulation at the end of each tunnel. In the same draft, we have also listed the APN Attribute Conveying Requirements https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1, the [REQ 1d] is that "APN ID MUST be acquired from the existing available information of the packet header without interference into the payload." APN works within a limited domain and the APN ID will be only meaningful to this network domain. The interpretation of the values of these fields is a local matter within the APN domain.

35. Use cases need to explain more about what is needed from the APN attribute and what policies are applied in the network. We need more detailed "killer" use case examples. #35
https://github.com/APN-Community/Issues/issues/35

In the latest version of this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04<https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6>, we provided an illustration section introducing a concrete use case, which covers the APN ID design, the APN ID configuration at the network edge, and the SLA guarantee in the backbone network such as network measurement and traffic steering. The benefits of using APN in this use case are also introduced.

Several concrete use cases that could benefit from APN have also been recorded in IETF drafts and already presented in previous meetings.
l  https://tools.ietf.org/html/draft-liu-apn-edge-usecase
l  https://tools.ietf.org/html/draft-zhang-apn-acceleration-usecase
l  https://tools.ietf.org/html/draft-yang-apn-sd-wan-usecase

36. The APN attribute should not become a way of carrying arbitrary metadata. It is not clear at this stage what information needs to be in the APN attribute versus what information could be in the APN attribute. #36
https://github.com/APN-Community/Issues/issues/36

Yes, we agree that the APN attribute should not become a way of carrying arbitrary metadata. The APN Header is specified in this draft https://datatracker.ietf.org/doc/draft-li-apn-header/. The carried APN ID and the APN Parameters are also defined. The APN Attribute Conveying Requirements are also specified in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1.

37. We also need more understanding of how APN is relevant in encrypted environments.  #37
https://github.com/APN-Community/Issues/issues/37

The APN attribute is acquired based on the existing information in the packet header such as 5-tuple (might be 2-tuple in an encrypted environment) and QinQ (S-VLAN and C-VLAN) at the edge devices of the APN domain, added to the data packets along with the tunnel encapsulation. This means that APN does not need to rely on the encrypted payload of the packet. Moreover, in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1, we have also listed the APN Attribute Conveying Requirements, the [REQ 1d] is that "APN ID MUST be acquired from the existing available information of the packet header without interference into the payload."

38. Should APN be applied to multiple transport/underlay protocols or should it be better to pick just one and use it in all APN-enabled networks? #38
https://github.com/APN-Community/Issues/issues/38

Currently the APN Header and its encapsulation in the IPv6 data plane are described in https://datatracker.ietf.org/doc/draft-li-apn-header/ and https://datatracker.ietf.org/doc/draft-li-apn-ipv6-encap/, respectively.

We want to focus on the IPv6 data plane where we believe APN will be most useful. However, in developing the APN Header, we want to do it in a generic way so that it is available for use in other data plane encapsulations should that turn out to be valuable. Designing a generic APN Header will not damage its value and use in IPv6, and seems to be the correct architectural solution to be future-proofed and most useful within the IETF.

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Tuesday, January 25, 2022 4:30 PM
To: apn <[email-masked]>
Cc: [email-masked]; [email-masked]; Dhruv Dhody <[email-masked]>; [email-masked]; [email-masked]; [email-masked]
Subject: [Apn] Issues to be closed #27-#32

Hi all,

All the issues to be closed are going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #27-#32 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

27. What does APN bring that isn't already being done within detnet? From: Lou Berger #27
https://github.com/APN-Community/Issues/issues/27

A new subsection on Detnet has been added to the gap analysis draft to especially clarify on the differences between Detnet and APN, https://datatracker.ietf.org/doc/html/draft-peng-apn-scope-gap-analysis-03#section-6.7.

For DetNet IP flow identification the IP 6-tuple is used, which consists of SourceIpAddress,   DestinationIpAddress, Dscp, Protocol, SourcePort, and DestinationPort, that is, there is no a kind of Flow ID carried for Detnet, but only the 6-tuple is directly used to identify the Detnet flows. While APN uses the APN ID to differentiate the traffic flows. APN can be used together with Detnet, that is, the APN ID is used to steer the traffic into a corresponding Detnet path which can guarantee latency boundary.

28. What does the APN architecture add that isn't in existing IETF architectures and solutions? From: Lou Berger #28
https://github.com/APN-Community/Issues/issues/28

The APN framework is described in this https://datatracker.ietf.org/doc/draft-li-apn-framework/, while the APN Header and its encapsulation in the IPv6 data plane are described in both https://datatracker.ietf.org/doc/draft-li-apn-header/ and https://datatracker.ietf.org/doc/draft-li-apn-ipv6-encap/. To further clarify the differences between the existing IETF work and the APN work, there is also this gap analysis draft https://datatracker.ietf.org/doc/draft-peng-apn-scope-gap-analysis/ for people's reference.

29. Why do we need an agnostic mechanisms instead of just hacking into existing mechanism individually? This adds to why having a single WG to focus on a technology agnostic mechanism would be useful before various data plane encapsulations are developed separately. From: Dhruv Dhody #29 https://github.com/APN-Community/Issues/issues/29

The APN Header is described in the draft https://datatracker.ietf.org/doc/draft-li-apn-header/, and it can be encapsulated in different data planes as requested in different network scenarios, e.g. its encapsulation in the IPv6 data plane as described in this draft https://datatracker.ietf.org/doc/draft-li-apn-ipv6-encap/.

In the APN wiki https://datatracker.ietf.org/wg/apn/documents/, we can see that currently the APN's work covers data plane, control plane and management plane. Although they are just some initial work, it can already show that APN is going to be a self-contained set of work. Doing this work without a common focus point would be very difficult to coordinate across multiple WGs. Taking IOAM and SPRING as examples, it is important to have a common place (IPPM, SPRING WG) where the consensus can be developed before extending various data planes across WGs that own it.

30. Looking 6437 IPv6 flow label and DetNet, they already solve this problem. Don't need another mechanism. If a problem does exist it does not have to be solve in the data plane, it could be the management plane. From: Greg Mirsky #30
https://github.com/APN-Community/Issues/issues/30

The gap analysis draft https://datatracker.ietf.org/doc/draft-peng-apn-scope-gap-analysis/ introduces the differences from the existing work and explained why IPv6 flow label and Detnet could not solve the problem. In theory the management plane could be used, but using the data plane is more straight forwards.

31. What different treatment will the network give my traffic if I'm in finance vs. marketing? From: Benjamin Kaduk #31
https://github.com/APN-Community/Issues/issues/31

We had provided an illustration section in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6, describing a concrete use case and the possible treatment such as SLA guarantee in the backbone network to the traffic from different departments https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6.5.

32. Is there violation of the user's privacy/security? From: Kireeti Kompella #32
https://github.com/APN-Community/Issues/issues/32

In the draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04, we have listed the APN Attribute Conveying Requirements https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1, the [REQ 1d] is that "APN ID MUST be acquired from the existing available information of the packet header without interference into the payload." In the Section 6 of the same draft, the design of the APN ID is also described as https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6.2 including user group id and application group id. The individual user will not be exposed.

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Monday, January 17, 2022 5:45 PM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Jeffrey (Zhaohui) Zhang <[email-masked]<mailto:[email-masked]>>; Bernier, Daniel <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #22-#26

Hi all,

All the issues to be closed are going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #22-#26 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

22. Why copying inner TOS to outer TOS and using existing equipment is not enough? From: Dino Farinacci #22
https://github.com/APN-Community/Issues/issues/22

We had provided an illustration section in this draft, and its second subsection also gave an example on the design of the APN ID, https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6. From this illustration, we can see why the TOS or DSCP would not be enough. What APN ID can be used for can be found in the use case drafts, and more are being explored.

Moreover, in most of the current deployments, the inner TOS is not simply copied to outer TOS and used in the network. Generally the operator's network edge will retag a TOS based on its own strategy and configuration.

23. Is APN able to express such "policies" so that a developer does not hardcode DSCP bits with a Excel spreadsheet on its desk to know what mapping means what? From: Daniel Bernier #23
https://github.com/APN-Community/Issues/issues/23

The APN header is defined in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-header-00#page-3. It does not express "policies" directly. Even if the "policies" are carried in the packets, the mapping would still be needed at the network nodes.

24. Is APN similar to or the same as Network Slice, just with a different name? From: Zhang Zhaohui/Tarek Saad #24
https://github.com/APN-Community/Issues/issues/24

They are different technologies. In the case of network slicing, the APN header is used to differentiate the traffic and it can be used to steer the corresponding traffic into appropriate network slice. However, the APN header can also be used in other scenarios such as fine-granular performance measurement etc..

25. Is the gap that existing approaches, e.g., Network Slice, only provide limited granularity but APN an unlimited one? From: Dirk Hugo #25
https://github.com/APN-Community/Issues/issues/25

APN and Network Slicing have different focuses. Network slicing is for separating the network resources while APN is for differentiating traffic. Both technologies can be used together or we don't have to. APN can suit more other different scenarios as described in this draft draft-li-apn-problem-statement-usecases-05.

26. Is slicing more general because addressing MULTI operator scenarios? From: Rüdiger Volk #26
https://github.com/APN-Community/Issues/issues/26

APN does not consider MULTI operator scenarios. APN only consider one operator's controlled and limited domain or multiple domains but still within a single operator.

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Monday, January 10, 2022 3:56 PM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: Black, David <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #19-#21

Hi all,

All the issues to be closed are going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #19-#21 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

19. Is that "structured attribute" more accurate than "tag"? How resulting resolution complexity will compare with 5-tuples being used? From: David Black #19
https://github.com/APN-Community/Issues/issues/19

The "structured attribute" (i.e. the APN Header) is defined in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-header-00#page-3. The APN header option is going to be located in one place in the packet header such as an IPv6 extension header - Hop-by-hop Options header. To resolve this APN header, only one place in the packet header needs to be checked against. While since 5-tuples are located in the various places in the packet header, especially in IPv6 data plane when IPv6 extension headers exist the transport layer information is being pushed further deep, these will make the resolution of the 5 tuples very hard and more complex than that of the APN header.

20. Does APN ID carry a piece of information that is potentially semantically *richer* than the five tuple and making that available to path elements that would not otherwise have that data? From: Ted Hardie #20
https://github.com/APN-Community/Issues/issues/20

The path elements that are not aware of APN or do not support APN will not be able to process the APN header since there are no corresponding configurations. The APN ID will only be processed in the nodes that are already enforced with policies against the APN ID.

21. How can the edge routers estimate APN ID and parameters (latency, bandwidth, etc.) just from 5-tuple? Is there any interface or API between application and APN domain controller? From: Takuya Miyasaka #21
https://github.com/APN-Community/Issues/issues/21

The edge routers will perform based on the configurations which indicates the matching relationship between the 5-tuple and the APN ID and/or parameters. The configurations could be enforced by the APN domain controller, which may be based on the agreements established between the services and the network operators, as illustrated in https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6.

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Tuesday, January 4, 2022 9:48 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Bob Hinden <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #16-#18

Hi all,

All the issues to be closed are going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #16-#18 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

16. What is the mechanism that forces this attribute to be stripped at the network operator's boundary? From: Alissa Cooper/ Benjamin Kaduk/ Jana Iyengar #16
https://github.com/APN-Community/Issues/issues/16

This draft https://datatracker.ietf.org/doc/draft-li-apn-framework/ specifies that the APN marking will be encapsulated in the outer tunnel encapsulation and removed together with the tunnel encapsulation at the end of each tunnel.

17. Is this literally a mechanism for creating and sharing arbitrary metadata about arbitrary aggregates across arbitrary boundaries, which creates as much or more room for trouble? From: Jana Iyengar #17
https://github.com/APN-Community/Issues/issues/17

APN works within a single operator's limited and controlled domain(s). There could be multiple domains within this operator, which is a normal deployment case. The APN attributes are encapsulated in the outer tunnel header to trigger the policy enforcement for the network service provisioning in a flexible and efficient way in the various nodes along the tunnel.

18. Could the flow label be used for APN, which is designed for similar purpose? From: Bob Hinden #18
https://github.com/APN-Community/Issues/issues/18

Using the IPv6 Flow Label for ECMP (RFC6438) has turned out to be the most widely deployed usage of the IPv6 flow label, even in the tunnel mode. In order to make the IPv6 Flow Label usable for other services, we would have a lot to do first before we could start reusing it.

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Tuesday, December 28, 2021 8:54 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: Black, David <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #13-#15

Hi all,

All the issued to be closed is going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #13-#15 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

13. Is opaque used solely wrt privacy of info from which APN ID is derived? Is that just opaque lookup based on ID? From: David Black #13
https://github.com/APN-Community/Issues/issues/13

In this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04, it states that "the APN attribute is acquired based on the existing information in the packet header such as 5-tuple and QinQ (S-VLAN and C-VLAN) at the edge devices of the APN domain".

There is also a requirement [REQ 1d] in 5.1. APN Attribute Conveying Requirements,
https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1
[REQ 1d].  APN ID MUST be acquired from the existing available information of the packet header without interference into the payload.

14. What is opaque lookup? Which parameters are supposed to use? What exactly are you looking? From: David Black #14
https://github.com/APN-Community/Issues/issues/14

The opaque lookup means about the lookup using APN ID. The APN ID itself does not have any privacy info but only a string of bits used by the network devices to perform policy enforcements locally.

15. Is opaque contradictory to the APN parameters? Does such a "treat as opaque" case actually use the full richness of attributes? From: Zhang Zhaohui/Benjamin Kaduk #15
https://github.com/APN-Community/Issues/issues/15

Opaque is for the APN ID. The APN parameters are used to express more detailed requirements on the network. It is not contradictory.

**********************************************
Happy New Year to All! :)
**********************************************

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Monday, December 20, 2021 9:39 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #10-#12

Hi all,

All the issued to be closed is going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #10-#12 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

10. Is APN designed for a single domain or multiple domains? From: Charles Eckel #10
https://github.com/APN-Community/Issues/issues/10

APN is designed for a single operator's limited and controlled domain(s). There could be multiple domains within this operator. At the provider edge node of each domain, the APN attribute could be used to steer the traffic into corresponding network services such as an explicit SRv6 path.

11. Is it a normal case that 1 SP has multiple domains? From: Greg Mirsky #11
https://github.com/APN-Community/Issues/issues/11

Yes, it is a very normal case that one SP has multiple domains under its administration, especially when the network scale is large.

12. Is that the entire value of these APN tunnels is to communicate information across domains? From: Jana Iyengar #12
https://github.com/APN-Community/Issues/issues/12

We would not call these tunnels are APN tunnels. They are the normal tunnels such as SRv6 policies or MPLS tunnels. APN attributes are encapsulated in the outer tunnel header to trigger the policy enforcement for the network service provisioning in a flexible and efficient way in the various nodes along the tunnel.

**********************************************
How time flies! Another Christmas is coming.
Merry Christmas to all! :)
**********************************************

Best Regards,
Shuping

From: Apn [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Monday, December 13, 2021 9:59 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [Apn] Issues to be closed #5-#9

Hi all,

All the issued to be closed is going to be listed in this link https://github.com/APN-Community/Issues/issues.

Following the issues we posted in the last week, we post our responses to the issues #5-#9 this week. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

5. If the APN is trying to make app get better service for it from the network, how does it do that? Especially when every app want better service? From: Dino Farinacci #5
https://github.com/APN-Community/Issues/issues/5

Following the APN framework as defined in the https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04, the traffic are tagged with APN ID at the network edge, against which the traffic can be steered into the network services such as SRv6 policies that can satisfy their various SLA requirements.

6. What is the problem domain of APN? What is the problem APN is trying to solve? From: Rick Taylor #6
https://github.com/APN-Community/Issues/issues/6

In this problem statement draft https://datatracker.ietf.org/doc/html/draft-li-apn-problem-statement-usecases-04, the three challenges that APN is trying to target at are listed as followings,

1.         Challenges of lack of fine-granularity service information

2.         Challenges of Traditional Differentiated Service Provisioning

3.         Challenges of Supporting New 5G and Edge Computing Technologies

Several concrete use cases that could benefit from APN have also been recorded in IETF drafts and presented in previous meetings.

l  https://tools.ietf.org/html/draft-liu-apn-edge-usecase

l  https://tools.ietf.org/html/draft-zhang-apn-acceleration-usecase

l  https://tools.ietf.org/html/draft-yang-apn-sd-wan-usecase

The work items to be covered were presented in the following slides,
https://datatracker.ietf.org/meeting/111/materials/slides-111-apn-8-apn-work-items-00.

7. Why do we need an abstract container for service info across all tunneling mechanisms? From: Watson Ladd #7
https://github.com/APN-Community/Issues/issues/7

As presented in the APN BoF @IETF111, from this use case we can see that the carried information can be used to trigger the IOAM performance measurement or perform fine-granular traffic steering at the edge of the intermediate domain without the need to further resolve the 5-tuple of the inner packets.
https://datatracker.ietf.org/meeting/111/materials/slides-111-apn-apn-use-cases-01

8. Do we have a protocol mechanism to enforce what's in the APN marking and when it's removed? From: Benjamin Kaduk #8
https://github.com/APN-Community/Issues/issues/8

The draft https://datatracker.ietf.org/doc/draft-li-apn-header/ specifies the APN header which includes the APN ID and/or parameters, that is, the APN marking.

This draft https://datatracker.ietf.org/doc/draft-li-apn-framework/ specifies that the APN marking will be encapsulated in the outer tunnel encapsulation and removed together with the tunnel encapsulation at the end of each tunnel.

9. How large is the space of APN attributes? If the existing field is enough to identify and individual (e.g. by physical port), could the attribute carry a user identifier? From: Ted Hardie #9
https://github.com/APN-Community/Issues/issues/9

The draft https://datatracker.ietf.org/doc/draft-li-apn-header/ specifies two types of APN ID which have different lengths. APN is not used to identify individuals. Usually a physical port can be used to identify a group of users, and the user group ID can be carried in APN.

Best Regards,
Shuping

From: Pengshuping (Peng Shuping)
Sent: Tuesday, December 7, 2021 9:45 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Lars Eggert <[email-masked]<mailto:[email-masked]>>; [email-masked]' <[email-masked]<mailto:[email-masked]>>; 'Bernier, Daniel' <[email-masked]<mailto:[email-masked]>>
Subject: Issues to be closed #1-#4

Hi all,

All the issued to be closed is going to be listed in this link https://github.com/APN-Community/Issues/issues.

In the following weeks, we are going to post our responses to the issues. Please either leave your comments in the mailing list or directly in the github, so we can finally close these issues. Thank you!

1. What happens to a flow if a hop can't meet the APN requirements? From: Lars Eggert #1
https://github.com/APN-Community/Issues/issues/1

As described in https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5, the APN requirements include APN Attribute Conveying Requirements and APN attribute Handling Requirements.

In the APN attributes, the carrying of the APN parameters is optional as stated in https://datatracker.ietf.org/doc/html/draft-li-apn-header-00#section-3. The typical APN parameters are the network performance requirements such as bandwidth, latency, etc.

If a hop cannot meet the APN requirements, which would mean that the hop cannot handle the APN attributes, then it will be up to the local configuration. We can explore more on this topic, but generally the flow needs to be forwarded without any interruption, probably in a default mode.

2. Does APN introduce a new data plane when it is supposed to work with any data plane? From: Sergey Fomin #2
https://github.com/APN-Community/Issues/issues/2

APN does not introduce a new data plane. APN uses and makes necessary extensions to the existing data plane to carry the APN header as defined in https://datatracker.ietf.org/doc/draft-li-apn-header/. The encapsulation example on the IPv6 data plane is suggested in https://datatracker.ietf.org/doc/draft-li-apn-ipv6-encap/.

3. Do we need to define a new data plane to carry this info, while you could use existing mechanisms? From: Sergey Fomin #3
https://github.com/APN-Community/Issues/issues/3

Some extensions may be needed. For example, new IPv6 HBH or DOH options would need to be defined to carry the APN header in the IPv6 data plane.
In https://datatracker.ietf.org/doc/draft-peng-apn-scope-gap-analysis/, we listed existing mechanisms and made some analysis and comparisons.

4. Is it expected that the APN ID and parameters be "normalized' (standardized) or be defined domain specific From: Daniel Bernier #4

The draft https://datatracker.ietf.org/doc/draft-li-apn-header/ specifies the APN header which includes the APN ID and parameters. APN works within an operator's controlled and limited domain, so the APN ID and parameters can be defined as domain specific.

Best Regards,
Shuping

From: Architecture-discuss [mailto:[email-masked]] On Behalf Of Pengshuping (Peng Shuping)
Sent: Monday, December 6, 2021 9:17 AM
To: apn <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [arch-d] Issues to be closed

Dear all,

Following the summary report on the APN@IETF112 [1], we list the questions/comments we received during the BoF as recorded in the meeting minutes [2] as below. Please have a look at these questions and let us know if there are any other key questions being missed out.

We are going to start answering these questions and try to close them one by one. Your attention and participation into the discussions are very welcomed. The tool would be the Github issue tracker. If you have better suggestion please let us know. Thank you!

[1] https://mailarchive.ietf.org/arch/msg/apn/OoOgezkAAbd2uFrY2Mk4ZxSbVzM/
[2] https://datatracker.ietf.org/meeting/111/materials/minutes-111-apn-00.txt

Part 1: General Questions

1.       What happens to a flow if a hop can't meet the APN requirements?

2.       Does APN introduce a new data plane when it is supposed to work with any data plane?

3.       Do we need to define a new data plane to carry this info, while you could use existing mechanisms?

4.       Is it expected that the APN ID and parameters be "normalized' (standardized) or be defined domain specific?

5.       If the APN is trying to make app get better service for it from the network, how does it do that? Especially when every app want better service?

6.       What is the problem domain of APN? What is the problem APN is trying to solve?

7.       Why do we need an abstract container for service info across all tunneling mechanisms?

8.       Do we have a protocol mechanism to enforce what's in the APN marking and when it's removed?

9.       How large is the space of APN attributes? If the existing field is enough to identify and individual (e.g. by physical port), could the attribute carry a user identifier?

Part 2: APN Domain

10.   Is APN designed for a single domain or multiple domains?

11.   Is it a normal case that 1 SP has multiple domains?

12.   Is that the entire value of these APN tunnels is to communicate information across domains?

Part 3: Opaque

13.   Is opaque used solely wrt privacy of info from which APN ID is derived? Is that just opaque lookup based on ID?

14.   What is opaque lookup? Which parameters are supposed to use? What exactly are you looking?

15.   Is opaque contradictory to the APN parameters? Does such a "treat as opaque" case actually use the full richness of attributes?

Part 4: Security/Privacy

16.   What is the mechanism that forces this attribute to be stripped at the network operator's boundary?

17.   Is this literally a mechanism for creating and sharing arbitrary metadata about arbitrary aggregates across arbitrary boundaries, which creates as much or more room for trouble?

Part 5: Flow Label

18.   Could the flow label be used for APN, which is designed for similar purpose?

Part 6: 5 Tuple

19.   Is that "structured attribute" more accurate than "tag"? How resulting resolution complexity will compare with 5-tuples being used?

20.   Does APN ID carry a piece of information that is potentially semantically *richer* than the five tuple and making that available to path elements that would not otherwise have that data?

21.   How can the edge routers estimate APN ID and parameters (latency, bandwidth, etc.) just from 5-tuple? Is there any interface or API between application and APN domain controller?

Part 6: Diffserv

22.   Why copying inner TOS to outer TOS and using existing equipment is not enough?

23.   Is APN able to express such "policies" so that a developer does not hardcode DSCP bits with a Excel spreadsheet on its desk to know what mapping means what?

Part 7: Network Slicing

24.   Is APN similar to or the same as Network Slice, just with a different name?

25.   Is the gap that existing approaches, e.g., Network Slice, only provide limited granularity but APN an unlimited one?

26.   Is slicing more general because addressing MULTI operator scenarios?

Part 8: DetNet

27.   What does APN bring that isn't already being done within DetNet?

Part 9: Open Mic

28.   What does the APN architecture add that isn't in existing IETF architectures and solutions?

29.   Why do we need an agnostic mechanisms instead of just hacking into existing mechanism individually? This adds to why having a single WG to focus on a technology agnostic mechanism would be useful before various data plane encapsulations are developed separately?

30.   If a problem does exist it does not have to be solved in the data plane, it could be the management plane.

31.   What different treatment will the network give my traffic if I'm in finance vs. marketing?

32.   Is there violation of the user's privacy/security?

Part 10: Chairs Summary (may overlap with previous questions as a summary)

33.   Why APN is needed when there are multiple existing mechanisms, just as DetNet and Network Slicing?

34.   How privacy affects APN, especially about accidental breach of privacy and subversion of decapsulation?

35.   Use cases need to explain more about what is needed from the APN attribute and what policies are applied in the network. We need more detailed "killer" use case examples.

36.   The APN attribute should not become a way of carrying arbitrary metadata. It is not clear at this stage what information needs to be in the APN attribute versus what information could be in the APN attribute.

37.   We also need more understanding of how APN is relevant in encrypted environments.

38.   Should APN be applied to multiple transport/underlay protocols or should it be better to pick just one and use it in all APN-enabled networks?

Best Regards,
Shuping

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 17 Feb 2022 14:20:33 -0000

Hi Shuping,

I think I missed responding to this. Thanks for your patience.

> All the issues to be closed are going to be listed in this link

> https://github.com/APN-Community/Issues/issues. 

> Following the issues we posted before, we post our responses

> to the issues #32-#38 this week. These issues are the last set

> of issues summarized in the Chairs' Summary section at the

> end of the BoF. Thanks to the Chairs for their sharp and clear

> summary which have captured almost all the key issues over

> the BoF. 

I really like your mode of work here: driving through the issues but raising
them on the list to ensure that there is visibility.

> Please either leave your comments in the mailing list or directly

> in the github, so we can finally close these issues. Thank you!

Responses and thoughts in line.

> 33. Why APN is needed when there are multiple existing

> mechanisms, just as DetNet and Network Slicing? #33

> In the latest version of this gap analysis draft 

https://datatracker.ietf.org/doc/html/draft-peng-apn-scope-gap-analysis-03,

> a few new subsections have been added to further

> clarify the differences between APN and the existing

> mechanisms such as DetNet and Network Slicing which

> are both resource partitioning techniques.

This is a really important distinction between resource partitioning and
traffic steering.

I see Sections 6.7 (DetNet) and 6.8 (Network Slicing), and these are fine
(IMHO) descriptions of the uses of identifiers in those technologies. 

But I don't see the term "resource partition" anywhere. The term "steering"
is used well in the scope discussion.

You may find a reference to draft-ietf-teast-rfc3272bis helpful to
distinguish these different concepts.

(Although, maybe some of this belongs in the APN framework draft.)

By the way, while I had the draft open, I notice you're using the old
version of the BCP 14 boilerplate, and you need to move it from the document
header down into Section 2.1

> 34. How privacy affects APN, especially about accidental breach

> of privacy and subversion of decapsulation? #34

> This draft https://datatracker.ietf.org/doc/draft-li-apn-framework/

> specifies that the APN marking will be encapsulated in the outer

>tunnel encapsulation and removed together with the tunnel

> encapsulation at the end of each tunnel. In the same draft, we

> have also listed the APN Attribute Conveying Requirements

https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1,

> the [REQ 1d] is that "APN ID MUST be acquired from the

> existing available information of the packet header without

> interference into the payload." APN works within a limited

> domain and the APN ID will be only meaningful to this network

> domain. The interpretation of the values of these fields is a local

> matter within the APN domain.

Parts of this are very good.

-          The APN ID does not use information from within the packet
payload

-          The APN ID only uses information from the packet header

-          The context of the APN ID is limited to within the APN domain

However, the APN ID is set according to policies within the domain. For
example, the policy may be that a certain pair of sender/receiver addresses
comprise a priority flow between prestigious users. That policy would be
converted to a specific APN ID (or more likely APN Attribute including an
APN ID) that would get special treatment in the network. So an attacker
might learn about the APN markings that indicate priority traffic, spot such
packets, and look to see who is sending/receiving them.

Thus, the really important part of what you have said is that APN is only
applied to an edge-to-edge IP-in-IP encapsulation. That is, you tunnel
across the APN domain.

What that means is that the source and destination addresses of the packet
are the end points of the tunnel (i.e., the domain edges), and nothing about
the payload source and destination can be deduced.

Furthermore (and I don't think you say this), encryption could be used on
the encapsulated packet as it is carried across the APN domain. That would
mean that someone inspecting a packet, discovering it is receiving "gold"
treatment, and trying to work out who/what qualifies as "gold" would not be
able to see the end-to-end packet.

This substantially reduces the privacy concerns.

I still wonder whether the write-up of these points could be clearer and
easier. It is enough that you have addressed all the issues, but it is
better if the reader can see the concern and then be lead step by step
through the solution.

> 35. Use cases need to explain more about what is needed from the APN

> attribute and what policies are applied in the network. We need more

> detailed "killer" use case examples. #35

> In the latest version of this draft

> https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04
<https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6>
,

> we provided an illustration section introducing a concrete

> use case, which covers the APN ID design, the APN ID configuration

> at the network edge, and the SLA guarantee in the backbone

> network such as network measurement and traffic steering.

> The benefits of using APN in this use case are also introduced. 

Yes! That was what was needed.

> Several concrete use cases that could benefit from APN have

> also been recorded in IETF drafts and already presented in previous

> meetings. 

> - https://tools.ietf.org/html/draft-liu-apn-edge-usecase  

> - https://tools.ietf.org/html/draft-zhang-apn-acceleration-usecase  

> - https://tools.ietf.org/html/draft-yang-apn-sd-wan-usecase 

These are good documents, but what they don't do is say "Therefore we need
the APN Attribute to carry the following information."

> 36. The APN attribute should not become a way of carrying

> arbitrary metadata. It is not clear at this stage what information

> needs to be in the APN attribute versus what information could

> be in the APN attribute. #36

> Yes, we agree that the APN attribute should not become a way

> of carrying arbitrary metadata. The APN Header is specified in

> this draft https://datatracker.ietf.org/doc/draft-li-apn-header/. 

> The carried APN ID and the APN Parameters are also defined. 

> The APN Attribute Conveying Requirements are also specified

> in this draft
https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1.

Right, so the header draft defines a field that can contain APN parameters.
This is, free-format data and you can understand why some people might think
that *any* data could be included in there.

In fact, the content of the APN-Para field is determined by the setting of
the APN-Para-Type field (a 16 bit field) enabling 16 different parameter
formats to be defined and included. 

What stops me writing a draft tomorrow describing bit 7 to mean "User's home
phone number, height, shoe size, computer serial number"? Or any other
meta-data? Indeed, why would I even need to write a draft?

So, there is another constraint: each APN Parameter field is exactly 32
bits. That means I would need several fields to carry my metadata. But
still, the option is open.

I think you can tighten this up by:

-          Asking for an IANA registry to track the bits

-          Making the assignment policy for that registry "IETF Review"

This does not stop arbitrary metadata, but it gives the IETF control over
it.

You might also describe the expected behavior when an APN-Para-Type field is
set and the processing node does not understand the meaning. I think that,
because each Parameter field is 32 bits, I can simply ignore and step over
the parameters I don't understand.

> 37. We also need more understanding of how APN is

> relevant in encrypted environments.  #37

> The APN attribute is acquired based on the existing information in

> the packet header such as 5-tuple (might be 2-tuple in an encrypted

> environment) and QinQ (S-VLAN and C-VLAN) at the edge devices of

> the APN domain, added to the data packets along with the tunnel

> encapsulation. This means that APN does not need to rely on the

> encrypted payload of the packet. Moreover, in this draft

https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1,

> we have also listed the APN Attribute Conveying Requirements,

> the [REQ 1d] is that "APN ID MUST be acquired from the existing

> available information of the packet header without interference

> into the payload." 

Right. It is not just the encryption of the payload, but those three fields
of the 5-tuple.

As you say, if there is encryption, the APN processing will be limited to
less information.

I suggest that you turn the text the other way round so that you say that
APN processing is based on:

-          Source and destination addresses

-          Incoming L2 (or MPLS) encapsulation

-          Incoming physical/virtual port information

-          The other fields of the 5-tuple if they are not encrypted

I think you also have to say that if the APN packet is itself encrypted
(e.g., the APN tunnel across the APN domain uses IPsec) then the APN
Attribute MUST NOT be encrypted.

> 38. Should APN be applied to multiple transport/underlay protocols

> or should it be better to pick just one and use it in all APN-enabled 

> networks? #38

> Currently the APN Header and its encapsulation in the IPv6 data plane

> are described in https://datatracker.ietf.org/doc/draft-li-apn-header/ 

> and https://datatracker.ietf.org/doc/draft-li-apn-ipv6-encap/, 

> respectively.

> We want to focus on the IPv6 data plane where we believe APN will

> be most useful. However, in developing the APN Header, we want to

> do it in a generic way so that it is available for use in other data plane

> encapsulations should that turn out to be valuable. Designing a

> generic APN Header will not damage its value and use in IPv6, and 

> seems to be the correct architectural solution to be future-proofed

> and most useful within the IETF.

I agree with this approach. It seems sensible.

It would be good to be sure that the drafts say this clearly.

Thanks,

Adrian

---

From: "Pengshuping (Peng Shuping)" <[email-masked]>
Date: Mon, 21 Feb 2022 07:46:39 +0000

Dear Adrian,

Truly appreciate your thoughtful reply and valuable suggestions!

We will revise and update the drafts accordingly to capture all the key points here.

Thank you!

Best Regards,
Shuping

From: Adrian Farrel [mailto:[email-masked]]
Sent: Thursday, February 17, 2022 10:21 PM
To: Pengshuping (Peng Shuping) <[email-masked]>; 'apn' <[email-masked]>
Cc: [email-masked]; 'Bob Hinden' <[email-masked]>; 'Daniel Voyer' <[email-masked]>
Subject: RE: Issues to be closed #33-#38

Hi Shuping,

> summary which have captured almost all the key issues over

I really like your mode of work here: driving through the issues but raising them on the list to ensure that there is visibility.
Shuping> Thank you.

> are both resource partitioning techniques.

This is a really important distinction between resource partitioning and traffic steering.
I see Sections 6.7 (DetNet) and 6.8 (Network Slicing), and these are fine (IMHO) descriptions of the uses of identifiers in those technologies.
But I don't see the term "resource partition" anywhere. The term "steering" is used well in the scope discussion.
You may find a reference to draft-ietf-teast-rfc3272bis helpful to distinguish these different concepts.

By the way, while I had the draft open, I notice you're using the old version of the BCP 14 boilerplate, and you need to move it from the document header down into Section 2.1

-          The APN ID does not use information from within the packet payload

However, the APN ID is set according to policies within the domain. For example, the policy may be that a certain pair of sender/receiver addresses comprise a priority flow between prestigious users. That policy would be converted to a specific APN ID (or more likely APN Attribute including an APN ID) that would get special treatment in the network. So an attacker might learn about the APN markings that indicate priority traffic, spot such packets, and look to see who is sending/receiving them.

Thus, the really important part of what you have said is that APN is only applied to an edge-to-edge IP-in-IP encapsulation. That is, you tunnel across the APN domain.
What that means is that the source and destination addresses of the packet are the end points of the tunnel (i.e., the domain edges), and nothing about the payload source and destination can be deduced.

Furthermore (and I don't think you say this), encryption could be used on the encapsulated packet as it is carried across the APN domain. That would mean that someone inspecting a packet, discovering it is receiving "gold" treatment, and trying to work out who/what qualifies as "gold" would not be able to see the end-to-end packet.

I still wonder whether the write-up of these points could be clearer and easier. It is enough that you have addressed all the issues, but it is better if the reader can see the concern and then be lead step by step through the solution.

> https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04<https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-6>,

> - https://tools.ietf.org/html/draft-yang-apn-sd-wan-usecase

These are good documents, but what they don't do is say "Therefore we need the APN Attribute to carry the following information."

> in this draft https://datatracker.ietf.org/doc/html/draft-li-apn-framework-04#section-5.1.

Right, so the header draft defines a field that can contain APN parameters. This is, free-format data and you can understand why some people might think that *any* data could be included in there.

In fact, the content of the APN-Para field is determined by the setting of the APN-Para-Type field (a 16 bit field) enabling 16 different parameter formats to be defined and included.

What stops me writing a draft tomorrow describing bit 7 to mean "User's home phone number, height, shoe size, computer serial number"? Or any other meta-data? Indeed, why would I even need to write a draft?

So, there is another constraint: each APN Parameter field is exactly 32 bits. That means I would need several fields to carry my metadata. But still, the option is open.

This does not stop arbitrary metadata, but it gives the IETF control over it.

You might also describe the expected behavior when an APN-Para-Type field is set and the processing node does not understand the meaning. I think that, because each Parameter field is 32 bits, I can simply ignore and step over the parameters I don't understand.

> available information of the packet header without interference

Right. It is not just the encryption of the payload, but those three fields of the 5-tuple.
As you say, if there is encryption, the APN processing will be limited to less information.

I suggest that you turn the text the other way round so that you say that APN processing is based on:

I think you also have to say that if the APN packet is itself encrypted (e.g., the APN tunnel across the APN domain uses IPsec) then the APN Attribute MUST NOT be encrypted.

Thanks,
Adrian