Subject: Reference based Routing (RbR)

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 14:58:48 +0200

Dear 6man WG,

Number of concurrent discussions in this WG triggered an idea which I would
like to share with you all to think about during the long weekend.

For some unknown reasons an army of folks has been cornered to think that
stuffing each packet with encoded or not, compressed or not hop by hop list
of nodes to traverse is best we can do to provide some form of path
engineering in some cases combined with network programming.

Some put on such lists IPv6 addresses, some try to just put pieces of those
addresses and some even try to squeeze mpls labels there under a different
cover (brrrr).

What if we come back to the roots and rethink the requirements ..

Few months back I wrote IP TE + NP proposal however the summary below is a
simplified version of it. Just for reference original document is here:
https://tools.ietf.org/html/draft-raszuk-teas-ip-te-np-00

*Description:*

Let's assume that in my network I would like to be able to construct 1000
engineered paths (different from what IGP would compute as SPF with its
default algorithm). Perhaps say that number of those paths would be 10x or
100x more. That means that essentially the paths itself could be expressed
in 16 bits.

Let's further observe that what is needed to accomplish any form of traffic
engineering are two elements:

* mark packets which are subject to such path engineering
* construct path engineered paths

Note that while I am using TE as an example additional network programming
functions including for example SFC processing is supported with no change
as well.

*Proposal: *

   - Let's allocate a /64 prefix which will be used in entire domain for
   the purpose of RbR
   - Let's mark each node in a domain with different /32 address (it is
   naturally ok to simply use current /32 IPv4 node addresses)
   - Let's use remaining 32 bits of the IPv6 address to carry an RbR pointer

To illustrate format of the address of the packet after it is encapsulated
on ingress into the domain looks like:

*|<----  /64 PREFIX ----->|<--- /32 LOCATOR --->|<--- /32 RbR POINTER --->|*

No need for any routing header at all ! No per packet overhead regardless
of the number of hops to traverse (except 40 bytes of IPv6 encapsulation).

What is Prefix & Locator is obvious. So now let's define Pointer:

*Pointer (or reference)* - /32 bits string which would uniquely in entire
domain identify the aggregates of flows. Indicates a reference to a set of
local actions for those packets which carrying such reference. Only those
routers which DA is in the /96 address need to inspect and execute the
pointer.

Pointers can be structured or unstructured - depending on
operators's choice and network requirements. In structured case for example
first 16 bits can represent 2^16 number of unique TE paths in the network
while remaining 16 bits local functions in each network elements (segment
endpoint). Such bit boundary is a local operators choice.

RbR pointers are assigned by the same entity (human or controller) which
normally is responsible for path engineering and network programming in the
domain.

RbR pointers are globally (domain wide) unique.

*However actions indicated by said RbR pointers are locally significant to
any network element. *

That is accomplished by propagating actions which correspond to such
references as tuple: LOCATOR + POINTER. Depending on the selected means of
propagation of such actions it also can be: PREFIX+LOCATOR + POINTER.

*Example: *

R1

/64 RbR prefix 2001:0db8:85a3:0001
/32 Node's locator:  1.1.1.1 (101:101)
/32 RbR pointer:  1::1

R2

/64 RbR prefix 2001:0db8:85a3:0001
/32 Node's locator:  1.1.1.2 (101:102)
/32 RbR pointer:  1::1

Actions programming:

101:101 + 1::1  --> rewrite DA to   2001:0db8:85a3:0001 + 101:102 + 1::1
101:102 + 1::1  --> rewrite DA to   2001:0db8:85a3:0001 + 101:103 + 1::1
etc ...

*Actions distribution:*

There are number of ways to distribute such instructions domain wide. One
network can choose to use configuration push via ansible automation. The
other may like to use existing BGP FlowSpec SAFI. Yet some other would like
to use OpenR message bus on a pub-sub basis.

Perhaps a yet one more set of engineers would like to get new BGP or IGP
extension defined.

At this stage all options are on the table.

*Scaling: *

As we all know there is no free lunch. Here I believe proposed solution is
a sweet spot compromise between soft state path setup in network elements
(MPLS RSVP-TE) and loading each packet with additional routing header
information.

Most importantly it needs to be observed that each participating midpoint
*only* needs to store actions which correspond to his own node. That
applies to both control plane(*) and especially local data plane RbR-FIB.
If action is send by any means towards a network element and does not
contain its LOCATOR in the first /32 bits it will be dropped either at
inbound processing or never propagated from control plane to RIB/FIB.

It has been studied that in real production networks to satisfy the
requirements of most path steering needs at most 2-3 sometimes 4 segments
may be needed. That effectively means that for such flows corresponding
actions need to be kept only in 2-4 routers in the entire domain. Rest of
the network elements may never need to be RbR aware as all they do is
forward based on /96 IPv6 prefix.

* For control plane it depends on the form of the distribution - if BGP is
used and given node is a route reflector all RbR action "routes" or "rules"
need to be stored there.

After all routers are build to carry state. IP prefixes are nothing by
state - more over - IP prefixes do change all the time (continues BGP
churn). So while thinking about scaling and state one needs to also keep in
mind the static nature of actions. Perhaps some TE paths will be pinned for
weeks or months and will never require a change of either control plane nor
RbR FIB.

Last RbR cost (packet tax) is constant and equals zero. It does not
increase if I need to engineer path at each of my 10 hops for specific
adjacency on each hop.

*Summarization: *

It is my strong believe that in any network any scheme must allow for
reachability summarization.

RbR locators can be easily summarized/aggregated across ASNs (under the
same administrative control) or across IGP areas/levels.

Packets traverse IP network as normal IPv6 packets would.

*Security: *

Most networks already have inbound ACLs (setup manually or dynamically) to
protect any external packet to enter their domain with destination address
equal to infrastructure space.

Here our /64 prefix can be either already part of this existing rule or a
single ACL line dropping all packets with DA = PREFIX or longer applied on
all external interfaces.

ACLs supporting basic IPv6 filtering is all what is required and that is
commonly supported by all IPv6 capable routers today.

*RbR-FIB:*

The /32 nature of node locators and RbR pointers have additional two
benefits:

- it can reuse years of operation experience and existing IPv4 addressing
schema in any network as is.

- it can reuse 32 bit RIB and FIBs and other data plane processing hardware
on any network element making support of RbR really easy even by legacy
hardware.

*IETF aspect:*

The RbR can be deployed today with full compliance of RFC8200 + use of
RFC2473.

Additional protocol enhancements for signalling actions can be standardized
independently if needed.

- - -

Apologies for long note. I just thought to share this idea with the WG in
the light of set of network requirements vs discussion of documents under
adoption call (namely CRH).

Kind regards,
Robert Raszuk

---

From: [email-masked]
Date: Sun, 24 May 2020 08:50:55 -0700

Robert,

I’ll point out that you could avoid messing with the address space at all by using the IPv6 flow label or a single MPLS label.

Tony

> On May 24, 2020, at 5:58 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Dear 6man WG,
> 
> Number of concurrent discussions in this WG triggered an idea which I would like to share with you all to think about during the long weekend. 
> 
> For some unknown reasons an army of folks has been cornered to think that stuffing each packet with encoded or not, compressed or not hop by hop list of nodes to traverse is best we can do to provide some form of path engineering in some cases combined with network programming. 
> 
> Some put on such lists IPv6 addresses, some try to just put pieces of those addresses and some even try to squeeze mpls labels there under a different cover (brrrr). 
> 
> Few months back I wrote IP TE + NP proposal however the summary below is a simplified version of it. Just for reference original document is here: https://tools.ietf.org/html/draft-raszuk-teas-ip-te-np-00 <https://tools.ietf.org/html/draft-raszuk-teas-ip-te-np-00>
> 
> Description:
> 
> Let's assume that in my network I would like to be able to construct 1000 engineered paths (different from what IGP would compute as SPF with its default algorithm). Perhaps say that number of those paths would be 10x or 100x more. That means that essentially the paths itself could be expressed in 16 bits. 
> 
> Let's further observe that what is needed to accomplish any form of traffic engineering are two elements: 
> 
> Note that while I am using TE as an example additional network programming functions including for example SFC processing is supported with no change as well. 
> 
> Proposal: 
> Let's allocate a /64 prefix which will be used in entire domain for the purpose of RbR
> Let's mark each node in a domain with different /32 address (it is naturally ok to simply use current /32 IPv4 node addresses)
> Let's use remaining 32 bits of the IPv6 address to carry an RbR pointer
> To illustrate format of the address of the packet after it is encapsulated on ingress into the domain looks like: 
> 
> |<----  /64 PREFIX ----->|<--- /32 LOCATOR --->|<--- /32 RbR POINTER --->|
> 
> No need for any routing header at all ! No per packet overhead regardless of the number of hops to traverse (except 40 bytes of IPv6 encapsulation). 
> 
> Pointer (or reference) - /32 bits string which would uniquely in entire domain identify the aggregates of flows. Indicates a reference to a set of local actions for those packets which carrying such reference. Only those routers which DA is in the /96 address need to inspect and execute the pointer.
> 
> Pointers can be structured or unstructured - depending on operators's choice and network requirements. In structured case for example first 16 bits can represent 2^16 number of unique TE paths in the network while remaining 16 bits local functions in each network elements (segment endpoint). Such bit boundary is a local operators choice. 
> 
> RbR pointers are assigned by the same entity (human or controller) which normally is responsible for path engineering and network programming in the domain. 
> 
> However actions indicated by said RbR pointers are locally significant to any network element. 
> 
> That is accomplished by propagating actions which correspond to such references as tuple: LOCATOR + POINTER. Depending on the selected means of propagation of such actions it also can be: PREFIX+LOCATOR + POINTER. 
> 
> Example: 
> 
> R1 
> 
> Actions distribution:
> There are number of ways to distribute such instructions domain wide. One network can choose to use configuration push via ansible automation. The other may like to use existing BGP FlowSpec SAFI. Yet some other would like to use OpenR message bus on a pub-sub basis. 
> 
> Perhaps a yet one more set of engineers would like to get new BGP or IGP extension defined. 
> 
> Scaling: 
> As we all know there is no free lunch. Here I believe proposed solution is a sweet spot compromise between soft state path setup in network elements (MPLS RSVP-TE) and loading each packet with additional routing header information. 
> 
> Most importantly it needs to be observed that each participating midpoint *only* needs to store actions which correspond to his own node. That applies to both control plane(*) and especially local data plane RbR-FIB. If action is send by any means towards a network element and does not contain its LOCATOR in the first /32 bits it will be dropped either at inbound processing or never propagated from control plane to RIB/FIB.
> 
> It has been studied that in real production networks to satisfy the requirements of most path steering needs at most 2-3 sometimes 4 segments may be needed. That effectively means that for such flows corresponding actions need to be kept only in 2-4 routers in the entire domain. Rest of the network elements may never need to be RbR aware as all they do is forward based on /96 IPv6 prefix. 
> 
> * For control plane it depends on the form of the distribution - if BGP is used and given node is a route reflector all RbR action "routes" or "rules" need to be stored there. 
> 
> After all routers are build to carry state. IP prefixes are nothing by state - more over - IP prefixes do change all the time (continues BGP churn). So while thinking about scaling and state one needs to also keep in mind the static nature of actions. Perhaps some TE paths will be pinned for weeks or months and will never require a change of either control plane nor RbR FIB. 
> 
> Last RbR cost (packet tax) is constant and equals zero. It does not increase if I need to engineer path at each of my 10 hops for specific adjacency on each hop.  
> 
> Summarization: 
> 
> It is my strong believe that in any network any scheme must allow for reachability summarization. 
> 
> RbR locators can be easily summarized/aggregated across ASNs (under the same administrative control) or across IGP areas/levels. 
> 
> Security: 
> Most networks already have inbound ACLs (setup manually or dynamically) to protect any external packet to enter their domain with destination address equal to infrastructure space. 
> 
> Here our /64 prefix can be either already part of this existing rule or a single ACL line dropping all packets with DA = PREFIX or longer applied on all external interfaces. 
> 
> ACLs supporting basic IPv6 filtering is all what is required and that is commonly supported by all IPv6 capable routers today. 
> 
> RbR-FIB:
> 
> The /32 nature of node locators and RbR pointers have additional two benefits: 
> 
> - it can reuse years of operation experience and existing IPv4 addressing schema in any network as is. 
> 
> - it can reuse 32 bit RIB and FIBs and other data plane processing hardware on any network element making support of RbR really easy even by legacy hardware. 
> 
> IETF aspect:
> 
> The RbR can be deployed today with full compliance of RFC8200 + use of RFC2473. 
> 
> Additional protocol enhancements for signalling actions can be standardized independently if needed. 
> 
> - - - 
> 
> Apologies for long note. I just thought to share this idea with the WG in the light of set of network requirements vs discussion of documents under adoption call (namely CRH). 
> 
> Kind regards,
> Robert Raszuk
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 18:44:48 +0200

Hi Tony,

If I would insert a reference into flow label space I am afraid I would
lose the current flow label functionality which perhaps in some cases may
not be desired.

To your second hint - by all means yes - anyone could use the MPLS label in
an MPLS network to indicate a context. Pointer can be a context. Some may
argue that 20 bits may not be enough thought.

However the exercise here is to provide a form of path steering for IPv6
networks where transport MPLS is not enabled by design.

Last I am not sure what "messing with address space" comment really meant
to indicate ...

- a fear that allocation of /64 prefix for a domain is hard
- a concern that using /32 locators is problematic
- an observation that placing /32 bit pointer in least significant bits of
the packets (within an already encapsulated IPv6 header) is the wrong thing
to do ? If so let me observe that most SR approaches today do rewrite the
entire DA address at each segment endpoint.

Many thx,
Robert

On Sun, May 24, 2020 at 5:50 PM <[email-masked]> wrote:

>
> Robert,
>
> I’ll point out that you could avoid messing with the address space at all
> by using the IPv6 flow label or a single MPLS label.
>
> Tony
>
>
> Number of concurrent discussions in this WG triggered an idea which I
> would like to share with you all to think about during the long weekend.
>
> Some put on such lists IPv6 addresses, some try to just put pieces of
> those addresses and some even try to squeeze mpls labels there under a
> different cover (brrrr).
>
> - it can reuse 32 bit RIB and FIBs and other data plane processing
> hardware on any network element making support of RbR really easy even by
> legacy hardware.
>
> *IETF aspect:*
>

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 18:44:48 +0200

Hi Tony,

Many thx,
Robert

---

From: [email-masked]
Date: Sun, 24 May 2020 11:10:35 -0700

Hi Robert,

> If I would insert a reference into flow label space I am afraid I would lose the current flow label functionality which perhaps in some cases may not be desired. 

Is someone actually using the flow label?  As far as I know it’s wholly unused.

> To your second hint - by all means yes - anyone could use the MPLS label in an MPLS network to indicate a context. Pointer can be a context. Some may argue that 20 bits may not be enough thought.

In which case you use a label stack and create a hierarchy of paths: a path traverses the source area, pushes into a trunk to transit the backbone, and then drops back to a single label to traverse the destination area.

> However the exercise here is to provide a form of path steering for IPv6 networks where transport MPLS is not enabled by design. 

Is that a problem that needs to be solved?

> Last I am not sure what "messing with address space" comment really meant to indicate ... 

Changing the semantics of any part of the address space may require more semantic and pragmatic changes than would be necessary via other approaches.

Tony

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 20:40:09 +0200

Hi,

> Is someone actually using the flow label?  As far as I know it’s wholly

I really don't know. Perhaps folks on the cc-ed lists could comment if
anyone is using it for anything.

RFC2473 does not give any guidance. In fact its section 6.5 recommends to
just stuff it with value of zero.

But if we can use it within encapsulated header this is great news. All
what I stated in my original note equally applies to the case where a
pointer travels in Flow Label space. And IMHO 20 bits is sufficient.

> Is that a problem that needs to be solved?

Well we have SRv6. We are now discussing adoption of new Routing Header
(CRH) as described in draft-bonica-6man-comp-rtg-hdr-22

If there is no problem to be solved I think we would not be having 100s if
not 1000s of messages about it and 10s of WG documents as well as few RFCs
:)

> Changing the semantics of any part of the address space may require more
> semantic and pragmatic changes than would be necessary via other approaches.

Very true. The placement of the pointer referencing action within the last
bits of the DA address was just an example.

The core of the main idea is that instead of listing a stack of transit
nodes to traverse in each packet, let's outsource some of it to the control
plane.

Where to carry the pointer in plain IPv6 seems like an open topic. We could
carry it in:

- 20 bit flow label
- last 32 bit of the DA
- new RH (RRH - Reference Routing Header ) of fixed size and immutable
entire content

Many thx for comments !!!
R.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 20:40:09 +0200

Hi,

- 20 bit flow label
- last 32 bit of the DA

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 25 May 2020 08:36:51 +1200

Hi Tony,

> Is someone actually using the flow label?  As far as I know it’s wholly unused.

It's set according to RFC 6437 by all modern Linux and Windows hosts for both TCP and UDP traffic. To what extent it's being used for load balancing is hard to tell, but
it isn't free for traffic engineering.

My own reaction to Robert's proposal is that it did indeed sound like a description of a way to use MPLS labels. 

<snip>
 
> Changing the semantics of any part of the address space may require more semantic and pragmatic changes than would be necessary via other approaches.

Overloading part of the interface identifier with local semantics is an obvious temptation, and really no more shocking than RFC8273. I am much more bothered by proposals to encode semantics in high order address bits.

But anyway... routing area topic? It doesn't seem to need any 6MAN action whatever.

   Brian

---

From: Tom Herbert <[email-masked]>
Date: Sun, 24 May 2020 13:50:14 -0700

> Is someone actually using the flow label?  As far as I know it’s wholly unused.
Tony,

Yes, flow labels are used. Windows 10 set non-zero flow labels, Linux
started setting them in 4.3, MacOS was setting them before that. Fow
label is used as input to ECMP, NIC flow steering, and in some flows
matching rules. Attempts to repurpose the flow label bits will likely
break things at this point (the idea comes up a lot :-) ).

Tom

---

From: Tom Herbert <[email-masked]>
Date: Sun, 24 May 2020 13:50:14 -0700

Tony,

Tom

---

From: Ole Troan <[email-masked]>
Date: Sun, 24 May 2020 22:50:53 +0200

> On 24 May 2020, at 20:10, [email-masked] wrote:
> 

Inside a limited domain using encapsulation, which is what all these solutions cover, you could likely use the flow label as a “path tag”. 
If there is much benefit in using an IPv6 header for the encapsulation is another matter. 

Ole

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 22:52:15 +0200

Hi Brian,

> It doesn't seem to need any 6MAN action whatever.

True - unless we collectively would decide to carry the reference in the
new RH.

Otherwise it is not even Routing Area topic either :)

See IETF is great to standardize new protocol extensions and add
complexity.

But when a lot of new functionality could be enabled with
existing protocols if vendors would agree to support it my personal
observation is that it has no good process to follow.

Best,
R.

PS. Encoding a reference into MPLS label could be an option if given domain
already uses MPLS. That comes with cost and complexity (running LDP between
all network elements or SR-MPLS). For now let's assume domain is not
enabled for using MPLS transport.

On Sun, May 24, 2020 at 10:36 PM Brian E Carpenter <
[email-masked]> wrote:

> It's set according to RFC 6437 by all modern Linux and Windows hosts for
> both TCP and UDP traffic. To what extent it's being used for load balancing
> is hard to tell, but
> Overloading part of the interface identifier with local semantics is an
> obvious temptation, and really no more shocking than RFC8273. I am much
> more bothered by proposals to encode semantics in high order address bits.
>

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 22:52:15 +0200

Hi Brian,

Best,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 22:56:34 +0200

Right Ole - this is exactly the scope here.

The original flow label will remain untouched in the inner header anyway
even if this is of some use outside of limited domain.

However as we discussed this is just a possible placeholder option. At this
very moment considering the full picture I am more inclined to suggest use
of least significant bits of the DA.

But completely open for suggestions and guidance

Best,
R.

On Sun, May 24, 2020 at 10:50 PM Ole Troan <[email-masked]> wrote:

> Inside a limited domain using encapsulation, which is what all these
> solutions cover, you could likely use the flow label as a “path tag”.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 22:56:34 +0200

Best,
R.

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 24 May 2020 22:03:47 +0100

> solutions cover, you could likely use the flow label as a “path

this is not a safe assumption.  The target deployment scenarios appear 
to be own-tenancy networks with an emphasis on large-scale datacentre 
implementations.  In this sort of situation it would be normal for the 
network fabric to use multiple links between routing nodes, thereby 
creating a requirement for load balancing.  Removing one of the entropy 
sources for bucket management would be unwise in these circumstances, 
particularly given the likelihood of elephant flows.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 23:06:23 +0200

Hi Nick,

FYI
https://blog.apnic.net/2018/01/11/ipv6-flow-label-misuse-hashing/

Thx,
R.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 25 May 2020 09:10:43 +1200

> Right Ole - this is exactly the scope here. 

That's what https://tools.ietf.org/html/rfc6437#section-4 calls "Flow State Establishment". But there are still some constraints on the value of the label.

Unless of course the IETF embraces limited-domain architectures and standards...

    Brian

> 
> The original flow label will remain untouched in the inner header anyway even if this is of some use outside of limited domain. 
> 
> However as we discussed this is just a possible placeholder option. At this very moment considering the full picture I am more inclined to suggest use of least significant bits of the DA. 
> 
> On Sun, May 24, 2020 at 10:50 PM Ole Troan <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
> 
>     > On 24 May 2020, at 20:10, [email-masked] <mailto:[email-masked]> wrote:
>     >

---

From: Tom Herbert <[email-masked]>
Date: Sun, 24 May 2020 14:10:28 -0700

Robert,

Have you looked a identifier/locator split for IPv6-- ILA or ILNP?

Tom

On Sun, May 24, 2020 at 5:59 AM Robert Raszuk <[email-masked]> wrote:
> Few months back I wrote IP TE + NP proposal however the summary below is a simplified version of it. Just for reference original document is here: https://tools.ietf.org/html/draft-raszuk-teas-ip-te-np-00
>
> Description:
>

---

From: Tom Herbert <[email-masked]>
Date: Sun, 24 May 2020 14:10:28 -0700

Robert,

Tom

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 24 May 2020 22:12:41 +0100

> https://blog.apnic.net/2018/01/11/ipv6-flow-label-misuse-hashing/

Not sure how this is relevant?

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 23:13:40 +0200

Oh yes ... I was part of this RRG. I like ILNP and LISP ... but here RbR
properties and use case is a bit different.

On Sun, May 24, 2020 at 11:10 PM Tom Herbert <[email-masked]> wrote:

> > For some unknown reasons an army of folks has been cornered to think
> that stuffing each packet with encoded or not, compressed or not hop by hop
> list of nodes to traverse is best we can do to provide some form of path
> > Few months back I wrote IP TE + NP proposal however the summary below is
> a simplified version of it. Just for reference original document is here:
> > Let's assume that in my network I would like to be able to construct
> 1000 engineered paths (different from what IGP would compute as SPF with
> its default algorithm). Perhaps say that number of those paths would be 10x
> or 100x more. That means that essentially the paths itself could be
> expressed in 16 bits.
> >
> > Note that while I am using TE as an example additional network
> programming functions including for example SFC processing is supported
> with no change as well.
> >
> > Proposal:
> >
> > Let's mark each node in a domain with different /32 address (it is
> > To illustrate format of the address of the packet after it is
> encapsulated on ingress into the domain looks like:
> >
> > No need for any routing header at all ! No per packet overhead
> regardless of the number of hops to traverse (except 40 bytes of IPv6
> encapsulation).
> >
> > Pointer (or reference) - /32 bits string which would uniquely in entire
> > Pointers can be structured or unstructured - depending on operators's
> choice and network requirements. In structured case for example first 16
> bits can represent 2^16 number of unique TE paths in the network while
> remaining 16 bits local functions in each network elements (segment
> > There are number of ways to distribute such instructions domain wide.
> One network can choose to use configuration push via ansible automation.
> The other may like to use existing BGP FlowSpec SAFI. Yet some other would
> like to use OpenR message bus on a pub-sub basis.
> >
> > As we all know there is no free lunch. Here I believe proposed solution
> is a sweet spot compromise between soft state path setup in network
> elements (MPLS RSVP-TE) and loading each packet with additional routing
> header information.
> >
> > Most importantly it needs to be observed that each participating
> midpoint *only* needs to store actions which correspond to his own node.
> That applies to both control plane(*) and especially local data plane
> RbR-FIB. If action is send by any means towards a network element and does
> not contain its LOCATOR in the first /32 bits it will be dropped either at
> > * For control plane it depends on the form of the distribution - if BGP
> is used and given node is a route reflector all RbR action "routes" or
> "rules" need to be stored there.
> >
> > Most networks already have inbound ACLs (setup manually or dynamically)
> to protect any external packet to enter their domain with destination
> address equal to infrastructure space.
> >
> > Here our /64 prefix can be either already part of this existing rule or
> a single ACL line dropping all packets with DA = PREFIX or longer applied
> on all external interfaces.
> >
> > Apologies for long note. I just thought to share this idea with the WG
> in the light of set of network requirements vs discussion of documents
> under adoption call (namely CRH).
> >
> > Kind regards,
> > Robert Raszuk

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 24 May 2020 23:13:40 +0200

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 25 May 2020 09:45:56 +1200

> https://blog.apnic.net/2018/01/11/ipv6-flow-label-misuse-hashing/  

The IETF is not the protocol police. If vendors ignore published standards, it really isn't something we can fix.

   Brian

> 
> Thx,
> R. 
> 
> On Sun, May 24, 2020 at 11:04 PM Nick Hilliard <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 24 May 2020 23:12:25 +0100

Brian E Carpenter wrote on 24/05/2020 22:45:
> The IETF is not the protocol police. If vendors ignore published 
> standards, it really isn't something we can fix.

Sorry, I should have clarified this in my last email.

Joel's post isn't relevant to this discussion.  What he's talking about 
is where you're attempting to load balance traffic flows to multiple 
servers, but where you're depending on the flow hash calculation to 
ensure that related flows end up being handed off to the same machine. 
You might want to do this with e.g. high capacity dns services, or DDOS 
traffic by splitting it up at the last hop to make it easier to handle.

Some vendors change the flow label mid-flow, which is a bit broken - not 
very broken, but a bit.

However, if you're using flow-based load-balanced services like what 
Joel is talking about, and if you honour the ipv6 flow label, you could 
end up handing the flow over to the wrong server which will cause state 
problems.  Of course, vendors shouldn't change their flow labels 
mid-flow, but mostly Joel's article shows that flow-based service 
hand-off is inherently fragile and can break with seemingly 
insignificant changes.

What I was referring to was standard flow-based load balancing on 
network links.  It should be obvious that this is not related to what 
Joel was talking about, as network links are not stateful. To reiterate, 
the ipv6 flow label would be highly relevant to own-tenancy DC scenarios.

Nick

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 25 May 2020 13:31:02 +1200

> ensure that related flows end up being handed off to the same machine. 

Er, yes. There's an RFC about that: https://tools.ietf.org/html/rfc7098

And indeed, it's broken if people upstream don't conform to RFC 6437:

"  A forwarding node
   MUST either leave a non-zero flow label value unchanged or change it
   only for compelling operational security reasons as described in
   Section 6.1."

(Section 6.1 specifies how to rewrite the flow label if you really
insist.)

It's a bit surprising that Joel's post doesn't refer to RFC 7098.

    Brian