Subject: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

From: Andrew Alston <[email-masked]>
Date: Thu, 21 Oct 2021 17:08:00 +0000

Hi V6Ops,

I thought I would raise the issues that follow here – and perhaps we can look and if these issues are real – and if so – what the solutions are.

During a close review of the compression proposals for SRv6 (CRH/G-Srv6/USID etc etc) I noticed some potential very real vulnerabilities in SRv6 itself – which by extensions would affect srv6, srv6 network programming and all the compression flavors – including CRH on which I am a co-author.  Having raised these issues with my CRH co-authors it was agreed that I raise these issues here so we could discuss them.

So a little background – SRrv6 is typically considered as something that should run in the confines of a  “limited-domain” – the question I started with is – can we actually define and maintain the boundaries of a limited-domain – and if so – how.  The conclusion I came to was that the concept of limited-domain in regards to SRv6 was an extremely fuzzy thing.

Now to understand this – we need first to look at the typical methods of protecting things

  *   In the case of MPLS – we have what I will loosely refer to as a “fail-closed” scenario – that is to say – unless MPLS is enabled explicitly on an interface – ingress packets will not be processed – this works because of a separate ether-type
  *   In the case of SR-MPLS – an identical situation occurs
  *   In the case of dodgy IGP traffic – again, we have a fail-closed scenario
  *   In the case of standard IPv6 – we have the option of utilizing BCP38 and ingress filtering – and the same thing in regards to IPv4.

Now – lets examine SRv6 for a second.

In a scenario of Host A (A linux box) with address 2001:db8:1:1::10/64   utilizing a gateway of 2001:db8:1:1::fefe – packets from 2001:db8:1:1::10 flowing towards that gateway would pass ingress filter checks based on BCP38 since the source address was legitimate, unless we explicitly filtered out where that packet could be destined for based on its DA (More on this in a bit)

Now, lets imagine for a second that Host A gets compromised – and an attack encapsulates a packet that has an entirely different source address – and whatever random DA address inside an SRH.  That SRH has a SID list in it – be it via a direct SID or via compression mechanisms, and a DA towards the SID itself.  The packet then routes – passing the ingress checks – and landing up at the router with the first SID.  Since this was the only SID in the list – the packet outer SRH is removed – and the inner packet is forwarded to the FIB – and you just bypassed BCP38.

In a similar mechanism, if the SRH states that the next protocol header is an IPv4 packet – when the de-encapsulation happens at last SID – the payload (the inner v4 packet) will then be forwarded via the FIB – and off you go.

Now, normally to protect against this as stated – an access list would be placed on the networks borders to prevent encapsulated packets and packets containing SID destinations from entering the network.  However – if we consider the above scenario where the attack is coming from a compromised server inside the traditional borders – we have a problem.  The application of access lists to every port containing a server is also potentially unrealistic and unmaintainable (not to mention could potentially on certain devices overload the TCAMs)

We also have an even more potentially deadly issue – and this is entirely theoretical at this point since I haven’t had time to really investigate and test it with real code.  Let us presume in the above that the same host A is compromised.  It encapsulates a packet with an SRH – the internal packet is an Ipv4 packet – and its destined at a broadcast address of an IP subnet that is bound to the same network as the de-encapsulating router.  The source of the internal v4 packet is spoofed – we’ve now managed to – through a use of the tunneling mechanism, effectively created a version of an old tool called smurf – in a manner that is going to be pretty hard to trace.

Another potential scenario – would be for an attacker in host A to jit some ebpf code – that matches – modifies and re-encapsulates incoming return packets and retransmits then. Each time applying the same SRH.  The inner packet could be pretty much anything – so long as the internal DA is set back to host A that is sending the packet.  The packet would then flow out as an SRH encapsulated packet, the outer header would get popped, the inner packet would flow back towards the original compromised host, which would match it modify it re-encapsulate it and start the loop again.  Doing this with ebpf and a kernel jit – would be pretty difficult to spot if you didn’t know what you are doing because you wouldn’t potentially see any obvious userland code.

As a final thought – consider a hypervisor based system – that has multiple VM’s on it – and the filtering implications of all of the above – and the filtering becomes even more difficult to maintain and more complex.

Again – the filtering per every port that may contains a server or a desktop – that may not be realistic – especially when we could be running multiple ports that are handing out source addresses via RA – so – how do we solve this – or is this a major flaw in srv6 itself – that needs some other solution (give srv6 its own protocol code and acknowledge that it isn’t ipv6 at all, allowing for a “fail-closed” scenario maybe?)

As an operator that runs extensive IPv6 – I’d really like to hear thoughts and comments and potentially we can find a way to address these issues.

Thanks

Andrew

---

From: Vasilenko Eduard <[email-masked]>
Date: Thu, 21 Oct 2021 17:23:09 +0000

All router’s ports should have RH type 4 (SRH) filtered out by default.
Network Admin should activate RH#4 support only for NNI ports.
Problem solved.
Ed/
From: v6ops [mailto:[email-masked]] On Behalf Of Andrew Alston
Sent: Thursday, October 21, 2021 8:08 PM
To: [email-masked]
Subject: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Hi V6Ops,

-          In the case of MPLS – we have what I will loosely refer to as a “fail-closed” scenario – that is to say – unless MPLS is enabled explicitly on an interface – ingress packets will not be processed – this works because of a separate ether-type
-          In the case of SR-MPLS – an identical situation occurs
-          In the case of dodgy IGP traffic – again, we have a fail-closed scenario
-          In the case of standard IPv6 – we have the option of utilizing BCP38 and ingress filtering – and the same thing in regards to IPv4.

Thanks

Andrew

---

From: Ron Bonica <[email-masked]>
Date: Thu, 21 Oct 2021 19:06:19 +0000

Ed,

Some SRv6 packets do not contain an SRH. The following is an example:

  *   The packet traverses only a few segments
  *   All of these segments are encoded in a single C-SID container
  *   All segments exhibit the NEXT-C-SID behavior.

However, I agree with you on one point. If every SRv6 packet contained a routing header of some sort, this problem wouldn't be so bad.

                                                                                                                         Ron

Juniper Business Use Only
From: v6ops <[email-masked]> On Behalf Of Vasilenko Eduard
Sent: Thursday, October 21, 2021 1:23 PM
To: Andrew Alston <Andrew.Alston=[email-masked]>; [email-masked]
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

[External Email. Be cautious of content]

All router's ports should have RH type 4 (SRH) filtered out by default.
Network Admin should activate RH#4 support only for NNI ports.
Problem solved.
Ed/
From: v6ops [mailto:[email-masked]] On Behalf Of Andrew Alston
Sent: Thursday, October 21, 2021 8:08 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Hi V6Ops,

I thought I would raise the issues that follow here - and perhaps we can look and if these issues are real - and if so - what the solutions are.

During a close review of the compression proposals for SRv6 (CRH/G-Srv6/USID etc etc) I noticed some potential very real vulnerabilities in SRv6 itself - which by extensions would affect srv6, srv6 network programming and all the compression flavors - including CRH on which I am a co-author.  Having raised these issues with my CRH co-authors it was agreed that I raise these issues here so we could discuss them.

So a little background - SRrv6 is typically considered as something that should run in the confines of a  "limited-domain" - the question I started with is - can we actually define and maintain the boundaries of a limited-domain - and if so - how.  The conclusion I came to was that the concept of limited-domain in regards to SRv6 was an extremely fuzzy thing.

Now to understand this - we need first to look at the typical methods of protecting things

  *   In the case of MPLS - we have what I will loosely refer to as a "fail-closed" scenario - that is to say - unless MPLS is enabled explicitly on an interface - ingress packets will not be processed - this works because of a separate ether-type
  *   In the case of SR-MPLS - an identical situation occurs
  *   In the case of dodgy IGP traffic - again, we have a fail-closed scenario
  *   In the case of standard IPv6 - we have the option of utilizing BCP38 and ingress filtering - and the same thing in regards to IPv4.

Now - lets examine SRv6 for a second.

In a scenario of Host A (A linux box) with address 2001:db8:1:1::10/64   utilizing a gateway of 2001:db8:1:1::fefe - packets from 2001:db8:1:1::10 flowing towards that gateway would pass ingress filter checks based on BCP38 since the source address was legitimate, unless we explicitly filtered out where that packet could be destined for based on its DA (More on this in a bit)

Now, lets imagine for a second that Host A gets compromised - and an attack encapsulates a packet that has an entirely different source address - and whatever random DA address inside an SRH.  That SRH has a SID list in it - be it via a direct SID or via compression mechanisms, and a DA towards the SID itself.  The packet then routes - passing the ingress checks - and landing up at the router with the first SID.  Since this was the only SID in the list - the packet outer SRH is removed - and the inner packet is forwarded to the FIB - and you just bypassed BCP38.

In a similar mechanism, if the SRH states that the next protocol header is an IPv4 packet - when the de-encapsulation happens at last SID - the payload (the inner v4 packet) will then be forwarded via the FIB - and off you go.

Now, normally to protect against this as stated - an access list would be placed on the networks borders to prevent encapsulated packets and packets containing SID destinations from entering the network.  However - if we consider the above scenario where the attack is coming from a compromised server inside the traditional borders - we have a problem.  The application of access lists to every port containing a server is also potentially unrealistic and unmaintainable (not to mention could potentially on certain devices overload the TCAMs)

We also have an even more potentially deadly issue - and this is entirely theoretical at this point since I haven't had time to really investigate and test it with real code.  Let us presume in the above that the same host A is compromised.  It encapsulates a packet with an SRH - the internal packet is an Ipv4 packet - and its destined at a broadcast address of an IP subnet that is bound to the same network as the de-encapsulating router.  The source of the internal v4 packet is spoofed - we've now managed to - through a use of the tunneling mechanism, effectively created a version of an old tool called smurf - in a manner that is going to be pretty hard to trace.

Another potential scenario - would be for an attacker in host A to jit some ebpf code - that matches - modifies and re-encapsulates incoming return packets and retransmits then. Each time applying the same SRH.  The inner packet could be pretty much anything - so long as the internal DA is set back to host A that is sending the packet.  The packet would then flow out as an SRH encapsulated packet, the outer header would get popped, the inner packet would flow back towards the original compromised host, which would match it modify it re-encapsulate it and start the loop again.  Doing this with ebpf and a kernel jit - would be pretty difficult to spot if you didn't know what you are doing because you wouldn't potentially see any obvious userland code.

As a final thought - consider a hypervisor based system - that has multiple VM's on it - and the filtering implications of all of the above - and the filtering becomes even more difficult to maintain and more complex.

Again - the filtering per every port that may contains a server or a desktop - that may not be realistic - especially when we could be running multiple ports that are handing out source addresses via RA - so - how do we solve this - or is this a major flaw in srv6 itself - that needs some other solution (give srv6 its own protocol code and acknowledge that it isn't ipv6 at all, allowing for a "fail-closed" scenario maybe?)

As an operator that runs extensive IPv6 - I'd really like to hear thoughts and comments and potentially we can find a way to address these issues.

Thanks

Andrew

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 22 Oct 2021 08:02:08 +0200

Le 21/10/2021 à 19:08, Andrew Alston a écrit :
> Hi V6Ops,
> 
> I thought I would raise the issues that follow here – and perhaps we can 
> look and if these issues are real – and if so – what the solutions are.
> 
> During a close review of the compression proposals for SRv6 
> (CRH/G-Srv6/USID etc etc) I noticed some potential very real 
> vulnerabilities in SRv6 itself – which by extensions would affect srv6, 
> srv6 network programming and all the compression flavors – including CRH 
> on which I am a co-author.  Having raised these issues with my CRH 
> co-authors it was agreed that I raise these issues here so we could 
> discuss them.
> 
> So a little background – SRrv6 is typically considered as something that 
> should run in the confines of a  “limited-domain” – the question I 
> started with is – can we actually define and maintain the boundaries of 
> a limited-domain – and if so – how.  The conclusion I came to was that 
> the concept of limited-domain in regards to SRv6 was an extremely fuzzy 
> thing.
> 
>   * In the case of MPLS – we have what I will loosely refer to as a
>     “fail-closed” scenario – that is to say – unless MPLS is enabled
>     explicitly on an interface – ingress packets will not be processed –
>     this works because of a separate ether-type
>   * In the case of standard IPv6 – we have the option of utilizing BCP38
>     and ingress filtering – and the same thing in regards to IPv4.
> 
> In a scenario of Host A (A linux box) with address 2001:db8:1:1::10/64 
>    utilizing a gateway of 2001:db8:1:1::fefe – packets from 
> 2001:db8:1:1::10 flowing towards that gateway would pass ingress filter 
> checks based on BCP38 since the source address was legitimate, unless we 
> explicitly filtered out where that packet could be destined for based on 
> its DA (More on this in a bit)

Is one of the points made the fact that BCP38 would not work with 
RFC8754?  Maybe an option is to update BCP38 or write another one titled 
"multi-version and multi-source IP ingress filtering", or simply an 
"IPv6 ingress filtering".  Or maybe an IPsec-enabled SRH.

Just some thoughts.

Alex

> 
> Now, lets imagine for a second that Host A gets compromised – and an 
> attack encapsulates a packet that has an entirely different source 
> address – and whatever random DA address inside an SRH.  That SRH has a 
> SID list in it – be it via a direct SID or via compression mechanisms, 
> and a DA towards the SID itself.  The packet then routes – passing the 
> ingress checks – and landing up at the router with the first SID.  Since 
> this was the only SID in the list – the packet outer SRH is removed – 
> and the inner packet is forwarded to the FIB – and you just bypassed BCP38.
> 
> In a similar mechanism, if the SRH states that the next protocol header 
> is an IPv4 packet – when the de-encapsulation happens at last SID – the 
> payload (the inner v4 packet) will then be forwarded via the FIB – and 
> off you go.
> 
> Now, normally to protect against this as stated – an access list would 
> be placed on the networks borders to prevent encapsulated packets and 
> packets containing SID destinations from entering the network.  However 
> – if we consider the above scenario where the attack is coming from a 
> compromised server inside the traditional borders – we have a problem.  
> The application of access lists to every port containing a server is 
> also potentially unrealistic and unmaintainable (not to mention could 
> potentially on certain devices overload the TCAMs)
> 
> We also have an even more potentially deadly issue – and this is 
> entirely theoretical at this point since I haven’t had time to really 
> investigate and test it with real code.  Let us presume in the above 
> that the same host A is compromised.  It encapsulates a packet with an 
> SRH – the internal packet is an Ipv4 packet – and its destined at a 
> broadcast address of an IP subnet that is bound to the same network as 
> the de-encapsulating router.  The source of the internal v4 packet is 
> spoofed – we’ve now managed to – through a use of the tunneling 
> mechanism, effectively created a version of an old tool called smurf – 
> in a manner that is going to be pretty hard to trace.
> 
> Another potential scenario – would be for an attacker in host A to jit 
> some ebpf code – that matches – modifies and re-encapsulates incoming 
> return packets and retransmits then. Each time applying the same SRH.  
> The inner packet could be pretty much anything – so long as the internal 
> DA is set back to host A that is sending the packet.  The packet would 
> then flow out as an SRH encapsulated packet, the outer header would get 
> popped, the inner packet would flow back towards the original 
> compromised host, which would match it modify it re-encapsulate it and 
> start the loop again.  Doing this with ebpf and a kernel jit – would be 
> pretty difficult to spot if you didn’t know what you are doing because 
> you wouldn’t potentially see any obvious userland code.
> 
> As a final thought – consider a hypervisor based system – that has 
> multiple VM’s on it – and the filtering implications of all of the above 
> – and the filtering becomes even more difficult to maintain and more 
> complex.
> 
> Again – the filtering per every port that may contains a server or a 
> desktop – that may not be realistic – especially when we could be 
> running multiple ports that are handing out source addresses via RA – so 
> – how do we solve this – or is this a major flaw in srv6 itself – that 
> needs some other solution (give srv6 its own protocol code and 
> acknowledge that it isn’t ipv6 at all, allowing for a “fail-closed” 
> scenario maybe?)
> 
> As an operator that runs extensive IPv6 – I’d really like to hear 
> thoughts and comments and potentially we can find a way to address these 
> issues.
> 
> Thanks
> 
> Andrew
> 
> 
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Andrew Alston <[email-masked]>
Date: Fri, 22 Oct 2021 06:53:59 +0000

> "IPv6 ingress filtering". Or maybe an IPsec-enabled SRH.

The issue is not really BCP38 itself - BCP38 is fine - the issue here is dynamic de-encapsulation.

Effectively - anything that can cause an arbitrary transit node to de-encapsulate and forward.  Once you can get a packet encapsulated, sent onto the network, and de-encapsulated by a random router, which then takes that internal packet and forwards it - unless you can read deeply enough into the packet to understand the internal packet and filter it, and have some seriously magic sauce that allows every transit node to know what to do with every internal packet - you have a problem.

Normally in a tunnel situation - the tunnel is de-encapsulated at specific configured points with specific behavior - in the SRv6 case - the moment it hits the first sid, the outer header is stripped and the internal - whatever it is - is forwarded via the FIB, and I know of no possible way to actively filter against that inner packet.  This is particularly true when no matter how deep you can look into the packet - another header can always be added to screw with this.

For example:

Lets consider I have a host of 2001:db8:30:30::10.

I craft an internal IPv6 packet with source 2001:db8:10:10::2 and destination address 2001:db8:20:20::2 

Now lets assume that we're using compression and my locator block is 2001:db8:30:: and I'm using a 16 bit SID of 5

I encapsulate that packet with an SRH with SA of the original host address (2001:db8:30:30::10) and DA 2001:db8:30:5:: and set it up so that’s the only sid.
The node sid 5 is at - will de-encap and forward - BCP38 fails at this point because the source passes the check - and the node where the de-encap occurs doesn't care, its gonna de-encap and forward regardless.

So - lets then take this a step further - and say I have fancy hardware that can look PAST an SRH - or a v6 header - and I still wanna be nasty.

I can craft a packet such that the outer header sends to 2001:db8:30:5::, the next header is again an SRH, and that sends to 2001:db8:30:6::, the next header is again an SRH, it sends to 2001:db8:30:5::, and then it de-encaps. 

My de-encap happens at the same place - but there is no way that I could ever look deep enough into the packet to see the *REAL* packet that’s gonna end up flowing after that de-encap - because no matter how deep you go - I can keep stacking headers.

The argument is going to come that - I can just filter for anything with an SRH - except, firstly, this assumes that every edge device I plug servers into can match and filter SRH - and I can point to plenty of ipv6 devices deployed out in the field out here that would have issues here, second, it assumes that there is sufficient tcam space to apply interface specific filters to every interface I may have a server in - this is no longer just "edge of the network" specific - because every in effect host because an "edge" for the purposes of this, and thirdly, it assumes that there is an SRH at all.

In the case of there NOT being an SRH - I have to start filtering on the locator block - that’s an LPM filter on every port/sub-interface that is not using SRv6.  There is no fail closed here as there would be with something like MPLS or anything with its own ether-type that has to be explicitly enabled.  Then, if we extend this scenario into the cloud.  You can end up in a situation where you have thousands of vlans facing a cloud stack - each of those will need a filter to protect, and there can never be a situation where one host is permitted to talk SRv6 that finds itself off an interface that contains other hosts that aren't - because you can't selectively filter in any realistic manner (Try doing SA/DA filter combinations across a few thousand interfaces with interface specific filters - see how long your tcams last)

Effectively - in very simple summary - when you make your "trusted" edge this wide - where every server on the network is a potential threat to inject packets the network is going to blindly act on - things get unmaintainable fast unless you have a default fail-closed scenario - that does not exist here.  We went through similar with normal source routing - we went through similar with RH0 - but here - we have created a way to effectively embed packets - have them flow onto the network - de-encapsulate them with zero real control - and forward them.  That, from an operator perspective, is a nightmare.

Andrew

---

From: Gert Doering <[email-masked]>
Date: Fri, 22 Oct 2021 09:01:13 +0200

Hi,

On Thu, Oct 21, 2021 at 05:08:00PM +0000, Andrew Alston wrote:
> As an operator that runs extensive IPv6 ??? I???d really like to hear thoughts and comments and potentially we can find a way to address these issues.

We've decided long ago that we see no value in SRv6.

MPLS-SR makes sense, SRv6 seems to just bring heaps of complications.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                      Vorstand: Sebastian v. Bomhard, Michael Emmer
Joseph-Dollinger-Bogen 14        Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                 HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444         USt-IdNr.: DE813185279

---

From: Andrew Alston <[email-masked]>
Date: Fri, 22 Oct 2021 07:14:45 +0000

Gert,

I don't disagree with your sentiments - however - be that as it may - if this stuff is going to exist, do we not have a responsibility to ensure that its not creating a situation that allows for wholesale security issues and the potential creation of some very nasty denial of service exploits that could affect us all? (See original email in this thread regarding smurf-v2 enabled by SRv6)

Andrew

From: v6ops <[email-masked]> On Behalf Of Gert Doering
Sent: Friday, October 22, 2021 10:01 AM
To: Andrew Alston <Andrew.Alston=[email-masked]>
Cc: [email-masked]
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Hi,

Gert Doering
-- NetMaster

_______________________________________________
v6ops mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/v6ops<https://www.ietf.org/mailman/listinfo/v6ops>

---

From: Gert Doering <[email-masked]>
Date: Fri, 22 Oct 2021 09:32:48 +0200

Hi,

On Fri, Oct 22, 2021 at 07:14:45AM +0000, Andrew Alston wrote:
> I don't disagree with your sentiments - however - be that as it
> may - if this stuff is going to exist, do we not have a responsibility
> to ensure that its not creating a situation that allows for wholesale
> security issues and the potential creation of some very nasty denial
> of service exploits that could affect us all? (See original email
> in this thread regarding smurf-v2 enabled by SRv6)

I read and fully understood what you've been saying - and I agree that
this sounds like a very real issue (without having tested it).

For the reasons you have listed I'm unsure whether there is a way 
to fix this (= it strengthens my resolve to not go there).

Trying to think up fixes

 - ACLs on ingress from all hosts that deny packets with SR EHs
 - ACLs on ingress from all hosts that deny packets *to* Routers that
   would decapsulate them (which requires that all router IPs come from
   a small number of subnets, nothing else is in there, and that you
   do not care about "ping router" for troubleshooting)
 - putting customer IPv6 into a 6PE VRF, preventing packets to be sent
   to the "real" control plane

none of these can be easily deployed in most networks, and worse, if you
actually want some hosts to participate in SR (which is, I think, the
kool-aid that has been sold to us on why SR is so great...), they all
fall apart.

Or you require RFC3514 adherence on the host, and filter on that.

(Wait, RFC3514 has never been updated to handle IPv6.  So this definitely
needs a -bis revision!)

Short summary: in the short time I spent on thinking through this, I see
no viable way to fix this class of attacks ("unrestricted egress tunnel
endpoints, reachable via an un-ACL-able protocol from malicious hosts").

Gert Doering
        -- NetMaster

---

From: Warren Kumari <[email-masked]>
Date: Fri, 22 Oct 2021 13:59:19 -0400

On Thu, Oct 21, 2021 at 1:08 PM Andrew Alston <Andrew.Alston=
[email-masked]> wrote:

> srv6 network programming and all the compression flavors – including CRH on
> which I am a co-author.  Having raised these issues with my CRH co-authors
> it was agreed that I raise these issues here so we could discuss them.
>
>
>
> should run in the confines of a  “limited-domain” – the question I started
> with is – can we actually define and maintain the boundaries of a
> limited-domain – and if so – how.  The conclusion I came to was that the
> concept of limited-domain in regards to SRv6 was an extremely fuzzy thing.
>
>
>
>    - In the case of MPLS – we have what I will loosely refer to as a
>    “fail-closed” scenario – that is to say – unless MPLS is enabled explicitly
>    on an interface – ingress packets will not be processed – this works
>    because of a separate ether-type
>    - In the case of standard IPv6 – we have the option of utilizing BCP38
> <no hats>
Yes.

"RFC 8799 - Limited Domains and Internet Protocols" says:
"Domain boundaries that are defined administratively (e.g., by address
filtering rules in routers) are prone to leakage caused by human error,
especially if the limited domain traffic appears otherwise normal to the
boundary routers. In this case, the network operator needs to take active
steps to protect the boundary. This form of leakage is much less likely if
nodes must be explicitly configured to handle a given limited-domain
protocol, for example, by installing a specific protocol handler."

There are a number of protocols which naturally have the concept of domain
baked into them - you've listed some, but there are also things which
primarily are L2 based, like RA and VRRP and similar. Apart from those,
much of it seems to come down to "what is the default behavior for the
protocol on the interface" - things like MPLS or (generally) OSPF need an
intentional act to extend the edge of the domain, and also generally need
the sender and receiver to both take an action. If I accidentally enable
OSPF on an interface towards you, you are likely not harmed because you are
not going to be willing to form an adjacency unless you have also done a
stupid on the matching interface. The fact that "failure" also requires a
chain (each interface/similar along a path needs config with helps with
this too, as does things like authentication (which is often/usually used
to prevent "Ooops" instead of malicious attacks).

It feels to me like the concept of limited-domains is sometimes used like a
magic incantation to avoid having to answer some of the interoperability
and security questions and concerns. Saying that a protocol should (or
SHOULD or MUST) only be used in a limited domain doesn't actually enforce
that it will not leak. I also do not think that it is reasonable that the
protection / filtering to enforce limited domains should require the
(potential) recipient to have to change their filtering stance; I should
not have to change my filtering policy to protect *my* network against
*you* accidentally forgetting a filter at the edge of your network.

If we do expect potential recipients to have to update filters, there are 2
failure modes:
1: the recipient doesn't (or misses an interface), and bad things happen or
2: we end up in the "default-deny" mode, where networks block all packets
except those that they explicitly allow. This ossifies the network and
makes deploying new things impossible.
Networks that experience breakage of the first type will quickly move to
the second option...

Ed's suggestion below ("All router’s ports should have RH type 4 (SRH)
filtered out by default.") is already heading down this path.

The way I see this playing out is operators quickly moving along the lines
of:
If I'm expected to filter "RH type 4 (SRH)" on all of my external
interfaces in case someone else decides to run SRH, what else should I
filter? Clearly 5 and 6 (CRH-16/32)... and 7, 8, ... 255.
I need to be ready *before* you enable $insert_new_protocol_here, so the
only sensible thing for me to do is filter all RH/43. Oh, and I don't know
what sort of new uses there might be for Hop-by-Hop that might affect me,
so I should clearly just filter all protocol 0 (perhaps in the future I
might allow specific ones... maybe). HIP? I don't use that, but I might in
the future, and I certainly don't have time to follow all of the new things
being proposed in the IETF/by vendors, so obviously I should filter that
everywhere...
Actually, why am I bothering to write this long filter list? I'll just
allow TCP and UDP, and call it done...

(When I started writing the above I thought I would need to add a "Warning:
Hyperbole ahead" marker -- but I'm no longer sure that that's the case).

This path leads us to an ossified IPv6, and no ability to deploy new
protocols. We need a solution where the border of limited-domains is clear,
and automatic - and where the things touching the outside border don't need
to actively protect themselves from failures of the border protection,
W
</no hats>

>   utilizing a gateway of 2001:db8:1:1::fefe – packets from 2001:db8:1:1::10
> flowing towards that gateway would pass ingress filter checks based on
> BCP38 since the source address was legitimate, unless we explicitly
> filtered out where that packet could be destined for based on its DA (More
> on this in a bit)
>
>
>
> attack encapsulates a packet that has an entirely different source address
> – and whatever random DA address inside an SRH.  That SRH has a SID list in
> it – be it via a direct SID or via compression mechanisms, and a DA towards
> the SID itself.  The packet then routes – passing the ingress checks – and
> landing up at the router with the first SID.  Since this was the only SID
> in the list – the packet outer SRH is removed – and the inner packet is
> forwarded to the FIB – and you just bypassed BCP38.
>
>
>
> In a similar mechanism, if the SRH states that the next protocol header is
> an IPv4 packet – when the de-encapsulation happens at last SID – the
> payload (the inner v4 packet) will then be forwarded via the FIB – and off
> you go.
>
>
>
> Now, normally to protect against this as stated – an access list would be
> placed on the networks borders to prevent encapsulated packets and packets
> containing SID destinations from entering the network.  However – if we
> consider the above scenario where the attack is coming from a compromised
> server inside the traditional borders – we have a problem.  The application
> of access lists to every port containing a server is also potentially
> unrealistic and unmaintainable (not to mention could potentially on certain
> devices overload the TCAMs)
>
>
>
> We also have an even more potentially deadly issue – and this is entirely
> theoretical at this point since I haven’t had time to really investigate
> and test it with real code.  Let us presume in the above that the same host
> A is compromised.  It encapsulates a packet with an SRH – the internal
> packet is an Ipv4 packet – and its destined at a broadcast address of an IP
> subnet that is bound to the same network as the de-encapsulating router.
> The source of the internal v4 packet is spoofed – we’ve now managed to –
> through a use of the tunneling mechanism, effectively created a version of
> an old tool called smurf – in a manner that is going to be pretty hard to
> trace.
>
>
>
> return packets and retransmits then. Each time applying the same SRH.  The
> inner packet could be pretty much anything – so long as the internal DA is
> set back to host A that is sending the packet.  The packet would then flow
> out as an SRH encapsulated packet, the outer header would get popped, the
> inner packet would flow back towards the original compromised host, which
> would match it modify it re-encapsulate it and start the loop again.  Doing
> this with ebpf and a kernel jit – would be pretty difficult to spot if you
> didn’t know what you are doing because you wouldn’t potentially see any
> obvious userland code.
>
>
>
> multiple VM’s on it – and the filtering implications of all of the above –
> and the filtering becomes even more difficult to maintain and more complex.
>
>
>
> desktop – that may not be realistic – especially when we could be running
> multiple ports that are handing out source addresses via RA – so – how do
> we solve this – or is this a major flaw in srv6 itself – that needs some
> other solution (give srv6 its own protocol code and acknowledge that it
> isn’t ipv6 at all, allowing for a “fail-closed” scenario maybe?)
>
>
>
> As an operator that runs extensive IPv6 – I’d really like to hear thoughts
> and comments and potentially we can find a way to address these issues.
>
>
>
> Thanks
>
>
>
> Andrew
>
>
>
>
>
>
>
>

-- 
The computing scientist’s main challenge is not to get confused by the
complexities of his own making.
  -- E. W. Dijkstra

---

From: [email-masked]
Date: Mon, 25 Oct 2021 11:04:53 +0200

Warren,

> The way I see this playing out is operators quickly moving along the lines of:
> If I'm expected to filter "RH type 4 (SRH)" on all of my external interfaces in case someone else decides to run SRH, what else should I filter? Clearly 5 and 6 (CRH-16/32)... and 7, 8, ... 255.
> I need to be ready *before* you enable $insert_new_protocol_here, so the only sensible thing for me to do is filter all RH/43. Oh, and I don't know what sort of new uses there might be for Hop-by-Hop that might affect me, so I should clearly just filter all protocol 0 (perhaps in the future I might allow specific ones... maybe). HIP? I don't use that, but I might in the future, and I certainly don't have time to follow all of the new things being proposed in the IETF/by vendors, so obviously I should filter that everywhere...
> Actually, why am I bothering to write this long filter list? I'll just allow TCP and UDP, and call it done...
> 
> (When I started writing the above I thought I would need to add a "Warning: Hyperbole ahead" marker -- but I'm no longer sure that that's the case).

Proposal for a principle: "If Internet traffic is carried across a limited domain, the technology used in the limited domain must not hinder end to end transparency."

You are welcome to suggest more concise ways of saying that.

In the above example, filtering packets with the SRH RH type _and_ destined towards your own limited domain prefix would adhere to that.

Cheers,
Ole

---

From: Andrew Alston <[email-masked]>
Date: Mon, 25 Oct 2021 09:12:20 +0000

Ole,

My problem here is that "limited domain" has become a vague and obscure term used to justify the violation of pretty much anything.

Furthermore - I would argue that in the case of srv6 - the concept of domain boundary has become so obscure and so vague as to be meaningless.  When limited domain used to have limited edge devices connecting to other networks - you could kinda get away with this - when your limited domain is now extended into a hybrid server/network world - and your domain edge is so vague and fuzzy as to encompass pretty much every port on the network - then the concept of limited domain is, in my view as an operator, meaningless.

I actually think it would be interesting to potentially form another working group in the IETF - to specifically look at the operational impact of ideas being proposed but on a "localized" basis - so while GROW is chartered to look at the impact of things on the wider internet - perhaps we need to start thinking about something that can analyze the impact of proposals from the perspective of a "limited domain" - inside that domain - and perhaps this could also be the place where we define what a "limited domain" really means - and what the boundaries of this really mean.

Might be something worth considering...

Andrew

-----Original Message-----
From: v6ops <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, October 25, 2021 12:05 PM
To: Warren Kumari <[email-masked]>
Cc: [email-masked]
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Warren,

Cheers,
Ole

---

From: Brian Carpenter <[email-masked]>
Date: Mon, 25 Oct 2021 22:28:50 +1300

Andrew,
That's what
https://www.rfc-editor.org/rfc/rfc8799.html#name-functional-requirements-of-
is getting at. We need more precision indeed, and that requirements list
was intended as a starting point. (We also have an example: RFC8994.)

Regards,
    Brian Carpenter
    (via tiny screen & keyboard)

On Mon, 25 Oct 2021, 22:12 Andrew Alston, <Andrew.Alston=
[email-masked]> wrote:

> Ole,
>
> My problem here is that "limited domain" has become a vague and obscure
> term used to justify the violation of pretty much anything.
>
> Furthermore - I would argue that in the case of srv6 - the concept of
> domain boundary has become so obscure and so vague as to be meaningless.
> When limited domain used to have limited edge devices connecting to other
> networks - you could kinda get away with this - when your limited domain is
> now extended into a hybrid server/network world - and your domain edge is
> so vague and fuzzy as to encompass pretty much every port on the network -
> then the concept of limited domain is, in my view as an operator,
> meaningless.
>
> I actually think it would be interesting to potentially form another
> working group in the IETF - to specifically look at the operational impact
> of ideas being proposed but on a "localized" basis - so while GROW is
> chartered to look at the impact of things on the wider internet - perhaps
> we need to start thinking about something that can analyze the impact of
> proposals from the perspective of a "limited domain" - inside that domain -
> and perhaps this could also be the place where we define what a "limited
> domain" really means - and what the boundaries of this really mean.
>
> > (When I started writing the above I thought I would need to add a
> "Warning: Hyperbole ahead" marker -- but I'm no longer sure that that's the
> case).
>
> Proposal for a principle: "If Internet traffic is carried across a limited
> domain, the technology used in the limited domain must not hinder end to
> end transparency."
>
> In the above example, filtering packets with the SRH RH type _and_
> destined towards your own limited domain prefix would adhere to that.
>
> Cheers,
> Ole
>

---

From: Andrew Alston <[email-masked]>
Date: Mon, 25 Oct 2021 09:31:38 +0000

So Brian,

Let me ask you this – would you say that, in your opinion, that SRv6 falls into the definition of limited-domain as per that document.  I’m still going through it closely to make up my own mind – but I’m curious as to your perspective on this

Andrew

From: Brian Carpenter <[email-masked]>
Sent: Monday, October 25, 2021 12:29 PM
To: Andrew Alston <[email-masked]>
Cc: Ole Troan <[email-masked]>; Warren Kumari <[email-masked]>; IPv6 Ops WG <[email-masked]>
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Andrew,
That's what https://www.rfc-editor.org/rfc/rfc8799.html#name-functional-requirements-of-<https://www.rfc-editor.org/rfc/rfc8799.html#name-functional-requirements-of-> is getting at. We need more precision indeed, and that requirements list was intended as a starting point. (We also have an example: RFC8994.)
Regards,
    Brian Carpenter
    (via tiny screen & keyboard)

On Mon, 25 Oct 2021, 22:12 Andrew Alston, <Andrew.Alston=[email-masked]<mailto:[email-masked]>> wrote:
Ole,

Andrew

-----Original Message-----
From: v6ops <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Monday, October 25, 2021 12:05 PM
To: Warren Kumari <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Warren,

Cheers,
Ole

---

From: Warren Kumari <[email-masked]>
Date: Mon, 25 Oct 2021 17:20:51 -0400

> domain, the technology used in the limited domain must not hinder end to

<no hats/>

I must admit I'm having a hard time parsing the last part of the sentence.
I think it is more:
"A limited domain must default to fail closed, and require explicit
configuration to open it."

This is similar to e.g MPLS - I have to manually enable MPLS on an
interface ('set protocols mpls interface et-0/0/0') before the border of my
limited domain extends out the interface, and my neighbor has to enable it
on their side too. We *can* both choose to do so, but if we don't, the
domain doesn't cross the border. Without requiring explicit configuration
it isn't really a "limited domain", it's just a "domain".

> destined towards your own limited domain prefix would adhere to that.

Filters get missed sometimes, both ingress and egress. Yes, I can filter
towards my limited domain prefix -- but this only protects *my SRH limited
domain*. I'm also likely to want to protect my customers (who cannot be
trusted to always filter perfectly), if I have an enterprise portion I have
to know whatever prefixes they decide to use and update my filters, etc.
The limited part of "limited domain" implies that it has a clear border
(and also that whatever you do within the privacy of your domain is your
own business!); having a soft border fails the "limited" sniff test.

I somewhat like the idea of having a well known prefix for "limited
domains" - if everyone used $prefix, we could default to filtering it on
external links, and, just like MPLS/OSPF/IS-IS/<whatever>, consciously
decide to allow it between consenting adults. This is far from perfect --
it requires more routes in my IGP, etc, but it's better than nothing.

W

---

From: Gert Doering <[email-masked]>
Date: Mon, 25 Oct 2021 23:31:09 +0200

Hi,

> it requires more routes in my IGP, etc, but it's better than nothing.

Indeed, that would be a good start.

Implementors would have to ensure that decapsulation of "things" only
happen for packets destined to this prefix, and not "outside of this
domain" interfaces (IXPs, customer attachment circuits, etc).

Gert Doering
        -- NetMaster

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 26 Oct 2021 10:38:04 +1300

Andrew,

Segment routing and newtork programming are called out in RFC8799 section 
4 as an example, so yes.

I now realise that the sections 6 & 7 of that RFC should have covered leakage as well other aspects of the domain boundary. But the main argument there is: identify the interior nodes and boundary nodes explicitly. If you do that, you have chance at both protecting the domain from outside interference and protecting the outside from leakage. So I'd welcome the kind of WG you suggest.

Regards
    Brian Carpenter
    Thinking of the IETF standards process: https://xkcd.com/2530/

On 25-Oct-21 22:31, Andrew Alston wrote:
> So Brian,
> 
> Let me ask you this – would you say that, in your opinion, that 
SRv6 falls into the definition of limited-domain as per that document.  I’m still going through it closely to make up my own mind – but I’m curious as to your perspective on this
> 
> Andrew
> 
> *From:* Brian Carpenter <[email-masked]>
> *Sent:* Monday, October 25, 2021 12:29 PM
> *To:* Andrew Alston <[email-masked]>
> *Cc:* Ole Troan <[email-masked]>; Warren Kumari <[email-masked]>; IPv6 Ops WG <[email-masked]>
> *Subject:* Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?
> 
> Andrew,
> 
> That's what https://www.rfc-editor.org/rfc/rfc8799.html#name-functional-requirements-of- <https://www.rfc-editor.org/rfc/rfc8799.html#name-functional-requirements-of-> is getting at. We need more precision indeed, and 
that requirements list was intended as a starting point. (We also have an 
example: RFC8994.)
> On Mon, 25 Oct 2021, 22:12 Andrew Alston, <Andrew.Alston=[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Ole,
> 
>     Furthermore - I would argue that in the case of srv6 - the concept of domain boundary has become so obscure and so vague as to be meaningless.  When limited domain used to have limited edge devices connecting 
to other networks - you could kinda get away with this - when your limited domain is now extended into a hybrid server/network world - and your domain edge is so vague and fuzzy as to encompass pretty much every port on 
the network - then the concept of limited domain is, in my view as an operator, meaningless.
> 
>     I actually think it would be interesting to potentially form another working group in the IETF - to specifically look at the operational impact of ideas being proposed but on a "localized" basis - so while GROW is 
chartered to look at the impact of things on the wider internet - perhaps 
we need to start thinking about something that can analyze the impact of proposals from the perspective of a "limited domain" - inside that domain 
- and perhaps this could also be the place where we define what a "limited domain" really means - and what the boundaries of this really mean.
>     From: v6ops <[email-masked] <mailto:[email-masked]>> On Behalf Of [email-masked] <mailto:[email-masked]>
>     To: Warren Kumari <[email-masked] <mailto:[email-masked]>>
>     Cc: [email-masked] <mailto:[email-masked]>
>      > I need to be ready *before* you enable $insert_new_protocol_here, so the only sensible thing for me to do is filter all RH/43. Oh, and I don't know what sort of new uses there might be for Hop-by-Hop that might 
affect me, so I should clearly just filter all protocol 0 (perhaps in the 
future I might allow specific ones... maybe). HIP? I don't use that, but I might in the future, and I certainly don't have time to follow all of the new things being proposed in the IETF/by vendors, so obviously I should filter that everywhere...
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/listinfo/v6ops <https://www.ietf.org/mailman/listinfo/v6ops>
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 26 Oct 2021 10:44:32 +1300

On 26-Oct-21 10:31, Gert Doering wrote:
>> domains"

fc00::/7 works well. RFC8994 is a worked example.

    Brian

> - if everyone used $prefix, we could default to filtering it on

---

From: Gert Doering <[email-masked]>
Date: Mon, 25 Oct 2021 23:47:06 +0200

Hi,

> fc00::/7 works well. RFC8994 is a worked example.

So how would that work for an ISP network trying to run SR6, protecting
its network from rogue hosts inside?  Without having GUAs on the SR6
routers that would happily decapsulate incoming SR6 packets, and 
without violating lots of rules about "do not leak ULAs outside
your network" (traceroute and other ICMP errors)?

I lack imagination today...

Gert Doering
        -- NetMaster

---

From: Andrew Alston <[email-masked]>
Date: Mon, 25 Oct 2021 22:12:40 +0000

It would help to have a dedicated prefix - as a start - does it really solve the issue though.

Imagine a device with a stack of cloud servers behind it - each server terminates on a separate sub-interface - and now you're trying to apply what is in effect an LPM filter to each and every interface (as compared to host based firewalling or other security mechanisms) - my question is then - how far will your tcam go - how practical is this in reality.

This is as opposed to if there is a separate ether-type where an interface is either configured to process it or drop it on the floor.  So yes - the prefix filtering will help - but I suspect that you're gonna find many many scenarios where this actually doesn’t work - and all you need is *one* filter miss that has a compromised server behind it to have real problems.

Something I haven't got around to fully testing yet - but let me throw out an interesting scenario on my list of tests to do.

Rogue host A takes an IPv4 packet and encapsulates it in an SRH - the First sid in there is any v6 router you like on the path - the packet gets there - it get de-encapped - same as everything I've said before.  Now - the inner packet is a v4 packet - it has a source set to random host attacker doesn’t like - and its destination is 255.255.255.255  - which thanks to rfc919 will not forward - when that deencap happens - does the packet get dropped because it can't be forwarded - or does it get treated as a local broadcast.  This is a rather undefined scenario - if however it DOES get treated as a local broadcast - well - then we have a really big problem - that’s called smurf-v2 (and even if 255.255.255.255 doesn’t work - more specific broadcasts that are attached may well work). At this point - when the broadcast packet hits the hosts behind those broadcasts - they will reply to the spoofed source - this will follow stock standard routing outbound - and the protections we would normally use aren’t gonna work (the source of replies to the broadcast packets would be the hosts - they are permitted to send packets to the internet)

Another scenario - sending to multicast addresses post de-encap - do we have a potential attack vector to poison ND?  Again - haven't had the time to test this.

Same thing applies to a whole long list of other things - effectively - if you get one compromised host on the network that can inject a packet that will de-encap and act on the inner packet - with absolutely zero mechanisms for verification of what is in that inner packet or how to handle it - the list of possible problems is - extensive to say the last.  All I am saying is - we need to really step back and think - and I think this needs a veryyyyy close look - because in initial tests  I have done - and without the above additional tests - I can tell you my results are looking positively scary (and once I complete the full set I'll publish some scenarios and results with more detail)

Andrew

On 26/10/2021, 00:48, "v6ops on behalf of Gert Doering" <[email-masked] on behalf of [email-masked]> wrote:

    Hi,

    Gert Doering
            -- NetMaster

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 26 Oct 2021 11:20:44 +1300

Yes. Rogue hosts *inside* the domain are always going to be a problem.
I don't see how a magic prefix can ever help with that.

Regards
    Brian

On 26-Oct-21 11:12, Andrew Alston wrote:
> This is as opposed to if there is a separate ether-type where an interface is either configured to process it or drop it on the floor.  So yes - 
the prefix filtering will help - but I suspect that you're gonna find many many scenarios where this actually doesn’t work - and all you need is *one* filter miss that has a compromised server behind it to have real problems.
> Rogue host A takes an IPv4 packet and encapsulates it in an SRH - the First sid in there is any v6 router you like on the path - the packet gets 
there - it get de-encapped - same as everything I've said before.  Now - the inner packet is a v4 packet - it has a source set to random host attacker doesn’t like - and its destination is 255.255.255.255  - which thanks to rfc919 will not forward - when that deencap happens - does the packet get dropped because it can't be forwarded - or does it get treated as a local broadcast.  This is a rather undefined scenario - if however it DOES get treated as a local broadcast - well - then we have a really big problem - that’s called smurf-v2 (and even if 255.255.255.255 doesn’t work - more specific broadcasts that are attached may well work). At this point - when the broadcast packet hits the hosts behind those broadcasts - they will reply to the spoofed source - this will follow stock standard routing outbound - and the protections we would normally use aren’t gonna work (the source of replies to the broadcast packets would be the hosts - they are permitted to send packets to the internet)
> 
> Another scenario - sending to multicast addresses post de-encap - do we 
have a potential attack vector to poison ND?  Again - haven't had the time to test this.
> 
> Same thing applies to a whole long list of other things - effectively - 
if you get one compromised host on the network that can inject a packet that will de-encap and act on the inner packet - with absolutely zero mechanisms for verification of what is in that inner packet or how to handle it - the list of possible problems is - extensive to say the last.  All I 
am saying is - we need to really step back and think - and I think this needs a veryyyyy close look - because in initial tests  I have done - and without the above additional tests - I can tell you my results are looking positively scary (and once I complete the full set I'll publish some scenarios and results with more detail)

---

From: Andrew Alston <[email-masked]>
Date: Mon, 25 Oct 2021 22:31:46 +0000

Except Brian,

Normally – a single rogue host because of other protections can’t do near this level of damage – because they can’t send packets that can get decapped in the middle of the network with no way to define what happens post decap with the inner packet.  Normally you could apply a whole range of measures to ensure that a single host may be able to send a stream of packets to some host – at that point – it gets found and stopped.  In this case – that rogue host now has the abiity to inject packets practically anywhere on the network that cause attacks to happen against external hosts – and good luck tracing it.

The ”magic” prefix lets you filter which hosts can send to SRH SID’s – but as I said – it’s far from perfect – because there are still a host of issues with that.  But – lets also be serious – the level of damage a single rogue host could do in these scenarios is wayyyyyy beyond what it could do if it couldn’t cause the decap and act on <insert random router anywhere in the domain>

Andrew

From: Brian E Carpenter <[email-masked]>
Date: Tuesday, 26 October 2021 at 01:20
To: Andrew Alston <[email-masked]>, Gert Doering <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

Regards
Brian

> Rogue host A takes an IPv4 packet and encapsulates it in an SRH - the First sid in there is any v6 router you like on the path - the packet gets
there - it get de-encapped - same as everything I've said before. Now - the inner packet is a v4 packet - it has a source set to random host attacker doesn’t like - and its destination is 255.255.255.255<http://255.255.255.255> - which thanks to rfc919 will not forward - when that deencap happens - does the packet get dropped because it can't be forwarded - or does it get treated as a local broadcast. This is a rather undefined scenario - if however it DOES get treated as a local broadcast - well - then we have a really big problem - that’s called smurf-v2 (and even if 255.255.255.255<http://255.255.255.255> doesn’t work - more specific broadcasts that are attached may well work). At this point - when the broadcast packet hits the hosts behind those broadcasts - they will reply to the spoofed source - this will follow stock standard routing outbound - and the protections we would normally use aren’t gonna work (the source of replies to the broadcast packets would be the hosts - they are permitted to send packets to the internet)

---

From: Warren Kumari <[email-masked]>
Date: Mon, 25 Oct 2021 18:43:52 -0400

> It would help to have a dedicated prefix - as a start - does it really

That depends on what we are meaning by "the issue" -- I think that it is
more "issues" :-)

What it does do is allow me to filter out *your* SRH traffic at my border
ingress (and also makes it easier for you to filter at egress) - this gets
us something that looks and behaves much more like an MPLS edge.

I fully agree that it doesn't protect against a compromised host that is
participating in the limited domain itself -- but, hopefully it does limit
some of the fallout from that to just within the domain itself. Sorry to
sound uncaring, but if you end up with a compromised server creating
malicious packets within your SRH domain, I'd much rather that they
only affect you and not me :-)
Having a crunchy edge may also help with determining what should actually
participate in the limited domain -- if you include a stack of cloud
servers participating in SRH, yes, they are now able send SRH packets. This
is a conscious decision that you've made -- it might be a perfectly fine
one, but you've intentionally extended the edge of the domain to include
this. In my ideal scenario, this prefix is by default filtered, and you'd
need to explicitly allow it on a per interface basis.

>
> Imagine a device with a stack of cloud servers behind it - each server
> terminates on a separate sub-interface - and now you're trying to apply
> what is in effect an LPM filter to each and every interface (as compared to
> host based firewalling or other security mechanisms) - my question is then
> - how far will your tcam go - how practical is this in reality.
>
> This is as opposed to if there is a separate ether-type where an interface
> is either configured to process it or drop it on the floor.

I much much prefer a separate ether-type -- but I suspect that that ship
has sailed.

> So yes - the prefix filtering will help - but I suspect that you're gonna
> find many many scenarios where this actually doesn’t work - and all you
> need is *one* filter miss that has a compromised server behind it to have
> real problems.
>
> Something I haven't got around to fully testing yet - but let me throw out
> an interesting scenario on my list of tests to do.
>
> Rogue host A takes an IPv4 packet and encapsulates it in an SRH - the
> First sid in there is any v6 router you like on the path - the packet gets
> there - it get de-encapped - same as everything I've said before.

I'm assuming that in this scenario your "Rogue host A" is one that has
intentionally added to the SRH domain (either by allowing your new
ether-type, or manually removing the "limited domain prefix filter")? If
so, then yes, bad things happen. This seems largely the same as "I decided
to add "Rogue host A" to my MPLS domain so it could MPLS", or I added
"Rogue host A" to my IGP, because reasons...
Making a host able to influence routing, whether it is part of
something like MPLS, SR<something>, or in your IGP makes that host able to
do bad things...

W

Now - the inner packet is a v4 packet - it has a source set to random host
> attacker doesn’t like - and its destination is 255.255.255.255  - which
> thanks to rfc919 will not forward - when that deencap happens - does the
> packet get dropped because it can't be forwarded - or does it get treated
> as a local broadcast.  This is a rather undefined scenario - if however it
> DOES get treated as a local broadcast - well - then we have a really big
> problem - that’s called smurf-v2 (and even if 255.255.255.255 doesn’t work
> - more specific broadcasts that are attached may well work). At this point
> - when the broadcast packet hits the hosts behind those broadcasts - they
> will reply to the spoofed source - this will follow stock standard routing
> outbound - and the protections we would normally use aren’t gonna work (the
> source of replies to the broadcast packets would be the hosts - they are
> permitted to send packets to the internet)

> Same thing applies to a whole long list of other things - effectively - if
> you get one compromised host on the network that can inject a packet that
> will de-encap and act on the inner packet - with absolutely zero mechanisms
> for verification of what is in that inner packet or how to handle it - the
> list of possible problems is - extensive to say the last.  All I am saying
> is - we need to really step back and think - and I think this needs a
> veryyyyy close look - because in initial tests  I have done - and without
> the above additional tests - I can tell you my results are looking
> positively scary (and once I complete the full set I'll publish some
> scenarios and results with more detail)
>
> Andrew
>
>
> On 26/10/2021, 00:48, "v6ops on behalf of Gert Doering" <
> [email-masked] on behalf of [email-masked]> wrote:
>
>     Hi,
>

---

From: Andrew Alston <[email-masked]>
Date: Mon, 25 Oct 2021 22:50:58 +0000

I'm assuming that in this scenario your "Rogue host A" is one that has intentionally added to the SRH domain (either by allowing your new ether-type, or manually removing the "limited domain prefix filter")? If so, then yes, bad things happen. This seems largely the same as "I decided to add "Rogue host A" to my MPLS domain so it could MPLS", or I added "Rogue host A" to my IGP, because reasons...
Making a host able to influence routing, whether it is part of something like MPLS, SR<something>, or in your IGP makes that host able to do bad things...

See that’s the thing Warren – with MPLS – it’s a default fail-closed – with a prefix list that has to be applied to every interface – well – things get missed.  And these prefix lists would have be applied to every L3 interface – and again – I fear you’re going to have tcam problems if you try and apply an LPM filter to every interface on a massive device with thousands of interfaces.  You can’t apply filters in the way you would for CoPP – these become specific prefix filters.  I’m just convinced that’s either practical or it’s going to really work,.  Also – if you look at the scenario I sketched – that rogue doesn’t doesn’t just affect you – because if it changes the source of the inner packet and causes it to broadcast – the replies are going to affect things on a far wider basis – your routers just became amplifiers for an attack aimed externally.

You are right – the ether-type may have sailed – but as I said – so far – my tests indicate that a single host would allow you to launch far more damaging attacks in this scenario than it would if it couldn’t inject weird packets

Andrew

Now - the inner packet is a v4 packet - it has a source set to random host attacker doesn’t like - and its destination is 255.255.255.255<http://255.255.255.255> - which thanks to rfc919 will not forward - when that deencap happens - does the packet get dropped because it can't be forwarded - or does it get treated as a local broadcast. This is a rather undefined scenario - if however it DOES get treated as a local broadcast - well - then we have a really big problem - that’s called smurf-v2 (and even if 255.255.255.255<http://255.255.255.255> doesn’t work - more specific broadcasts that are attached may well work). At this point - when the broadcast packet hits the hosts behind those broadcasts - they will reply to the spoofed source - this will follow stock standard routing outbound - and the protections we would normally use aren’t gonna work (the source of replies to the broadcast packets would be the hosts - they are permitted to send packets to the internet)

Andrew

On 26/10/2021, 00:48, "v6ops on behalf of Gert Doering" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

    Hi,

    Gert Doering
            -- NetMaster

---

From: Warren Kumari <[email-masked]>
Date: Mon, 25 Oct 2021 18:51:42 -0400

On Mon, Oct 25, 2021 at 6:32 PM Andrew Alston <Andrew.Alston=
[email-masked]> wrote:

> Except Brian,
>
>
>
> Normally – a single rogue host because of other protections can’t do near
> this level of damage – because they can’t send packets that can get
> decapped in the middle of the network with no way to define what happens
> post decap with the inner packet.  Normally you could apply a whole range
> of measures to ensure that a single host may be able to send a stream of
> packets to some host – at that point – it gets found and stopped.  In this
> case – that rogue host now has the abiity to inject packets practically
> anywhere on the network that cause attacks to happen against external hosts
> – and good luck tracing it.
>
>
>
> The ”magic” prefix lets you filter which hosts can send to SRH SID’s – but
> as I said – it’s far from perfect – because there are still a host of
> issues with that.
>

Yes. I'm **certainly** not disagreeing that it's far from perfect....

> But – lets also be serious – the level of damage a single rogue host could
> do in these scenarios is wayyyyyy beyond what it could do if it couldn’t
> cause the decap and act on <insert random router anywhere in the domain>
>

Yup. Allowing a host to participate in making routing decisions (whether
that is participating in an IGP, or allowing it to steer its own traffic
and encap/decap) makes the host privileged. I'm trying to make it so that
it's harder for unprivileged things to accidentally (or maliciously) become
part of the privileged domain

>
>
> Andrew
>
>
>
>
>
> *From: *Brian E Carpenter <[email-masked]>
> *Date: *Tuesday, 26 October 2021 at 01:20
> *To: *Andrew Alston <[email-masked]>, Gert Doering <
> [email-masked]>
> *Cc: *"[email-masked]" <[email-masked]>
> *Subject: *Re: [v6ops] Security issues in RFC8754 and related/subsequent
> drafts?
>
>
>
> > This is as opposed to if there is a separate ether-type where an
> interface is either configured to process it or drop it on the floor. So
> yes -
> the prefix filtering will help - but I suspect that you're gonna find many
> many scenarios where this actually doesn’t work - and all you need is *one*
> filter miss that has a compromised server behind it to have real problems.
> >
> > Something I haven't got around to fully testing yet - but let me throw
> out an interesting scenario on my list of tests to do.
> >
> there - it get de-encapped - same as everything I've said before. Now -
> the inner packet is a v4 packet - it has a source set to random host
> problem - that’s called smurf-v2 (and even if 255.255.255.255 doesn’t
> work - more specific broadcasts that are attached may well work). At this
> point - when the broadcast packet hits the hosts behind those broadcasts -
> they will reply to the spoofed source - this will follow stock standard
> routing outbound - and the protections we would normally use aren’t gonna
> work (the source of replies to the broadcast packets would be the hosts -
> they are permitted to send packets to the internet)
> >
> if you get one compromised host on the network that can inject a packet
> that will de-encap and act on the inner packet - with absolutely zero
> mechanisms for verification of what is in that inner packet or how to
> handle it - the list of possible problems is - extensive to say the last.
> All I
> am saying is - we need to really step back and think - and I think this
> needs a veryyyyy close look - because in initial tests I have done - and
> without the above additional tests - I can tell you my results are looking

---

From: Warren Kumari <[email-masked]>
Date: Mon, 25 Oct 2021 19:11:32 -0400

On Mon, Oct 25, 2021 at 6:51 PM Andrew Alston <
[email-masked]> wrote:

> See that’s the thing Warren – with MPLS – it’s a default fail-closed
>
Nod. It sounds like you think I'm disagreeing with you, which confuses me
some -- I think that we are both looking for a failed closed system.

> – with a prefix list that has to be applied to every interface – well –
> things get missed.
>
Yes. Hence my "by default it would be applied".

In the same way I manually add an interface to MPLS:
protocol {
   mpls {
       interface et-0/0/0.0;
   }
}

I would manually add the interface to allow SRH:
protocol {
   srh {
   interface et-0/0/0;
   }
}
(or perhaps something like "set interface et-0/0/0.0 family srh allow'", or
whatever).

The solution would default to fail-closed -- there is a default "deny
$srh_prefix" on the interface and I manually permit it by telling the
device that the interface participates in SRH (or whatever other limited
domain protocols are defined to use space). The above example has some
syntactic sugar, but the actual implementation is simply that the device
allows packets from $limited_domain_prefix when you set this...

>   And these prefix lists would have be applied to every L3 interface – and
> again – I fear you’re going to have tcam problems if you try and apply an
> LPM filter to every interface on a massive device with thousands of
> interfaces.  You can’t apply filters in the way you would for CoPP – these
> become specific prefix filters.
>

I don't think that it is a specific prefix filter (or I'm misunderstanding
you) -- if you add the interface to the SRH limited domain, the filter
blocking the (well-known) prefix is removed. It's a single, well known
prefix.

> I’m just convinced that’s either practical or it’s going to really work,.
>

I think that my proposal is largely the same as a different ether-type - it
is implemented as a default block ACL, which is removed when you add the
interface to the domain (instead of allowing a different ether-type).
Depending on implementation it may burn a single TCAM slot per interface,
but implementing it should be simple....

> Also – if you look at the scenario I sketched – that rogue doesn’t doesn’t
> just affect you – because if it changes the source of the inner packet and
> causes it to broadcast – the replies are going to affect things on a far
> wider basis – your routers just became amplifiers for an attack aimed
> externally.
>

Yes, we are in full agreement.

>
>
> You are right – the ether-type may have sailed – but as I said – so far –
> my tests indicate that a single host would allow you to launch far more
> damaging attacks in this scenario than it would if it couldn’t inject weird
> packets
>

Again, I fully agree.... which is why we don't want unexpected hosts to be
able to do this :-)

W

---

From: Andrew Alston <[email-masked]>
Date: Tue, 26 Oct 2021 00:39:51 +0000

Thanks Warren for the clarifications – I think we’re pretty much aligned, just my misreading of certain things – apologies for that.

The idea of a fixed prefix that is a default fail-closed in the way you proposed is certainly a major step in the right direction and I like that idea – it would address a LOT of my concerns.  By doing this we could also potentially avoid interface-specific filters and dependent on implementation not send up blowing up the tcams.

I think – the other thing we could consider is saying that we get potentially a /16 worth of space for this – and strongly suggest that to make up the first 48 bits we then append the asn in the next 32 bits – if you did this – you open the door to inter-domain controlled srv6 between operators – but without the risk of duplicate addressing etc.

So yeah – this is certainly worth considering

Andrew

From: Warren Kumari <[email-masked]>
Date: Tuesday, 26 October 2021 at 02:12
To: Andrew Alston <[email-masked]>
Cc: Gert Doering <[email-masked]>, Brian E Carpenter <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [v6ops] Security issues in RFC8754 and related/subsequent drafts?

On Mon, Oct 25, 2021 at 6:51 PM Andrew Alston <[email-masked]<mailto:[email-masked]>> wrote:
I'm assuming that in this scenario your "Rogue host A" is one that has intentionally added to the SRH domain (either by allowing your new ether-type, or manually removing the "limited domain prefix filter")? If so, then yes, bad things happen. This seems largely the same as "I decided to add "Rogue host A" to my MPLS domain so it could MPLS", or I added "Rogue host A" to my IGP, because reasons...
Making a host able to influence routing, whether it is part of something like MPLS, SR<something>, or in your IGP makes that host able to do bad things...

See that’s the thing Warren – with MPLS – it’s a default fail-closed
Nod. It sounds like you think I'm disagreeing with you, which confuses me some -- I think that we are both looking for a failed closed system.

– with a prefix list that has to be applied to every interface – well – things get missed.
Yes. Hence my "by default it would be applied".

I would manually add the interface to allow SRH:
protocol {
   srh {
   interface et-0/0/0;
   }
}
(or perhaps something like "set interface et-0/0/0.0 family srh allow'", or whatever).

The solution would default to fail-closed -- there is a default "deny $srh_prefix" on the interface and I manually permit it by telling the device that the interface participates in SRH (or whatever other limited domain protocols are defined to use space). The above example has some syntactic sugar, but the actual implementation is simply that the device allows packets from $limited_domain_prefix when you set this...

  And these prefix lists would have be applied to every L3 interface – and again – I fear you’re going to have tcam problems if you try and apply an LPM filter to every interface on a massive device with thousands of interfaces.  You can’t apply filters in the way you would for CoPP – these become specific prefix filters.

I don't think that it is a specific prefix filter (or I'm misunderstanding you) -- if you add the interface to the SRH limited domain, the filter blocking the (well-known) prefix is removed. It's a single, well known prefix.

I think that my proposal is largely the same as a different ether-type - it is implemented as a default block ACL, which is removed when you add the interface to the domain (instead of allowing a different ether-type). Depending on implementation it may burn a single TCAM slot per interface, but implementing it should be simple....

Also – if you look at the scenario I sketched – that rogue doesn’t doesn’t just affect you – because if it changes the source of the inner packet and causes it to broadcast – the replies are going to affect things on a far wider basis – your routers just became amplifiers for an attack aimed externally.

Again, I fully agree.... which is why we don't want unexpected hosts to be able to do this :-)

W

Andrew

Andrew

    Hi,

    Gert Doering
            -- NetMaster

---

From: Mark Smith <[email-masked]>
Date: Tue, 26 Oct 2021 13:03:08 +1100

The ULA address space is designed for this.

If a single /48 isn't big enough, generate another one.

However, that doesn't, nor will a new IANA /16 solve the leaks out of a
"controlled domain" problem.

Only the following solve that:

- a default free network (networks' default routes are what makes packets
try to leave by default (leak) if there is no matching local destination).
This won't solve the unauthorised host inside the network problem.

- authenticated packets such that unauthorised or unintended nodes cannot
send or receive them and successful process them i.e. AH at a minimum. That
solves the problem, however then you have the key management issue.

- a local network only protocol such as MPLS, limited to use within the
network, rather than a general purpose host-to-host/end-to-end global
internetworking protocol like IPv6. Not as good as the authenticated packet
option, however as long as hosts don't have this protocol enabled by
default it is better than the default free network option.

For a new IANA prefix to be assured closed, it would have to be null routed
in every router by default at the factory that could conceivably be used in
an SRv6 domain. The safest thing would be for this null route to be present
by default in literally every IPv6 router.

That sort of constraint hasn't worked 100% reliably with IPv6 Link-Local
addressed packets. They're supposed to be trapped on the link if they have
either an LLA source or destination address.

However, they are known to leak off when the DA is a GUA and the SA is a
LLA because router implementations aren't doing SA filtering. That can
happen fairly easily - if a host receives a GUA DNS resolver addresses, but
for some reason doesn't have a GUA SA to use, and has a defaultIPv6  router
(i.e. the RA doesn't have PIOs in it - and I've seen a broken residential
IPv6 CPE do this in the last couple of months) default address selection
says to fall back to using an LLA SA for all DNS queries to the GUA DA.

Regards,
Mark.

On Tue, 26 Oct 2021, 11:40 Andrew Alston, <Andrew.Alston=
[email-masked]> wrote:

> Thanks Warren for the clarifications – I think we’re pretty much aligned,
> just my misreading of certain things – apologies for that.
>
>
>
> The idea of a fixed prefix that is a default fail-closed in the way you
> proposed is certainly a major step in the right direction and I like that
> idea – it would address a LOT of my concerns.  By doing this we could also
> potentially avoid interface-specific filters and dependent on
> implementation not send up blowing up the tcams.
>
>
>
> I think – the other thing we could consider is saying that we get
> potentially a /16 worth of space for this – and strongly suggest that to
> make up the first 48 bits we then append the asn in the next 32 bits – if
> you did this – you open the door to inter-domain controlled srv6 between
> operators – but without the risk of duplicate addressing etc.
>
>
>
> *From: *Warren Kumari <[email-masked]>
> *Date: *Tuesday, 26 October 2021 at 02:12
> *To: *Andrew Alston <[email-masked]>
> *Cc: *Gert Doering <[email-masked]>, Brian E Carpenter <
> [email-masked]>, "[email-masked]" <[email-masked]>
> I think that my proposal is largely the same as a different ether-type -
> it is implemented as a default block ACL, which is removed when you add the

---

From: Gert Doering <[email-masked]>
Date: Tue, 26 Oct 2021 08:38:51 +0200

Hi,

> us something that looks and behaves much more like an MPLS edge.

Does it?

This would then require that all SRH decapsulating nodes *inside* your 
network understand that they must only ever decap SRH if it's addressed
to ULA addresses ("my limited domain prefix"), and not to any GUA the 
router might also have.

Like, because of IXP, upstream, customer attachment interfaces...

... which can be done, but I think would have to be made very clear in
the standard documents: only ever decapsulate for destination addresses
that have been enabled (empowered?) by admin config.

Gert Doering
        -- NetMaster

---

From: [email-masked]
Date: Tue, 26 Oct 2021 09:12:51 +0200

> I must admit I'm having a hard time parsing the last part of the sentence. I think it is more:
> "A limited domain must default to fail closed, and require explicit configuration to open it."

It means: an operator's implementation of a limited domain should not impede anyone transitting that network to use the same mechanism.

O.