Subject: Re: [6lo] ND cache entries creation on first-hop routers

From: Michael Richardson <[email-masked]>
Date: Tue, 02 Jul 2019 11:37:36 -0400

I think that the discussion here is particularly relevant to constrained
devices/routers on route-over MESH(RPL,etc.) networks.

I also think that for L=0 networks, which RPL creates with RPL DIO messages
rather than (just) RAs, and 6LRs that need to support join operations
(like draft-ietf-6tisch-minimal-security) this may matter.

In particular, in the minimal-security case, we need to partition the ND
cache such that untrusted (unverified) malicious pledge nodes can not
attack the ND cache.

The behaviour 2.2.1.  Host Sending Unsolicited NA, should probably
never flush an old entry out of the ND.  I think that under attack
(whether from untrusted pledges, or from p0woned devices already on the
network), it is better to prefer communication from existing nodes rather
than new ones.  2.2.1.2 mentions this.

{typo:
      -It's recommended that thsi functionality is configurable and
      +It's recommended that this functionality is configurable and
}

I didn't really understand 2.2.2: is it exploiting some corner case in the
spec, or maybe just some part I am not well clued in about.  So maybe an
extra paragraph to explain things.

I kinda like the ping all routers trick.

Jen Linkova <[email-masked]> wrote:
    > I wrote a short draft to discuss and document an operational issue
    > related to the ND state machine and packet loss caused by how routers
    > create ND cache entries for new host addressed:

    > https://datatracker.ietf.org/doc/draft-linkova-v6ops-nd-cache-init/

    > (taking into account some vendors have implemented one of the proposed
    > solution already, I guess it's a well-known problem but it might still
    > worth documenting)

    > Comments are appreciated!

    > --
    > SY, Jen Linkova aka Furry

    > --------------------------------------------------------------------
    > IETF IPv6 working group mailing list
    > [email-masked]
    > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
    > --------------------------------------------------------------------

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Jen Linkova <[email-masked]>
Date: Wed, 3 Jul 2019 16:51:57 +1000

> (like draft-ietf-6tisch-minimal-security) this may matter.

Disclaimer: I have very limited knowledge in that area.

> cache such that untrusted (unverified) malicious pledge nodes can not

The next version of the draft will have much more details on
discussing the security considerations indeed.

> never flush an old entry out of the ND.

I'd say that the router behaviour for creating a STALE entry upon
receiving an unsolicited NA should be the same as for creating an
entry for any other reason (e.g. for receiving an RS with SLLAO).
The same safety rules shall apply.

> I think that under attack

I guess your routers do purge old stale entries?

>       +It's recommended that this functionality is configurable and

Thanks, will fix in -01.

> extra paragraph to explain things.

It's just using the standard ND process: when the node B receives an
NS from node A and that NS contains the node B  address as a target
address and SLLAO contains node A LLA,
the node B would respond with NA and would create a STALE entry for
the node A - https://tools.ietf.org/html/rfc4861#section-7.2.3

> I kinda like the ping all routers trick.

I think it's a hack ;( we do have a mechanism for communicating
neighbours addresses/reachability called ND. It would be nice to
utilise its machinery.
Pinging creates additional overhead on routers and could get filtered.
But I'd not be surprised if it's the only way we have realistically to
mitigate the issue..

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 3 Jul 2019 08:13:34 +0000

Hello Michael

6LoWPAN ND is immune to the remote DOS attacks on the ND cache, the ones coming from the outside of the subnet, i.e., from a place that is out of touch and virtually nowhere.
This is because in an RFC 6775/8505-only network, there is no reactive operation, a packet coming from the outside of the subnet for a node that is not registered to the router is just dropped. Just like an AP does not copy a packet on the wireless for a MAC that is not associated.

I'm baffled that the reactive ND is still the official technique for IPv6 lookup at 6MAN. The operations that you have to do in an enterprise-class router on a data packet, punt the packet to software, run ND, program the hardware, protect ND cache and CPU against DOS, are just from another age. So we have to run our own things (e.g., a LISP MSMR) on our fabrics when modernizing ND would do the trick in a standard fashion (see draft-thubert-6lo-unicast-lookup) .

Your point below remains correct, since the attack you describe is from a node that reaches the router at L2. Arguably, that attack is physically much harder to perform than the DOS packet from outer space.

All the best,

Pascal

> -----Original Message-----
> From: 6lo <[email-masked]> On Behalf Of Michael Richardson
> Sent: mardi 2 juillet 2019 17:38
> To: [email-masked]; Jen Linkova <[email-masked]>; [email-masked]
> Cc: V6 Ops List <[email-masked]>; 6man <[email-masked]>
> Subject: Re: [6lo] ND cache entries creation on first-hop routers
> 
> 
> rather than (just) RAs, and 6LRs that need to support join operations (like
> draft-ietf-6tisch-minimal-security) this may matter.
> 
> In particular, in the minimal-security case, we need to partition the ND cache
> such that untrusted (unverified) malicious pledge nodes can not attack the ND
> cache.
> 
> The behaviour 2.2.1.  Host Sending Unsolicited NA, should probably never flush
> an old entry out of the ND.  I think that under attack (whether from untrusted
> pledges, or from p0woned devices already on the network), it is better to
> prefer communication from existing nodes rather than new ones.  2.2.1.2
> mentions this.
> 
> {typo:
>       +It's recommended that this functionality is configurable and }
> 
> I didn't really understand 2.2.2: is it exploiting some corner case in the spec, or
> maybe just some part I am not well clued in about.  So maybe an extra
> paragraph to explain things.
> 
> Michael Richardson <[email-masked]>, Sandelman Software Works  -
> = IPv6 IoT consulting =-
> 
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 3 Jul 2019 08:18:34 +0000

Hello Jen

The routers that Michael is talking about are real routers, deployed in large volumes in the field, e.g., in Smartgrid networks. 
They are not running ND as RFC 4861/4862 but as RFC 6775/8505, and then operate RPL routing within the ML subnet.
It would be great to have a section that describes that new ND operation and shows how it changes the deal. I'd be happy to help you on that if you need.
There are some hints in https://datatracker.ietf.org/doc/draft-thubert-6man-ipv6-over-wireless/.

All the best,

Pascal

> From: 6lo <[email-masked]> On Behalf Of Jen Linkova
> Sent: mercredi 3 juillet 2019 08:52
> To: Michael Richardson <[email-masked]>
> Cc: [email-masked]; 6man <[email-masked]>; V6 Ops List <[email-masked]>;
> [email-masked]
> > I think that the discussion here is particularly relevant to
> > constrained devices/routers on route-over MESH(RPL,etc.) networks.
> >
> > I also think that for L=0 networks, which RPL creates with RPL DIO
> > messages rather than (just) RAs, and 6LRs that need to support join
> > operations (like draft-ietf-6tisch-minimal-security) this may matter.
> 
> > In particular, in the minimal-security case, we need to partition the
> > ND cache such that untrusted (unverified) malicious pledge nodes can
> > not attack the ND cache.
> 
> The next version of the draft will have much more details on discussing the
> security considerations indeed.
> 
> I'd say that the router behaviour for creating a STALE entry upon receiving an
> unsolicited NA should be the same as for creating an entry for any other
> reason (e.g. for receiving an RS with SLLAO).
> > (whether from untrusted pledges, or from p0woned devices already on
> > the network), it is better to prefer communication from existing nodes
> > rather than new ones.  2.2.1.2 mentions this.
> 
> > I didn't really understand 2.2.2: is it exploiting some corner case in
> > the spec, or maybe just some part I am not well clued in about.  So
> > maybe an extra paragraph to explain things.
> 
> It's just using the standard ND process: when the node B receives an NS from
> node A and that NS contains the node B  address as a target address and
> SLLAO contains node A LLA, the node B would respond with NA and would
> create a STALE entry for the node A -
> https://tools.ietf.org/html/rfc4861#section-7.2.3
> 
> I think it's a hack ;( we do have a mechanism for communicating neighbours
> addresses/reachability called ND. It would be nice to utilise its machinery.
> But I'd not be surprised if it's the only way we have realistically to mitigate the
> issue..
> 
> >
> _______________________________________________
> 6lo mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/6lo

---

From: Lorenzo Colitti <[email-masked]>
Date: Wed, 3 Jul 2019 21:54:14 +0900

On Wed, Jul 3, 2019 at 5:14 PM Pascal Thubert (pthubert) <[email-masked]>
wrote:

> 6LoWPAN ND is immune to the remote DOS attacks on the ND cache, the ones
> coming from the outside of the subnet, i.e., from a place that is out of
> touch and virtually nowhere.
> This is because in an RFC 6775/8505-only network, there is no reactive
> operation, a packet coming from the outside of the subnet for a node that
> is not registered to the router is just dropped. Just like an AP does not
> copy a packet on the wireless for a MAC that is not associated.
>

There are problems with registration-based models as well though.

First, complexity. Recovering state in the presence of router crashes is
complex. Also, depending on what guarantees the network needs to provide to
hosts, a registration-based model will likely use more router memory in the
common case that most hosts are well-behaved (because it cannot
aggressively time out entries that with classic ND can simply be thrown
away after a while).

Second, an explicit registration model where the router can refuse to
create an address entry provides a supported path for operators to limit
the number of IP addresses used by hosts, which has the negative
consequences described in RFC 7934. In fact, such a model is explicitly NOT
RECOMMENDED by RFC 7934 for general-purpose hosts. The relevant text is "it
is RECOMMENDED that the network give the host the ability to use new
addresses without requiring explicit requests."

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 3 Jul 2019 13:39:59 +0000

Hi again Lorenzo,

First, complexity. Recovering state in the presence of router crashes is complex.

  *   True; so far we have not designed for the case where the router dies and no-one knows. In the fabric models, the router relays to a central mapping system that it can query later. The case of an intermediate network could be addressed e.g., with a RA(I’m back), an interesting subject if 6MAN takes over the work. Compare this with the case where the router crashes, loses all ND caches, and has 100s of flows through for which a full ND operation (punt, lookup, program) must be done on the first packet.

Also, depending on what guarantees the network needs to provide to hosts, a registration-based model will likely use more router memory in the common case that most hosts are well-behaved (because it cannot aggressively time out entries that with classic ND can simply be thrown away after a while).

Ø    The router can reject a registration if out of memory and the node needs to register somewhere else or free an old registration of his. At least the use of memory is deterministic. Compare this with an ND cache that has less room than the active flows, LRU will create a permanent situation of flush/lookup.

Second, an explicit registration model where the router can refuse to create an address entry provides a supported path for operators to limit the number of IP addresses used by hosts, which has the negative consequences described in RFC 7934. In fact, such a model is explicitly NOT RECOMMENDED by RFC 7934 for general-purpose hosts. The relevant text is "it is RECOMMENDED that the network give the host the ability to use new addresses without requiring explicit requests."

Ø  The router can always refrain from creating an NCE if it does not want to, e.g., by policies that protect the ND cache. The registration does not create that situation. Thanks to your contribution, RFC 8505 SHOULDs conformance to RFC 7934 and has “The ability to return errors to address registrations is not intended to be used to restrict the ability of hosts to form and use multiple addresses. “. The rest is in the hands of the network admins, make sure they deploy resources adequately.

Cheers;

Pascal

---

From: Michael Richardson <[email-masked]>
Date: Wed, 03 Jul 2019 14:59:18 -0400

> the node A - https://tools.ietf.org/html/rfc4861#section-7.2.3

I read through https://tools.ietf.org/html/rfc4861#section-7.3.3
and it says:

   The first time a node sends a packet to a neighbor whose entry is
   STALE, the sender changes the state to DELAY and sets a timer to
   expire in DELAY_FIRST_PROBE_TIME seconds.  If the entry is still in
   the DELAY state when the timer expires, the entry's state changes to
   PROBE.  If reachability confirmation is received, the entry's state
   changes to REACHABLE.

   Upon entering the PROBE state, a node sends a unicast Neighbor
   Solicitation message to the neighbor using the cached link-layer
   address.  While in the PROBE state, a node retransmits Neighbor
   Solicitation messages every RetransTimer milliseconds until
   reachability confirmation is obtained.  Probes are retransmitted even
   if no additional packets are sent to the neighbor.  If no response is
   received after waiting RetransTimer milliseconds after sending the
   MAX_UNICAST_SOLICIT solicitations, retransmissions cease and the
   entry SHOULD be deleted.  Subsequent traffic to that neighbor will
   recreate the entry and perform address resolution again.

Is PROBE and DELAY different states? It almost feels like a typo here.

    > But I'd not be surprised if it's the only way we have realistically to

Yes, it's a hack.  It leverages existing behaviour, can be deployed
unilaterally by mobile phone makers, and does not require any new state in
the end device, as they don't even have to listen for the ECHO REPLY.

Let me suggest a different hack: use stateful DHCPv6.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 03 Jul 2019 15:11:31 -0400

Pascal Thubert (pthubert) <[email-masked]> wrote:
    > 6LoWPAN ND is immune to the remote DOS attacks on the ND cache, the
    > ones coming from the outside of the subnet, i.e., from a place that is
    > out of touch and virtually nowhere.

    > operation, a packet coming from the outside of the subnet for a node
    > that is not registered to the router is just dropped. Just like an AP
    > does not copy a packet on the wireless for a MAC that is not
    > associated.

There are a few attacks on the ND cache that I can think of.
One of them that we see on the IETF network is the script kiddies who
sequentially scan IP addresses.  We have a lot of them, and so we flood the
wifi with ARP queries (v4) and NS (v6).  We have mitigations for this.

In the route-over 6tisch/RPL space, we don't (as you indicate), use NS by the
router, we know who is on our network, and we would just have no /128 routes,
and just drop the packets.  Is this the attack that you are speak of as a
remote DOS?

    > Your point below remains correct, since the attack you describe is from
    > a node that reaches the router at L2. Arguably, that attack is
    > physically much harder to perform than the DOS packet from outer
    > space.

When I mentioned attacks on the ND cache, I am referring to those that can
occur from within the 6tisch network from malicious pledge nodes.  We have to
limit the NCE usage by untrusted nodes so that we have space for as many
registered nodes.
I think you are agreeing with me above.

I believe that the issue that Jen is describing would for unaware leaves that
were sleepy.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 4 Jul 2019 08:38:35 +1200

On 03-Jul-19 20:13, Pascal Thubert (pthubert) wrote:

...
> I'm baffled that the reactive ND is still the official technique for IPv6 lookup at 6MAN.

How can it be otherwise when a node can give itself a new address at any time without notice?

I'm not arguing with you about RFC 6775/8505 networks, but that doesn't apply everywhere.

    Brian

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 3 Jul 2019 20:50:20 +0000

Hello Michael 

Please see below 

> and just drop the packets.  Is this the attack that you are speak of as a

Yes, 

Scanning in v6 can hardly be used to discover addresses but it can be used as a DOS attack against the CPU because the silicon needs to punt on cache miss. It can also be used to DOS the neighbor cache. And it is hard to differentiate from legitimate new traffic.

The possibility for such an attack comes from the reactive nature of ND, which was designed when IP routing was in software and the network a happy Woodstock. Times have changed and a modern ND should protect itself and the addresses it manages (think SeND and SAVI) and should be friendly to fabrics, to wireless and to hardware forwarders. We have that on tiny devices but it’s still missing in the big irons.

> I think you are agreeing with me above.

Yes I certainly followed you there and agree. We had that discussion for 6TiSCH minimal security. My point was that this local DOS attack requires a physical presence and is much harder to achieve than the remote DOS.

> I believe that the issue that Jen is describing would for unaware leaves that

Unaware Leaves that follow the ROLL specs register their addresses and sleep. They can sleep for the whole lifetime of their registration, the router will maintain an ND entry and the network will protect the address ownership.
To your point the node must wake up and refresh the registration before it expires.

What’s left of Jen’s issue with registration is the asymmetrical routing. In the ROLL specs the router that has the registration injects the route in the MLSN and gets the traffic back so the issue does not exist. In a transit network the router would forward the registration state to a 6LBR that can answer unicast lookups from other routers, still incurring a delay, or may push a state to other routers or to a load balancer so the router with the registration gets the traffic first.

All the best,

Pascal

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 3 Jul 2019 21:03:15 +0000

Sorry Brian I miss your point.

The discussion here is to find a way for the node to autoconf an address and then notice the router(s) proactively so the ND NCE is ready when the first packet comes from the outside for this node.

I agree wholeheartedly with Jen’s requirement.

I agree less with the idea of overstretching ND as it stands to achieve this, for reasons I gave already. So prepopulate the NCE, yes, but do it right with protocol elements that guarantee a state that is accurate, secure, and persistent for a committed lifetime, not with yet another patch to the old structure.

I’m interested to have a parallel discussion on where RFC 8505 can not apply. In the products and use cases I’m aware of, it could, since we are actually faking it by snooping ND and DHCP to achieve similar but less accurate results.

Take care,

Pascal

> Le 3 juil. 2019 à 22:39, Brian E Carpenter <[email-masked]> a écrit :
>

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 4 Jul 2019 07:40:16 +0900

> Let me suggest a different hack: use stateful DHCPv6.

DHCPv6 doesn't fix this because a) DHCP goes to a server, not the router,
b) DHCPv6 uses link-local addresses and not the global addresses, c) there
is nothing in the DHCPv6 specs about explicitly populating ND caches.

For off-link servers, the router can perform layering violations such as
"snoop DHCPv6 replies and add them to the neighbour cache". But such
layering violations don't require DHCPv6. For example, the router can also
snoop DAD packets, or glean ND cache entries from forwarded packets.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 4 Jul 2019 11:20:00 +1200

> I agree wholeheartedly with Jen’s requirement.

Yes, I am not against satisfying this requirement, but we have a considerable legacy of stacks that don't do this - so until that legacy has vanished, the routers need to deal with it somehow.

> I’m interested to have a parallel discussion on where RFC 8505 can not apply. In the products and use cases I’m aware of, it could, since we are actually faking it by snooping ND and DHCP to achiev…

So if you are advocating a generalisation of RFC8505 to non-6lo LANs, that's certainly a discussion we could have, IMHO.

Regards
    Brian

---

From: Michael Richardson <[email-masked]>
Date: Wed, 03 Jul 2019 20:57:44 -0400

Brian E Carpenter <[email-masked]> wrote:
    >> I’m interested to have a parallel discussion on where RFC 8505 can not
    >> apply. In the products and use cases I’m aware of, it could, since we
    >> are actually faking it by snooping ND and DHCP to achieve similar but
    >> less accurate results.

    > So if you are advocating a generalisation of RFC8505 to non-6lo LANs,
    > that's certainly a discussion we could have, IMHO.

I think that it could be applied in situations of servers, such as data
centers where there are multiple tenants. (Many VM infrastructures have
shared front-end networks)

I think that temporary addressess are not a feature in some of those
deployments that everyone wants, and thus having a registration system is a
feature.

This does not solve the smartphone on new WIFI issue, which is a different
situation completely.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Jen Linkova <[email-masked]>
Date: Fri, 5 Jul 2019 13:32:29 +1000

Hi Pascal,

> There are some hints in https://datatracker.ietf.org/doc/draft-thubert-6man-ipv6-over-wireless/.

Oh, that's an interesting plot twist ;) Migrating from 4861/4862 to 6775/8505?
Sounds like smth we might want to explore as a strategy (while the
tactical fixes my draft mostly talks about might be still needed too).
I'll re-read RFCs  6775/8505 and will be back to you with some
questions later..Thanks for the suggestion!