Subject: Re: [homenet] How many people have installed the homenet code?

From: Tim Coote <[email-masked]>
Date: Thu, 21 Apr 2016 13:24:33 +0100

Hullo

Sorry, this is a bit of a granular question, and this may not be the right place for it - I’m relying on the value of running code to the group ;-) . My background is s/w engineering, rather than networking, so I could well have made some fundamentally poor assumptions.

There was a discussion back in Oct 2015 about running hnetd (I presume) on mainstream linux distros (http://bit.ly/1XKc3Oi <http://bit.ly/1XKc3Oi>). Henning Rogge raised the question and Markus Stenberg responded that it had been tried on Debian 7. The discussion also referred to building hnetd on OpenWRT (http://bit.ly/1VDSg5P <http://bit.ly/1VDSg5P>) and (http://bit.ly/1VDSs4Q <http://bit.ly/1VDSs4Q>) based on using a larger distro and shncpd for a subset of the homenet protocols. The hnetd repo implies that it can be made to work with generic linux router firmware, which I took to mean any linux distro.

I think that this is an important issue, based on my experience of trying to build home automation products in the IoT space. Environments without carefully engineered networks can behave in unexpected ways, even with small numbers of end devices and realistic deployments will require significant software instrumentation and the ability to change the instrumentation to understand what is going. So, I believe, you need a full fat distro for such environments that’s responsible, say, for handling 6LowPAN devices, abstraction of such devices for use by other services, and managing their lifecycles.

So I tried to spin hnetd up on a fedora vm, and found it fighting the distro set-up. Maybe the implementation isn’t supposed to co-exist with other network controlling software on the same computer. What’s my best approach to getting the homenet protocols running as a proof of concept? I cannot find a ‘top down’ view of how the different components fit together - the descriptions seem to assume a depth of knowledge of openwrt that I don’t have.

tia
Tim

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 21 Apr 2016 16:00:01 +0200

> So I tried to spin hnetd up on a fedora vm, and found it fighting the
> distro set-up. Maybe the implementation isn’t supposed to co-exist with
> other network controlling software on the same computer. What’s my best
> approach to getting the homenet protocols running as a proof of concept?

Hnetd is designed to integrate into the OpenWRT ecosystem, which it does
remarkably well.  Because of that, it depends on a large part of the
OpenWRT stack, such as their DHCP and RA clients and servers.  While it is
possible to run hnetd

This was the second most important reason for writing shncpd: to make an
implementation of Homenet that is easy to port to any Unix system.
I think I've been reasonably successful with that, but shncpd needs some
more work [1].

[1] https://www.irif.univ-paris-diderot.fr/~jch/software/homenet/shncpd.html
    has a list of known limitations.

> So I tried to spin hnetd up on a fedora vm, and found it fighting the
> distro set-up.

Even if you're just testing plain babeld, even without hnetd, you must
kill network manager.  If you don't, thinkgs may appear to work at first,
but NM will take revenge later.

-- Juliusz

---

From: Tim Coote <[email-masked]>
Date: Thu, 21 Apr 2016 16:30:10 +0100

Thanks.
> On 21 Apr 2016, at 15:00, Juliusz Chroboczek <[email-masked]> wrote:
> 
It’s the IPv6 stuff that I’m most interested in - IoT with ipv4 (NAT) is too expensive to support at scale. At least for the retail market. My sense was that the api to the openwrt tools was a likely stumbling block.  I did get the babeld stuff working (thanks for what you’ve done there), but gave up with tools like dibbler for PD as the challenge is the coordination across nodes.
> but NM will take revenge later.
I hated NM. But I think that it’s getting better. It does provide an initial overall visualisation of what’s going on. Something will be required to show the overall picture across all digital entities involved in a system, including ‘phones with both wifi and LTE connections. For the consumer there needs to be at least a green/red light (cf a CD pipeline status), but finer granularity would also have value.  I would hope to encourage the NM community to at least take cognisance of the Homenet protocols.

At the complexity that is emerging in homes, a configuration defined by a gui will not scale: too hard to get to a known state, especially with >1 node.

Maybe I need to create a separate openwrt vm if I cannot run the hnetd stack on a more complete distro.
> — Juliusz
Tim

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 21 Apr 2016 17:59:25 +0200

> It’s the IPv6 stuff that I’m most interested in - IoT with ipv4 (NAT) is
> too expensive to support at scale. At least for the retail market. My
> sense was that the api to the openwrt tools was a likely stumbling
> block.

Tim, please try shncpd and let me know how it goes.  I'm quite willing to
add features, but I'm waiting for a user community to emerge so I know
what features people actually care about.

-- Juliusz

---

From: Markus Stenberg <[email-masked]>
Date: Thu, 21 Apr 2016 20:32:57 +0300

While we have tested all hnetd versions on Debian stable as well, I cannot recommend it as there is no configuration or real integration with the system beyond "edit/create shell scripts". 

Obviously anything else automated on the associated interfaces has to be disabled as Juliusz noted. 

Disclaimer: I do not find Linux distribution integration fun so OpenWrt is enough for me. Of course I am for sale given enough money ;) ( consulting mainly on some other stuff at the moment though )

-Markus

Sent from my preciouss, aka iPad ;)

> On 21.4.2016, at 15.24, Tim Coote <[email-masked]> wrote:
> 
> Hullo
> 
> There was a discussion back in Oct 2015 about running hnetd (I presume) on mainstream linux distros (http://bit.ly/1XKc3Oi). Henning Rogge raised the question and Markus Stenberg responded that it had been tried on Debian 7. The discussion also referred to building hnetd on OpenWRT (http://bit.ly/1VDSg5P) and (http://bit.ly/1VDSs4Q) based on using a larger distro and shncpd for a subset of the homenet protocols. The hnetd repo implies that it can be made to work with generic linux router firmware, which I took to mean any linux distro.
> 
> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet

---

From: Tore Anderson <[email-masked]>
Date: Thu, 21 Apr 2016 20:33:21 +0200

Hello,

* Tim Coote

> > Even if you're just testing plain babeld, even without hnetd, you
> > must kill network manager.  If you don't, thinkgs may appear to
> > work at first, but NM will take revenge later.  

> I hated NM. But I think that it’s getting better. It does provide an
> initial overall visualisation of what’s going on. Something will be
> required to show the overall picture across all digital entities
> involved in a system, including ‘phones with both wifi and LTE
> connections. For the consumer there needs to be at least a green/red
> light (cf a CD pipeline status), but finer granularity would also
> have value.  I would hope to encourage the NM community to at least
> take cognisance of the Homenet protocols.

NM is being very actively developed, and in my experience its
developers are generally very receptive to bug reports and other
requests. So I'd encourage anyone who have issues with NM to join the
IRC channel and/or reports bugs - chances are they get fixed quickly.

As it happens, NM version 1.2 vas released yesterday. One of the things
that has been improved is NM's ability to stay away from interfaces
that's being managed by something else, so maybe an upgrade would be
enough to fix Juliusz's problems. If not, try adding the following to
NetworkManager.conf:

[main]
no-auto-default=*

Another thing IPv6 folks might be interested in knowing is that it now
defaults to using RFC7217 IID generation instead of EUI-64.

> hnetd stack on a more complete distro.

An quicker and more lightweight solution is to isolate hnet inside a
network namespace:

ip netns add hnet-playground
ip link set dev eth1 netns hnet-playground
ip netns exec hnet-playground shncpd --start --if eth1 # or whatever
ip netns exec hnet-playground /bin/bash
# [...do stuff inside the hnet playground here...]

Tore

---

From: Tim Coote <[email-masked]>
Date: Fri, 22 Apr 2016 15:54:59 +0100

> On 21 Apr 2016, at 18:32, Markus Stenberg <[email-masked]> wrote:
> 
Thanks, Markus. That’s what I’d concluded. I didn’t want to try to unpick the openwrt tools and integrate with larger distros tools, which sort of knocks on the head Tore’s suggestion of using namespaces as I’d still have the porting problem. Building a separate vm for openwrt seems like some complexity that I could do without at the moment, too, although I can see value in separating the network configuration from the application configuration.

I’ll start with Juliusz’ suggestion.

tc

---

From: Tim Coote <[email-masked]>
Date: Sat, 23 Apr 2016 14:31:18 +0100

> On 21 Apr 2016, at 16:59, Juliusz Chroboczek <[email-masked]> wrote:
> 

Ok. I’m starting by running shncpd on a boundary router and tried a trivial installation. I can see that RAs are generated and picked up as expected. (when I remembered to turn off the various radvd instances...

I don’t see how dns gets updated. Are such updates out of scope of shncpd? Are they in scope for the other homenet protocols?

tim

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Apr 2016 18:39:08 +0200

> I’m starting by running shncpd on a boundary router and tried a trivial
> installation.

Excellent, thanks.

> I don’t see how dns gets updated. Are such updates out of scope of
> shncpd?

Do you mean, (1) how is a DNS resolver advertised to clients, or
(2) how clients are registered in DNS ?

(1) is done by using the -N flag on the router advertising an external
connection (-E).  This flag can be repeated multiple times.

(2) is a host issue, so I believe it is better handled outside of shncpd,
but I'm quite willing to be convinced otherwise.  (The obvious alternative
would be to have shncpd update DNS when it gives out a DHCP lease, but
that would mean giving up on stateless autoconf.)

> Are they in scope for the other homenet protocols?

Markus, Steven, Ted?  What's the plan here?  Do we count on mDNS proxying,
or should we be advertising an RFC 2136 server over HNCP?

-- Juliusz

---

From: Markus Stenberg <[email-masked]>
Date: Sat, 23 Apr 2016 20:08:49 +0300

> connection (-E).  This flag can be repeated multiple times.

hnetd grabs this automatically from wan-facing DHCP client, but again, requires integration to work nicely (odhcp6c glue script supported on most Linux platforms). Clients on the other hand are given always just first-hop router address.

> that would mean giving up on stateless autoconf.)

Well, DHCPv4 is stateful anyway, and you could in theory bind state from there as well (at least if you do IPv4).

> or should we be advertising an RFC 2136 server over HNCP?

I think the plan varies ;-)

hnetd (and current HNCP + my expired autoconf draft) are based on the idea of using mDNS _and/or stateful DHCPv4 and/or stateful DHCPv6 to determine what’s on each link, and advertise the routers responsible for each link across the homenet. It does work with ‘current stuff’ for some relatively high % of likelihood.

Ted’s draft proposes either learn-from-mDNS (=proxy DNS-update) and/or (manually/automatically configured client-sourced) DNS-update scheme. I am worried about zone merging + conflict resolution, although if it works out it sounds like a good solution. 

(Zone merging + plain hybrid proxy is at least very problematic, if you want hybrid proxies to remain stateless. I have looked at it and it is neither pretty nor efficient.)

Cheers,

-Markus

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Apr 2016 19:35:27 +0200

> hnetd grabs this automatically from wan-facing DHCP client,

Yep.  Shncpd might learn to act as a DHCPv4 and DHCPv6-PD client at some
point.  (But first implement IPv4 edge router election, which is one of
the three missing features that I feel are necessary, security aside.)

> there as well (at least if you do IPv4).

Yes.  I'm not sure how shncpd should update the state, though, since it
doesn't act as a DNS resolver, it only passes the address of an existing
DNS resolver to clients, and I don't want to bind it to any specific
implementation of a DNS resolver.

> I think the plan varies ;-)

Yeah, that's what I gathered.

> hnetd (and current HNCP + my expired autoconf draft) are based on the
> idea of using mDNS _and/or stateful DHCPv4 and/or stateful DHCPv6 to
> determine what’s on each link,

Quoting Stephen:

   I agree with you as well, initially odhcpd did only do RAs and
   stateless DHCPv6 however the user community liked their stateful
   (v4isms) all too much so I had to give in. ;)

   -- https://github.com/sbyx/odhcpd/issues/55

> Ted’s draft proposes either learn-from-mDNS (=proxy DNS-update) and/or
> (manually/automatically configured client-sourced) DNS-update
> scheme. I am worried about zone merging + conflict resolution, although
> if it works out it sounds like a good solution.

> (Zone merging + plain hybrid proxy is at least very problematic, if you
> want hybrid proxies to remain stateless. I have looked at it and it is
> neither pretty nor efficient.)

I haven't looked at it, but my instinct would be to agree with you --
proxying in the presence of merging sounds like a can of worms that we
don't want to open.

Do any of the Wise People on this list have opinions about RFC 2136?

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Sat, 23 Apr 2016 23:03:29 -0400

Juliusz, the problem is that existing home network devices that do
DNS-based service discovery do not support DNS update.   They could, but
they don't, because we didn't define an easy way for them to do it.   Just
2136 isn't enough, because there's no authentication scheme, and name
servers typically don't take updates from devices, nor is the default
domain on a typical home network actually even under the control of the
owner of that network.

The reason for doing mdns snooping is that we have no choice.   It's not a
great solution.   However, I believe it can be done in a way that is clean
and works.   It will not be stateless, although whether the state needs to
be persistent is an interesting question.  mDNS isn't actually entirely
stateless anyway, FWIW.

If you think this is a can of worms you'd rather not open, I can understand
that, but Stuart and I have had some pretty good conversations about this,
and I remain convinced that we can make it work, so I'd encourage you to
see what comes out of that process rather than assuming that the situation
is hopeless.

---

From: Markus Stenberg <[email-masked]>
Date: Sun, 24 Apr 2016 09:18:42 +0300

On 24.4.2016, at 6.03, Ted Lemon <[email-masked]> wrote:
> Juliusz, the problem is that existing home network devices that do DNS-based service discovery do not support DNS update.   They could, but they don't, because we didn't define an easy way for them to do it.   Just 2136 isn't enough, because there's no authentication scheme, and name servers typically don't take updates from devices, nor is the default domain on a typical home network actually even under the control of the owner of that network.
> 
> The reason for doing mdns snooping is that we have no choice.   It's not a great solution.   However, I believe it can be done in a way that is clean and works.   It will not be stateless, although whether the state needs to be persistent is an interesting question.  mDNS isn’t actually entirely stateless anyway, FWIW.

mDNS has only locally published state (and non-local state caching which can be ignored if you do not care about query efficiency but you really should), and as hybrid proxies do not publish local state, they can be implemented statelessly. 
 
For home network case, probably just the DNS-SD legacy browse (‘flat names, hide domain’ scheme) with per-link hybrid proxy zones would work fine; as Stuart said, mDNS state caching (if implemented by the hybrid proxies) would make it relatively efficient (in terms of multicast; there would still be N unicasts per lookup where N is the number of links=zones, which would not be cached, but 0 multicasts if the mDNS caches on the hybrid proxies are up to date). As added bonus, no bogus domains would be shown to users, and you would just have two ‘printer’s if you were silly enough not to give them better names and they lived on different links.

> If you think this is a can of worms you’d rather not open, I can understand that, but Stuart and I have had some pretty good conversations about this, and I remain convinced that we can make it work, so I'd encourage you to see what comes out of that process rather than assuming that the situation is hopeless.

I am a Finn, we are pessimistic by default. But I am looking forward to what you come up with ;) 

If we want to get really pessimistic about this whole thing, though, it seems to me that the market is going for ‘cloudy’ solutions to anything, where there is no p2p communication _even on the same network_ but instead some cloud server intermediary, which when unavailable/expired/having a bad hair day, means you are simply not doing what you wanted to do.

Cheers,

-Markus

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 24 Apr 2016 18:29:46 +0200

> they don't, because we didn't define an easy way for them to do it.

I'd be grateful if you could expand on that.  Why can't we define a way
for clients to do DDNS?  (I'm asking for technical reasons, not
administrative reasons, I'm aware that this would probably be outside the
Homenet charter.)

> Just 2136 isn't enfough, because there's no authentication scheme,

I don't understand this argument.  How is non-secured DDNS any less secure
than mDNS?  What am I missing?

> If you think this is a can of worms you'd rather not open, I can
> understand that, but Stuart and I have had some pretty good conversations
> about this, and I remain convinced that we can make it work,
> so I'd encourage you to see what comes out of that process rather than
> assuming that the situation is hopeless.

Oh, sure, we Poles are not quite as pessimistic as the Finns.  I'm
actually of a divided mind here -- I rather like distributed solutions
(hence prefer mDNS to DDNS) but dislike proxying.  Part of me just wishes
we'd mandate site-local multicast and do mDNS over that.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Sun, 24 Apr 2016 22:39:27 -0400

On Sun, Apr 24, 2016 at 12:29 PM, Juliusz Chroboczek <
[email-masked]> wrote:

> for clients to do DDNS?

We can and should.   The problem is that we won't see that code ship in new
devices anytime soon, so we still have to make mDNS work.

> than mDNS?  What am I missing?

This is an implementation issue, not a security issue--sorry for not making
that clear.   In order to preserve the same security characteristics that
mDNS has, we have to ensure that the update actually originated on the
local link, which requires a different sort of listener than is present in
a typical DNS server.   And existing DNS servers typically don't have any
way to support unauthenticated updates on a first-come, first-served basis,
so if you allow unauthenticated updates, you don't have any way to avoid
collisions.   Otherwise you are correct.   The answer is to write a
document that describes how to do that, and if you read the homenet naming
arch document, you can see that I actually sketched out a solution there,
which I expect to go in a different document, likely in a different working
group.

> we'd mandate site-local multicast and do mDNS over that

The problem with site-local multicast for mDNS is that multicast isn't a
great solution even on the local wire when that wire is wireless.    And,
you have to do modify the client anyway.

Furthermore, if you consider the mdns hybrid proxy stateless, then you can
have a DNS server that is roughly that stateless too.   I think it provides
better service continuity if you are willing to retain some state, but
everything will still work even if you don't, just as the hybrid proxy does.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 25 Apr 2016 07:48:33 +0200

> a typical DNS server.

Makes perfect sense, thanks for the explanation.

> great solution even on the local wire when that wire is wireless.

Mmh, if you're an IGMP listener, then you have the lists of subscribers,
so you could in principle convert multicast to multiple link-layer
unicasts.  But that's perhaps somewhat more of a layering violation than
I'm comfortable with.  So yeah, your point stands.

> better service continuity if you are willing to retain some state

Mmh, okay, I'll think it over.  But I reserve the right to grumble.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Mon, 25 Apr 2016 08:21:59 -0400

> Mmh, okay, I'll think it over.  But I reserve the right to grumble.

Noted. :)

---

From: Tim Coote <[email-masked]>
Date: Mon, 25 Apr 2016 20:50:01 +0100

> On 23 Apr 2016, at 17:39, Juliusz Chroboczek <[email-masked]> wrote:
Well that went well and created what I hope have been useful interactions. 

So my next question is ‘what is External and what is Internal?’ How is this topolgical aspect identified and the ‘shared, agreed topology’ communicated between routing entities (I’m eschewing the term router here, to emphasise for my own benefit that routers aren’t necessarily on separate pieces of hardware - and in consumer IoT, I’m expecting the combination to be increasingly common), and how do they update the flags on the shncpd instances that have been started? Or is it expected that the user nominates the topology and it’s static?

I’ve seen such topologies shift over varying timescales (seconds to years). Sometimes the change is a fault. Sometimes it’s expected. Both usecases are important. I think that I read in the homenet documentation that directed graph topologies are anticipated, but I’m not clear how. It may be that they’re either scoped out, or future work.

Sorry if that’s a bit of a neophyte’s question.
Tim

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 25 Apr 2016 22:34:43 +0200

> So my next question is ‘what is External and what is Internal?’

  - External: connected to the ISP's (non-Homenet) router.

  - Internal: interior link within your Homenet, connected to other
    Homenet routers and to (unmodified) hosts.

Here's a possible topology, where interfaces are marked "e" for External
and umarked for Internal.  A, B, C and D are Homenet routers, "+" are
(layer 2) switches.

                    B -- Host
                    |
                    |
            e       |       e
  ISP A ----- A ----+---- D ----- ISP B
                \       /
                 \     /
                  \   /--Host
                    C
                    |
                    |
            Host ---+--- Host

On an Internal link, we speak HNCP and Babel.  On an External link, we
speak neither HNCP nor Babel.

(The OpenWRT implementation additionally sets static routing, NAT and
firewall rules on External links, but shncpd doesn't.)

> How is this topolgical aspect identified and the ‘shared, agreed
> topology’ communicated between routing entities

The difference between External and Internal is defined statically -- the
expectation is that you have a dedicated Ethernet port to your ISP, which
is statically configured as External.  A useful mental model is that of
a router with one Ethernet port marked Wan being External, and four
Ethernet ports marked LAN being Internal.

(The OpenWRT implementation has the ability to automatically detect
External ports, but it relies on acting as a DHCPv6-PD and DHCPv4 client,
so it's not implemented in shncpd yet.)

> Or is it expected that the user nominates the topology and it’s static?

HNCP and Babel only run on Internal ports, so only the Internal topology
is detected automatically.  You can add and remove Internal cables at
will, and this will be detected within a few seconds, but you cannot
switch a port between Internal and External without reconfiguration.

> I think that I read in the homenet documentation that directed graph
> topologies are anticipated,

We don't require an acyclic graph (see figure above).  Arbitrary Internal
topologies are supported, and HNCP and Babel cooperate to do The Right Thing.

-- Juliusz